2024-05-09T04:19:40.656 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-09T04:19:40.656 DEBUG:teuthology.run:Teuthology command: teuthology --name vshankar-2024-05-09_04:13:07-fs-wip-63259-3-testing-default-smithi --verbose --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/backtrace} --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2024-05-09_04:13:07-fs-wip-63259-3-testing-default-smithi/7699554 -- /home/teuthworker/archive/vshankar-2024-05-09_04:13:07-fs-wip-63259-3-testing-default-smithi/7699554/orig.config.yaml 2024-05-09T04:19:40.700 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-09T04:19:40.783 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2024-05-09_04:13:07-fs-wip-63259-3-testing-default-smithi/7699554 branch: wip-63259-3 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/backtrace} email: vshankar@redhat.com first_in_suite: false job_id: '7699554' kernel: flavor: default kdb: true sha1: 806ec36b21129f113ce356c007dd59694d340b37 last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: vshankar-2024-05-09_04:13:07-fs-wip-63259-3-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-63259-3 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy sha1: 165a15e51d08073b5317c443415b0d8a7141b1ab ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 165a15e51d08073b5317c443415b0d8a7141b1ab selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-63259-3 sha1: 165a15e51d08073b5317c443415b0d8a7141b1ab owner: scheduled_vshankar@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 3813 sha1: 165a15e51d08073b5317c443415b0d8a7141b1ab sleep_before_teardown: 0 subset: 31/1024 suite: fs suite_branch: wip-63259-3 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_165a15e51d08073b5317c443415b0d8a7141b1ab/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 165a15e51d08073b5317c443415b0d8a7141b1ab syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi145.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLbQwlhmHtv3FPtQ4lWSaMnaJDWcYWkvMgaLuyzLEAV9V+L8YHnFXnK/VG8fxHcr0xO4Ji0IGJpz/N3gwrufNIU= smithi192.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKFyOmXPPT42NFJaLMrEgmPLkAxUtASgsUI6YWYHHTb3wl+Jzf5CH15lRCG5CrSf9NjCEFwjvdapjhI71FidzDU= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_backtrace teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-09_04:13:07 tube: smithi user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-09T04:19:40.783 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_165a15e51d08073b5317c443415b0d8a7141b1ab/qa; will attempt to use it 2024-05-09T04:19:40.784 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_165a15e51d08073b5317c443415b0d8a7141b1ab/qa/tasks 2024-05-09T04:19:40.784 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-09T04:19:40.785 INFO:teuthology.task.internal:Checking packages... 2024-05-09T04:19:40.810 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '165a15e51d08073b5317c443415b0d8a7141b1ab' 2024-05-09T04:19:40.810 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-09T04:19:40.810 INFO:teuthology.packaging:ref: None 2024-05-09T04:19:40.810 INFO:teuthology.packaging:tag: None 2024-05-09T04:19:40.810 INFO:teuthology.packaging:branch: wip-63259-3 2024-05-09T04:19:40.810 INFO:teuthology.packaging:sha1: 165a15e51d08073b5317c443415b0d8a7141b1ab 2024-05-09T04:19:40.810 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-63259-3 2024-05-09T04:19:40.955 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3568.g165a15e5 2024-05-09T04:19:40.956 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-09T04:19:40.965 INFO:teuthology.task.internal:no buildpackages task found 2024-05-09T04:19:40.965 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-09T04:19:41.013 INFO:teuthology.task.internal:Saving configuration 2024-05-09T04:19:41.029 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-09T04:19:41.046 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-09T04:19:41.069 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi145.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-05-09_04:13:07-fs-wip-63259-3-testing-default-smithi/7699554', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-09 04:14:34.661578', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLbQwlhmHtv3FPtQ4lWSaMnaJDWcYWkvMgaLuyzLEAV9V+L8YHnFXnK/VG8fxHcr0xO4Ji0IGJpz/N3gwrufNIU='} 2024-05-09T04:19:41.093 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi192.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-05-09_04:13:07-fs-wip-63259-3-testing-default-smithi/7699554', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-09 04:14:34.660115', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKFyOmXPPT42NFJaLMrEgmPLkAxUtASgsUI6YWYHHTb3wl+Jzf5CH15lRCG5CrSf9NjCEFwjvdapjhI71FidzDU='} 2024-05-09T04:19:41.093 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-09T04:19:41.100 INFO:teuthology.task.internal:roles: ubuntu@smithi145.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-05-09T04:19:41.101 INFO:teuthology.task.internal:roles: ubuntu@smithi192.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-05-09T04:19:41.101 INFO:teuthology.run_tasks:Running task console_log... 2024-05-09T04:19:41.194 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f87efd52f70>, signals=[15]) 2024-05-09T04:19:41.195 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-09T04:19:41.203 INFO:teuthology.task.internal:Opening connections... 2024-05-09T04:19:41.203 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi145.front.sepia.ceph.com 2024-05-09T04:19:41.207 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T04:19:41.286 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi192.front.sepia.ceph.com 2024-05-09T04:19:41.286 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T04:19:41.360 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-09T04:19:41.368 DEBUG:teuthology.orchestra.run.smithi145:> uname -m 2024-05-09T04:19:41.385 INFO:teuthology.orchestra.run.smithi145.stdout:x86_64 2024-05-09T04:19:41.386 DEBUG:teuthology.orchestra.run.smithi145:> cat /etc/os-release 2024-05-09T04:19:41.443 INFO:teuthology.orchestra.run.smithi145.stdout:NAME="CentOS Stream" 2024-05-09T04:19:41.443 INFO:teuthology.orchestra.run.smithi145.stdout:VERSION="9" 2024-05-09T04:19:41.443 INFO:teuthology.orchestra.run.smithi145.stdout:ID="centos" 2024-05-09T04:19:41.443 INFO:teuthology.orchestra.run.smithi145.stdout:ID_LIKE="rhel fedora" 2024-05-09T04:19:41.443 INFO:teuthology.orchestra.run.smithi145.stdout:VERSION_ID="9" 2024-05-09T04:19:41.443 INFO:teuthology.orchestra.run.smithi145.stdout:PLATFORM_ID="platform:el9" 2024-05-09T04:19:41.443 INFO:teuthology.orchestra.run.smithi145.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-09T04:19:41.443 INFO:teuthology.orchestra.run.smithi145.stdout:ANSI_COLOR="0;31" 2024-05-09T04:19:41.443 INFO:teuthology.orchestra.run.smithi145.stdout:LOGO="fedora-logo-icon" 2024-05-09T04:19:41.443 INFO:teuthology.orchestra.run.smithi145.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-09T04:19:41.443 INFO:teuthology.orchestra.run.smithi145.stdout:HOME_URL="https://centos.org/" 2024-05-09T04:19:41.444 INFO:teuthology.orchestra.run.smithi145.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-09T04:19:41.444 INFO:teuthology.orchestra.run.smithi145.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-09T04:19:41.444 INFO:teuthology.orchestra.run.smithi145.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-09T04:19:41.444 INFO:teuthology.lock.ops:Updating smithi145.front.sepia.ceph.com on lock server 2024-05-09T04:19:41.465 DEBUG:teuthology.orchestra.run.smithi192:> uname -m 2024-05-09T04:19:41.483 INFO:teuthology.orchestra.run.smithi192.stdout:x86_64 2024-05-09T04:19:41.483 DEBUG:teuthology.orchestra.run.smithi192:> cat /etc/os-release 2024-05-09T04:19:41.541 INFO:teuthology.orchestra.run.smithi192.stdout:NAME="CentOS Stream" 2024-05-09T04:19:41.541 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION="9" 2024-05-09T04:19:41.541 INFO:teuthology.orchestra.run.smithi192.stdout:ID="centos" 2024-05-09T04:19:41.541 INFO:teuthology.orchestra.run.smithi192.stdout:ID_LIKE="rhel fedora" 2024-05-09T04:19:41.541 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION_ID="9" 2024-05-09T04:19:41.541 INFO:teuthology.orchestra.run.smithi192.stdout:PLATFORM_ID="platform:el9" 2024-05-09T04:19:41.541 INFO:teuthology.orchestra.run.smithi192.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-09T04:19:41.541 INFO:teuthology.orchestra.run.smithi192.stdout:ANSI_COLOR="0;31" 2024-05-09T04:19:41.541 INFO:teuthology.orchestra.run.smithi192.stdout:LOGO="fedora-logo-icon" 2024-05-09T04:19:41.541 INFO:teuthology.orchestra.run.smithi192.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-09T04:19:41.541 INFO:teuthology.orchestra.run.smithi192.stdout:HOME_URL="https://centos.org/" 2024-05-09T04:19:41.541 INFO:teuthology.orchestra.run.smithi192.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-09T04:19:41.541 INFO:teuthology.orchestra.run.smithi192.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-09T04:19:41.542 INFO:teuthology.orchestra.run.smithi192.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-09T04:19:41.542 INFO:teuthology.lock.ops:Updating smithi192.front.sepia.ceph.com on lock server 2024-05-09T04:19:41.556 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-09T04:19:41.566 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-09T04:19:41.603 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-09T04:19:41.603 DEBUG:teuthology.orchestra.run.smithi145:> test '!' -e /home/ubuntu/cephtest 2024-05-09T04:19:41.606 DEBUG:teuthology.orchestra.run.smithi192:> test '!' -e /home/ubuntu/cephtest 2024-05-09T04:19:41.622 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-09T04:19:41.635 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-09T04:19:41.636 DEBUG:teuthology.orchestra.run.smithi145:> test -z $(ls -A /var/lib/ceph) 2024-05-09T04:19:41.663 DEBUG:teuthology.orchestra.run.smithi192:> test -z $(ls -A /var/lib/ceph) 2024-05-09T04:19:41.734 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-09T04:19:41.787 INFO:teuthology.run_tasks:Running task kernel... 2024-05-09T04:19:41.801 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'} 2024-05-09T04:19:41.801 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}}, timeout 300 2024-05-09T04:19:41.801 DEBUG:teuthology.orchestra.run.smithi145:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-09T04:19:41.802 DEBUG:teuthology.orchestra.run.smithi192:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-09T04:19:41.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T04:19:41.819 DEBUG:teuthology.task.kernel:sha1 for mon.a is 806ec36b21129f113ce356c007dd59694d340b37 2024-05-09T04:19:41.819 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-05-09T04:19:41.819 DEBUG:teuthology.orchestra.run.smithi145:> uname -r 2024-05-09T04:19:41.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T04:19:41.820 DEBUG:teuthology.task.kernel:sha1 for mon.b is 806ec36b21129f113ce356c007dd59694d340b37 2024-05-09T04:19:41.820 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-05-09T04:19:41.820 DEBUG:teuthology.orchestra.run.smithi192:> uname -r 2024-05-09T04:19:41.876 INFO:teuthology.orchestra.run.smithi145.stdout:5.14.0-437.el9.x86_64 2024-05-09T04:19:41.876 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-05-09T04:19:41.876 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-05-09T04:19:41.876 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-05-09T04:19:41.880 INFO:teuthology.orchestra.run.smithi192.stdout:5.14.0-437.el9.x86_64 2024-05-09T04:19:41.880 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-05-09T04:19:41.880 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-05-09T04:19:41.881 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-05-09T04:19:42.019 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum upgrade -y linux-firmware 2024-05-09T04:19:42.043 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum upgrade -y linux-firmware 2024-05-09T04:19:45.088 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS Stream 9 - BaseOS 5.8 MB/s | 8.1 MB 00:01 2024-05-09T04:19:46.164 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - BaseOS 3.2 MB/s | 8.1 MB 00:02 2024-05-09T04:19:48.589 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS Stream 9 - AppStream 7.9 MB/s | 19 MB 00:02 2024-05-09T04:19:48.940 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - AppStream 11 MB/s | 19 MB 00:01 2024-05-09T04:19:53.014 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS Stream 9 - CRB 7.8 MB/s | 6.2 MB 00:00 2024-05-09T04:19:53.649 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - CRB 6.3 MB/s | 6.2 MB 00:00 2024-05-09T04:19:54.440 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS Stream 9 - Extras packages 63 kB/s | 16 kB 00:00 2024-05-09T04:19:55.075 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - Extras packages 74 kB/s | 16 kB 00:00 2024-05-09T04:19:57.118 INFO:teuthology.orchestra.run.smithi192.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-05-09T04:20:02.276 INFO:teuthology.orchestra.run.smithi192.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-05-09T04:20:04.406 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-05-09T04:20:04.407 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-05-09T04:20:04.407 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-05-09T04:20:04.518 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.b... 2024-05-09T04:20:04.518 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-05-09T04:20:04.678 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-05-09T04:20:04.679 DEBUG:teuthology.orchestra.run.smithi192:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-05-09T04:20:04.683 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi192.front.sepia.ceph.com to complete... 2024-05-09T04:20:19.432 INFO:teuthology.orchestra.run.smithi145.stdout:Extra Packages for Enterprise Linux 871 kB/s | 21 MB 00:24 2024-05-09T04:20:24.596 INFO:teuthology.orchestra.run.smithi145.stdout:lab-extras 10 kB/s | 1.7 kB 00:00 2024-05-09T04:20:26.654 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2024-05-09T04:20:26.655 INFO:teuthology.orchestra.run.smithi145.stdout:Nothing to do. 2024-05-09T04:20:26.655 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2024-05-09T04:20:26.762 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.a... 2024-05-09T04:20:26.762 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-05-09T04:20:27.099 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-05-09T04:20:27.099 DEBUG:teuthology.orchestra.run.smithi145:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-05-09T04:20:27.101 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi145.front.sepia.ceph.com to complete... 2024-05-09T04:21:30.501 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09 04:21:30 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-05-09T04:21:30.502 INFO:teuthology.orchestra.run.smithi192.stderr:FINISHED --2024-05-09 04:21:30-- 2024-05-09T04:21:30.502 INFO:teuthology.orchestra.run.smithi192.stderr:Total wall clock time: 1m 26s 2024-05-09T04:21:30.502 INFO:teuthology.orchestra.run.smithi192.stderr:Downloaded: 1 files, 341M in 1m 25s (4.01 MB/s) 2024-05-09T04:21:30.504 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.b... 2024-05-09T04:21:30.505 DEBUG:teuthology.orchestra.run.smithi192:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-05-09T04:21:32.601 INFO:teuthology.orchestra.run.smithi192.stdout:Verifying... ######################################## 2024-05-09T04:21:32.734 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing... ######################################## 2024-05-09T04:21:32.850 INFO:teuthology.orchestra.run.smithi192.stdout:Updating / installing... 2024-05-09T04:21:40.864 INFO:teuthology.orchestra.run.smithi192.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-05-09T04:21:44.346 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09 04:21:44 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-05-09T04:21:44.347 INFO:teuthology.orchestra.run.smithi145.stderr:FINISHED --2024-05-09 04:21:44-- 2024-05-09T04:21:44.347 INFO:teuthology.orchestra.run.smithi145.stderr:Total wall clock time: 1m 17s 2024-05-09T04:21:44.347 INFO:teuthology.orchestra.run.smithi145.stderr:Downloaded: 1 files, 341M in 1m 17s (4.44 MB/s) 2024-05-09T04:21:44.350 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.a... 2024-05-09T04:21:44.350 DEBUG:teuthology.orchestra.run.smithi145:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-05-09T04:21:46.458 INFO:teuthology.orchestra.run.smithi145.stdout:Verifying... ######################################## 2024-05-09T04:21:46.675 INFO:teuthology.orchestra.run.smithi145.stdout:Preparing... ######################################## 2024-05-09T04:21:46.781 INFO:teuthology.orchestra.run.smithi145.stdout:Updating / installing... 2024-05-09T04:21:54.774 INFO:teuthology.orchestra.run.smithi145.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-05-09T04:22:18.733 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi192.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-05-09T04:22:18.733 DEBUG:teuthology.orchestra.run.smithi192:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-05-09T04:22:18.810 INFO:teuthology.orchestra.run.smithi192.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-05-09T04:22:18.810 INFO:teuthology.orchestra.run.smithi192.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-05-09T04:22:18.811 INFO:teuthology.orchestra.run.smithi192.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-05-09T04:22:18.811 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-05-09T04:22:18.811 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-05-09T04:22:18.811 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-05-09T04:22:18.811 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-05-09T04:22:18.811 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-05-09T04:22:18.811 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-05-09T04:22:18.811 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-05-09T04:22:18.811 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-05-09T04:22:18.811 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-05-09T04:22:18.811 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-05-09T04:22:18.811 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-05-09T04:22:18.811 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-05-09T04:22:18.811 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-05-09T04:22:18.811 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-05-09T04:22:18.812 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-05-09T04:22:18.812 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-05-09T04:22:18.812 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-05-09T04:22:18.812 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-05-09T04:22:18.812 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-05-09T04:22:18.812 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-05-09T04:22:18.812 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-05-09T04:22:18.812 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-05-09T04:22:18.812 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-05-09T04:22:18.812 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-05-09T04:22:18.812 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-05-09T04:22:18.812 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-05-09T04:22:18.812 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-05-09T04:22:18.812 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-05-09T04:22:18.812 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-05-09T04:22:18.813 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-05-09T04:22:18.813 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-05-09T04:22:18.813 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-05-09T04:22:18.813 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-05-09T04:22:18.813 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-05-09T04:22:18.813 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-05-09T04:22:18.813 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-05-09T04:22:18.813 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-05-09T04:22:18.813 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-05-09T04:22:18.813 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-05-09T04:22:18.813 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-05-09T04:22:18.813 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-05-09T04:22:18.813 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-05-09T04:22:18.813 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-05-09T04:22:18.813 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-05-09T04:22:18.814 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-05-09T04:22:18.814 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-05-09T04:22:18.814 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-05-09T04:22:18.814 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-05-09T04:22:18.814 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-05-09T04:22:18.814 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-05-09T04:22:18.814 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-05-09T04:22:18.814 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-05-09T04:22:18.814 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-05-09T04:22:18.814 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-05-09T04:22:18.814 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-05-09T04:22:18.814 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-05-09T04:22:18.814 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-05-09T04:22:18.814 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-05-09T04:22:18.815 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-05-09T04:22:18.815 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-05-09T04:22:18.815 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-05-09T04:22:18.815 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-05-09T04:22:18.815 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-05-09T04:22:18.815 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-05-09T04:22:18.815 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-05-09T04:22:18.815 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-05-09T04:22:18.815 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-05-09T04:22:18.815 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-05-09T04:22:18.815 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-05-09T04:22:18.816 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-05-09T04:22:18.816 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-05-09T04:22:18.816 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-05-09T04:22:18.816 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-05-09T04:22:18.816 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-05-09T04:22:18.816 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-05-09T04:22:18.816 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-05-09T04:22:18.816 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-05-09T04:22:18.816 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-05-09T04:22:18.816 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-05-09T04:22:18.816 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-05-09T04:22:18.816 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-05-09T04:22:18.816 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-05-09T04:22:18.816 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-05-09T04:22:18.816 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-05-09T04:22:18.817 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-05-09T04:22:18.817 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-05-09T04:22:18.817 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-05-09T04:22:18.817 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-05-09T04:22:18.817 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-05-09T04:22:18.817 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-05-09T04:22:18.817 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-05-09T04:22:18.817 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-05-09T04:22:18.817 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-05-09T04:22:18.817 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-05-09T04:22:18.817 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-05-09T04:22:18.817 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-05-09T04:22:18.817 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-05-09T04:22:18.817 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-05-09T04:22:18.817 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-05-09T04:22:18.818 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-05-09T04:22:18.818 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-05-09T04:22:18.818 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-05-09T04:22:18.818 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-05-09T04:22:18.818 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-05-09T04:22:18.818 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-05-09T04:22:18.818 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-05-09T04:22:18.818 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-05-09T04:22:18.818 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-05-09T04:22:18.818 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-05-09T04:22:18.818 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-05-09T04:22:18.818 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-05-09T04:22:18.818 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-05-09T04:22:18.818 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-05-09T04:22:18.819 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-05-09T04:22:18.819 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-05-09T04:22:18.819 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-05-09T04:22:18.819 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-05-09T04:22:18.819 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-05-09T04:22:18.819 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-05-09T04:22:18.819 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-05-09T04:22:18.819 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-05-09T04:22:18.819 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-05-09T04:22:18.819 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-05-09T04:22:18.819 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-05-09T04:22:18.819 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-05-09T04:22:18.819 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-05-09T04:22:18.819 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-05-09T04:22:18.819 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-05-09T04:22:18.820 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-05-09T04:22:18.820 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-05-09T04:22:18.820 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-05-09T04:22:18.820 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-05-09T04:22:18.820 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-05-09T04:22:18.820 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-05-09T04:22:18.820 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-05-09T04:22:18.820 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-05-09T04:22:18.820 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-05-09T04:22:18.820 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-05-09T04:22:18.820 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-05-09T04:22:18.820 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-05-09T04:22:18.820 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-05-09T04:22:18.820 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-05-09T04:22:18.820 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-05-09T04:22:18.821 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-05-09T04:22:18.821 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-05-09T04:22:18.821 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-05-09T04:22:18.821 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-05-09T04:22:18.821 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-05-09T04:22:18.821 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-05-09T04:22:18.821 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-05-09T04:22:18.821 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-05-09T04:22:18.821 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-05-09T04:22:18.821 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-05-09T04:22:18.821 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-05-09T04:22:18.821 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-05-09T04:22:18.821 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-05-09T04:22:18.821 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-05-09T04:22:18.821 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-05-09T04:22:18.821 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-05-09T04:22:18.822 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-05-09T04:22:18.822 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-05-09T04:22:18.822 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-05-09T04:22:18.822 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-05-09T04:22:18.822 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-05-09T04:22:18.822 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-05-09T04:22:18.822 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-05-09T04:22:18.822 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-05-09T04:22:18.822 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-05-09T04:22:18.822 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-05-09T04:22:18.822 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-05-09T04:22:18.822 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-05-09T04:22:18.822 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-05-09T04:22:18.823 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-05-09T04:22:18.823 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-05-09T04:22:18.823 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-05-09T04:22:18.823 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-05-09T04:22:18.823 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-05-09T04:22:18.823 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-05-09T04:22:18.823 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-05-09T04:22:18.823 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-05-09T04:22:18.823 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-05-09T04:22:18.823 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-05-09T04:22:18.823 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-05-09T04:22:18.823 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-05-09T04:22:18.823 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-05-09T04:22:18.823 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-05-09T04:22:18.823 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-05-09T04:22:18.823 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-05-09T04:22:18.824 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-05-09T04:22:18.824 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-05-09T04:22:18.824 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-05-09T04:22:18.824 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-05-09T04:22:18.824 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-05-09T04:22:18.824 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-05-09T04:22:18.824 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-05-09T04:22:18.824 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-05-09T04:22:18.824 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-05-09T04:22:18.824 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-05-09T04:22:18.824 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-05-09T04:22:18.825 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-05-09T04:22:18.825 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-05-09T04:22:18.825 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-05-09T04:22:18.825 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-05-09T04:22:18.825 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-05-09T04:22:18.825 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-05-09T04:22:18.825 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-05-09T04:22:18.825 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-05-09T04:22:18.825 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-05-09T04:22:18.825 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-05-09T04:22:18.825 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-05-09T04:22:18.825 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-05-09T04:22:18.825 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-05-09T04:22:18.825 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-05-09T04:22:18.825 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-05-09T04:22:18.825 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-05-09T04:22:18.826 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-05-09T04:22:18.826 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-05-09T04:22:18.826 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-05-09T04:22:18.826 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-05-09T04:22:18.826 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-05-09T04:22:18.826 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-05-09T04:22:18.826 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-05-09T04:22:18.826 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-05-09T04:22:18.826 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-05-09T04:22:18.826 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-05-09T04:22:18.826 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-05-09T04:22:18.826 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-05-09T04:22:18.826 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-05-09T04:22:18.826 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-05-09T04:22:18.827 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-05-09T04:22:18.827 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-05-09T04:22:18.827 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-05-09T04:22:18.827 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-05-09T04:22:18.827 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-05-09T04:22:18.827 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-05-09T04:22:18.827 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-05-09T04:22:18.827 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-05-09T04:22:18.827 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-05-09T04:22:18.827 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-05-09T04:22:18.827 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-05-09T04:22:18.827 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-05-09T04:22:18.827 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-05-09T04:22:18.827 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-05-09T04:22:18.827 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-05-09T04:22:18.828 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-05-09T04:22:18.828 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-05-09T04:22:18.828 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-05-09T04:22:18.828 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-05-09T04:22:18.828 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-05-09T04:22:18.828 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-05-09T04:22:18.828 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-05-09T04:22:18.828 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-05-09T04:22:18.828 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-05-09T04:22:18.828 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-05-09T04:22:18.828 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-05-09T04:22:18.828 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-05-09T04:22:18.828 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-05-09T04:22:18.828 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-05-09T04:22:18.828 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-05-09T04:22:18.829 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-05-09T04:22:18.829 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-05-09T04:22:18.829 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-05-09T04:22:18.829 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-05-09T04:22:18.829 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-05-09T04:22:18.829 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-05-09T04:22:18.829 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-05-09T04:22:18.829 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-05-09T04:22:18.829 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-05-09T04:22:18.829 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-05-09T04:22:18.829 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-05-09T04:22:18.829 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-05-09T04:22:18.829 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-05-09T04:22:18.829 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-05-09T04:22:18.829 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-05-09T04:22:18.829 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-05-09T04:22:18.830 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-05-09T04:22:18.830 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-05-09T04:22:18.830 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-05-09T04:22:18.830 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-05-09T04:22:18.830 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-05-09T04:22:18.830 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-05-09T04:22:18.830 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-05-09T04:22:18.830 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-05-09T04:22:18.830 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-05-09T04:22:18.830 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-05-09T04:22:18.830 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-05-09T04:22:18.830 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-05-09T04:22:18.830 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-05-09T04:22:18.830 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-05-09T04:22:18.830 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-05-09T04:22:18.831 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-05-09T04:22:18.831 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-05-09T04:22:18.831 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-05-09T04:22:18.831 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-05-09T04:22:18.831 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-05-09T04:22:18.831 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-05-09T04:22:18.831 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-05-09T04:22:18.831 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-05-09T04:22:18.831 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-05-09T04:22:18.831 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-05-09T04:22:18.831 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-05-09T04:22:18.831 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-05-09T04:22:18.831 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-05-09T04:22:18.831 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-05-09T04:22:18.831 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-05-09T04:22:18.832 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-05-09T04:22:18.832 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-05-09T04:22:18.832 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-05-09T04:22:18.832 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-05-09T04:22:18.832 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-05-09T04:22:18.832 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-05-09T04:22:18.832 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-05-09T04:22:18.832 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-05-09T04:22:18.832 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-05-09T04:22:18.832 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-05-09T04:22:18.832 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-05-09T04:22:18.832 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-05-09T04:22:18.832 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-05-09T04:22:18.832 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-05-09T04:22:18.832 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-05-09T04:22:18.832 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-05-09T04:22:18.832 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-05-09T04:22:18.833 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-05-09T04:22:18.833 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-05-09T04:22:18.833 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-05-09T04:22:18.833 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-05-09T04:22:18.833 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-05-09T04:22:18.833 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-05-09T04:22:18.833 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-05-09T04:22:18.833 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-05-09T04:22:18.833 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-05-09T04:22:18.833 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-05-09T04:22:18.833 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-05-09T04:22:18.833 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-05-09T04:22:18.833 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-05-09T04:22:18.833 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-05-09T04:22:18.833 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-05-09T04:22:18.833 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-05-09T04:22:18.833 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-05-09T04:22:18.834 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-05-09T04:22:18.834 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-05-09T04:22:18.834 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-05-09T04:22:18.834 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-05-09T04:22:18.834 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-05-09T04:22:18.834 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-05-09T04:22:18.834 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-05-09T04:22:18.834 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-05-09T04:22:18.834 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-05-09T04:22:18.834 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-05-09T04:22:18.834 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-05-09T04:22:18.834 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-05-09T04:22:18.834 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-05-09T04:22:18.834 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-05-09T04:22:18.835 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-05-09T04:22:18.835 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-05-09T04:22:18.835 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-05-09T04:22:18.835 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-05-09T04:22:18.835 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-05-09T04:22:18.835 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-05-09T04:22:18.835 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-05-09T04:22:18.835 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-05-09T04:22:18.835 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-05-09T04:22:18.835 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-05-09T04:22:18.835 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-05-09T04:22:18.835 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-05-09T04:22:18.835 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-05-09T04:22:18.835 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-05-09T04:22:18.835 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-05-09T04:22:18.835 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-05-09T04:22:18.835 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-05-09T04:22:18.836 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-05-09T04:22:18.836 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-05-09T04:22:18.836 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-05-09T04:22:18.836 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-05-09T04:22:18.836 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-05-09T04:22:18.836 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-05-09T04:22:18.836 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-05-09T04:22:18.836 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-05-09T04:22:18.836 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-05-09T04:22:18.836 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-05-09T04:22:18.836 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-05-09T04:22:18.836 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-05-09T04:22:18.836 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-05-09T04:22:18.836 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-05-09T04:22:18.836 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-05-09T04:22:18.836 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-05-09T04:22:18.837 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-05-09T04:22:18.837 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-05-09T04:22:18.837 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-05-09T04:22:18.837 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-05-09T04:22:18.837 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-05-09T04:22:18.837 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-05-09T04:22:18.837 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-05-09T04:22:18.837 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-05-09T04:22:18.837 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-05-09T04:22:18.837 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-05-09T04:22:18.837 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-05-09T04:22:18.837 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-05-09T04:22:18.837 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-05-09T04:22:18.837 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-05-09T04:22:18.837 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-05-09T04:22:18.838 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-05-09T04:22:18.838 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-05-09T04:22:18.838 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-05-09T04:22:18.838 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-05-09T04:22:18.838 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-05-09T04:22:18.838 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-05-09T04:22:18.838 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-05-09T04:22:18.838 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-05-09T04:22:18.838 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-05-09T04:22:18.838 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-05-09T04:22:18.838 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-05-09T04:22:18.838 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-05-09T04:22:18.838 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-05-09T04:22:18.838 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-05-09T04:22:18.838 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-05-09T04:22:18.839 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-05-09T04:22:18.839 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-05-09T04:22:18.839 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-05-09T04:22:18.839 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-05-09T04:22:18.839 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-05-09T04:22:18.839 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-05-09T04:22:18.839 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-05-09T04:22:18.839 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-05-09T04:22:18.839 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-05-09T04:22:18.839 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-05-09T04:22:18.839 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-05-09T04:22:18.839 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-05-09T04:22:18.839 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-05-09T04:22:18.839 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-05-09T04:22:18.839 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-05-09T04:22:18.839 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-05-09T04:22:18.840 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-05-09T04:22:18.840 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-05-09T04:22:18.840 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-05-09T04:22:18.840 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-05-09T04:22:18.840 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-05-09T04:22:18.840 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-05-09T04:22:18.840 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-05-09T04:22:18.840 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-05-09T04:22:18.840 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-05-09T04:22:18.840 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-05-09T04:22:18.840 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-05-09T04:22:18.840 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-05-09T04:22:18.840 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-05-09T04:22:18.840 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-05-09T04:22:18.840 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-05-09T04:22:18.840 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-05-09T04:22:18.841 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-05-09T04:22:18.841 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-05-09T04:22:18.841 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-05-09T04:22:18.841 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-05-09T04:22:18.841 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-05-09T04:22:18.841 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-05-09T04:22:18.841 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-05-09T04:22:18.841 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-05-09T04:22:18.841 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-05-09T04:22:18.841 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-05-09T04:22:18.841 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-05-09T04:22:18.841 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-05-09T04:22:18.841 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-05-09T04:22:18.841 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-05-09T04:22:18.841 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-05-09T04:22:18.841 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-05-09T04:22:18.842 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-05-09T04:22:18.842 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-05-09T04:22:18.842 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-05-09T04:22:18.842 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-05-09T04:22:18.842 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-05-09T04:22:18.842 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-05-09T04:22:18.842 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-05-09T04:22:18.842 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-05-09T04:22:18.842 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-05-09T04:22:18.842 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-05-09T04:22:18.842 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-05-09T04:22:18.842 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-05-09T04:22:18.842 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-05-09T04:22:18.842 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-05-09T04:22:18.843 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-05-09T04:22:18.843 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-05-09T04:22:18.843 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-05-09T04:22:18.843 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-05-09T04:22:18.843 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-05-09T04:22:18.843 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-05-09T04:22:18.843 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-05-09T04:22:18.843 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-05-09T04:22:18.843 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-05-09T04:22:18.843 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-05-09T04:22:18.843 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-05-09T04:22:18.843 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-05-09T04:22:18.843 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-05-09T04:22:18.843 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-05-09T04:22:18.843 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-05-09T04:22:18.844 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-05-09T04:22:18.844 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-05-09T04:22:18.844 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-05-09T04:22:18.844 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-05-09T04:22:18.844 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-05-09T04:22:18.844 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-05-09T04:22:18.844 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-05-09T04:22:18.844 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-05-09T04:22:18.844 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-05-09T04:22:18.844 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-05-09T04:22:18.844 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-05-09T04:22:18.844 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-05-09T04:22:18.844 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-05-09T04:22:18.844 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-05-09T04:22:18.845 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-05-09T04:22:18.845 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-05-09T04:22:18.845 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-05-09T04:22:18.845 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-05-09T04:22:18.845 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-05-09T04:22:18.845 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-05-09T04:22:18.845 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-05-09T04:22:18.845 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-05-09T04:22:18.845 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-05-09T04:22:18.845 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-05-09T04:22:18.845 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-05-09T04:22:18.845 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-05-09T04:22:18.845 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-05-09T04:22:18.845 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-05-09T04:22:18.845 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-05-09T04:22:18.845 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-05-09T04:22:18.846 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-05-09T04:22:18.846 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-05-09T04:22:18.846 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-05-09T04:22:18.846 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-05-09T04:22:18.846 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-05-09T04:22:18.846 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-05-09T04:22:18.846 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-05-09T04:22:18.846 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-05-09T04:22:18.846 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-05-09T04:22:18.846 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-05-09T04:22:18.846 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-05-09T04:22:18.846 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-05-09T04:22:18.846 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-05-09T04:22:18.846 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-05-09T04:22:18.846 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-05-09T04:22:18.847 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-05-09T04:22:18.847 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-05-09T04:22:18.847 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-05-09T04:22:18.847 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-05-09T04:22:18.847 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-05-09T04:22:18.847 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-05-09T04:22:18.847 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-05-09T04:22:18.847 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-05-09T04:22:18.847 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-05-09T04:22:18.847 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-05-09T04:22:18.847 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-05-09T04:22:18.847 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-05-09T04:22:18.847 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-05-09T04:22:18.847 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-05-09T04:22:18.847 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-05-09T04:22:18.847 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-05-09T04:22:18.848 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-05-09T04:22:18.848 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-05-09T04:22:18.848 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-05-09T04:22:18.848 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-05-09T04:22:18.848 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-05-09T04:22:18.848 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-05-09T04:22:18.848 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-05-09T04:22:18.848 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-05-09T04:22:18.848 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-05-09T04:22:18.848 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-05-09T04:22:18.848 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-05-09T04:22:18.848 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-05-09T04:22:18.848 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-05-09T04:22:18.848 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-05-09T04:22:18.848 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-05-09T04:22:18.849 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-05-09T04:22:18.849 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-05-09T04:22:18.849 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-05-09T04:22:18.849 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-05-09T04:22:18.849 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-05-09T04:22:18.849 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-05-09T04:22:18.849 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-05-09T04:22:18.849 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-05-09T04:22:18.849 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-05-09T04:22:18.849 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-05-09T04:22:18.849 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-05-09T04:22:18.849 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-05-09T04:22:18.849 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-05-09T04:22:18.849 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-05-09T04:22:18.850 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-05-09T04:22:18.850 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-05-09T04:22:18.850 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-05-09T04:22:18.850 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-05-09T04:22:18.850 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-05-09T04:22:18.850 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-05-09T04:22:18.850 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-05-09T04:22:18.850 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-05-09T04:22:18.850 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-05-09T04:22:18.850 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-05-09T04:22:18.850 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-05-09T04:22:18.853 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-05-09T04:22:18.853 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-05-09T04:22:18.854 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-05-09T04:22:18.854 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-05-09T04:22:18.854 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-05-09T04:22:18.854 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-05-09T04:22:18.854 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-05-09T04:22:18.854 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-05-09T04:22:18.854 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-05-09T04:22:18.854 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-05-09T04:22:18.854 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-05-09T04:22:18.854 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-05-09T04:22:18.854 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-05-09T04:22:18.854 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-05-09T04:22:18.854 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-05-09T04:22:18.854 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-05-09T04:22:18.854 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-05-09T04:22:18.855 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-05-09T04:22:18.855 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-05-09T04:22:18.855 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-05-09T04:22:18.855 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-05-09T04:22:18.855 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-05-09T04:22:18.855 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-05-09T04:22:18.855 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-05-09T04:22:18.855 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-05-09T04:22:18.855 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-05-09T04:22:18.855 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-05-09T04:22:18.855 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-05-09T04:22:18.855 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-05-09T04:22:18.855 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-05-09T04:22:18.855 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-05-09T04:22:18.855 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-05-09T04:22:18.856 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-05-09T04:22:18.856 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-05-09T04:22:18.856 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-05-09T04:22:18.856 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-05-09T04:22:18.856 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-05-09T04:22:18.856 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-05-09T04:22:18.856 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-05-09T04:22:18.856 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-05-09T04:22:18.856 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-05-09T04:22:18.856 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-05-09T04:22:18.856 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-05-09T04:22:18.856 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-05-09T04:22:18.856 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-05-09T04:22:18.856 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-05-09T04:22:18.856 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-05-09T04:22:18.857 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-05-09T04:22:18.857 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-05-09T04:22:18.857 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-05-09T04:22:18.857 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-05-09T04:22:18.857 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-05-09T04:22:18.857 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-05-09T04:22:18.857 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-05-09T04:22:18.857 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-05-09T04:22:18.857 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-05-09T04:22:18.857 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-05-09T04:22:18.857 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-05-09T04:22:18.857 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-05-09T04:22:18.857 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-05-09T04:22:18.857 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-05-09T04:22:18.857 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-05-09T04:22:18.858 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-05-09T04:22:18.858 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-05-09T04:22:18.858 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-05-09T04:22:18.858 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-05-09T04:22:18.858 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-05-09T04:22:18.858 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-05-09T04:22:18.858 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-05-09T04:22:18.858 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-05-09T04:22:18.858 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-05-09T04:22:18.858 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-05-09T04:22:18.858 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-05-09T04:22:18.858 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-05-09T04:22:18.858 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-05-09T04:22:18.858 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-05-09T04:22:18.858 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-05-09T04:22:18.859 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-05-09T04:22:18.859 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-05-09T04:22:18.859 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-05-09T04:22:18.859 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-05-09T04:22:18.859 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-05-09T04:22:18.859 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-05-09T04:22:18.859 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-05-09T04:22:18.859 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-05-09T04:22:18.859 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-05-09T04:22:18.859 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-05-09T04:22:18.859 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-05-09T04:22:18.859 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-05-09T04:22:18.859 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-05-09T04:22:18.859 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-05-09T04:22:18.859 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-05-09T04:22:18.859 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-05-09T04:22:18.860 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-05-09T04:22:18.860 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-05-09T04:22:18.860 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-05-09T04:22:18.860 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-05-09T04:22:18.860 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-05-09T04:22:18.860 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-05-09T04:22:18.860 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-05-09T04:22:18.860 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-05-09T04:22:18.860 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-05-09T04:22:18.860 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-05-09T04:22:18.860 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-05-09T04:22:18.860 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-05-09T04:22:18.860 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-05-09T04:22:18.860 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-05-09T04:22:18.861 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-05-09T04:22:18.861 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-05-09T04:22:18.861 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-05-09T04:22:18.861 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-05-09T04:22:18.861 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-05-09T04:22:18.861 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-05-09T04:22:18.861 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-05-09T04:22:18.861 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-05-09T04:22:18.861 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-05-09T04:22:18.861 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-05-09T04:22:18.861 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-05-09T04:22:18.861 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-05-09T04:22:18.861 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-05-09T04:22:18.861 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-05-09T04:22:18.861 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-05-09T04:22:18.862 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-05-09T04:22:18.862 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-05-09T04:22:18.862 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-05-09T04:22:18.862 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-05-09T04:22:18.862 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-05-09T04:22:18.862 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-05-09T04:22:18.862 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-05-09T04:22:18.862 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-05-09T04:22:18.862 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-05-09T04:22:18.862 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-05-09T04:22:18.862 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-05-09T04:22:18.862 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-05-09T04:22:18.862 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-05-09T04:22:18.862 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-05-09T04:22:18.862 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-05-09T04:22:18.863 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-05-09T04:22:18.863 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-05-09T04:22:18.863 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-05-09T04:22:18.863 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-05-09T04:22:18.863 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-05-09T04:22:18.863 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-05-09T04:22:18.863 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-05-09T04:22:18.863 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-05-09T04:22:18.863 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-05-09T04:22:18.863 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-05-09T04:22:18.863 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-05-09T04:22:18.863 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-05-09T04:22:18.863 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-05-09T04:22:18.863 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-05-09T04:22:18.863 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-05-09T04:22:18.864 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-05-09T04:22:18.864 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-05-09T04:22:18.864 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-05-09T04:22:18.864 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-05-09T04:22:18.864 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-05-09T04:22:18.864 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-05-09T04:22:18.864 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-05-09T04:22:18.864 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-05-09T04:22:18.864 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-05-09T04:22:18.864 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-05-09T04:22:18.864 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-05-09T04:22:18.864 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-05-09T04:22:18.864 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-05-09T04:22:18.864 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-05-09T04:22:18.864 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-05-09T04:22:18.865 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-05-09T04:22:18.865 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-05-09T04:22:18.865 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-05-09T04:22:18.865 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-05-09T04:22:18.865 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-05-09T04:22:18.865 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-05-09T04:22:18.865 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-05-09T04:22:18.865 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-05-09T04:22:18.865 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-05-09T04:22:18.865 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-05-09T04:22:18.865 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-05-09T04:22:18.865 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-05-09T04:22:18.865 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-05-09T04:22:18.865 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-05-09T04:22:18.865 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-05-09T04:22:18.865 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-05-09T04:22:18.866 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-05-09T04:22:18.866 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-05-09T04:22:18.866 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-05-09T04:22:18.866 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-05-09T04:22:18.866 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-05-09T04:22:18.866 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-05-09T04:22:18.866 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-05-09T04:22:18.866 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-05-09T04:22:18.866 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-05-09T04:22:18.866 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-05-09T04:22:18.866 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-05-09T04:22:18.866 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-05-09T04:22:18.866 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-05-09T04:22:18.866 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-05-09T04:22:18.867 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-05-09T04:22:18.867 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-05-09T04:22:18.867 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-05-09T04:22:18.867 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-05-09T04:22:18.867 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-05-09T04:22:18.867 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-05-09T04:22:18.867 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-05-09T04:22:18.867 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-05-09T04:22:18.867 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-05-09T04:22:18.867 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-05-09T04:22:18.867 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-05-09T04:22:18.867 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-05-09T04:22:18.867 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-05-09T04:22:18.867 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-05-09T04:22:18.867 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-05-09T04:22:18.868 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-05-09T04:22:18.868 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-05-09T04:22:18.868 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-05-09T04:22:18.868 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-05-09T04:22:18.868 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-05-09T04:22:18.868 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-05-09T04:22:18.868 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-05-09T04:22:18.868 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-05-09T04:22:18.868 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-05-09T04:22:18.868 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-05-09T04:22:18.868 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-05-09T04:22:18.868 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-05-09T04:22:18.868 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-05-09T04:22:18.868 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-05-09T04:22:18.868 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-05-09T04:22:18.868 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-05-09T04:22:18.869 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-05-09T04:22:18.869 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-05-09T04:22:18.869 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-05-09T04:22:18.869 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-05-09T04:22:18.869 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-05-09T04:22:18.869 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-05-09T04:22:18.869 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-05-09T04:22:18.869 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-05-09T04:22:18.869 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-05-09T04:22:18.869 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-05-09T04:22:18.869 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-05-09T04:22:18.869 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-05-09T04:22:18.869 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-05-09T04:22:18.869 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-05-09T04:22:18.869 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-05-09T04:22:18.870 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-05-09T04:22:18.870 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-05-09T04:22:18.870 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-05-09T04:22:18.870 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-05-09T04:22:18.870 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-05-09T04:22:18.870 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-05-09T04:22:18.870 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-05-09T04:22:18.870 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-05-09T04:22:18.870 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-05-09T04:22:18.870 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-05-09T04:22:18.870 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-05-09T04:22:18.870 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-05-09T04:22:18.870 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-05-09T04:22:18.870 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-05-09T04:22:18.870 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-05-09T04:22:18.871 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-05-09T04:22:18.871 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-05-09T04:22:18.871 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-05-09T04:22:18.871 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-05-09T04:22:18.871 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-05-09T04:22:18.871 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-05-09T04:22:18.871 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-05-09T04:22:18.871 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-05-09T04:22:18.871 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-05-09T04:22:18.871 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-05-09T04:22:18.871 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-05-09T04:22:18.871 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-05-09T04:22:18.871 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-05-09T04:22:18.871 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-05-09T04:22:18.871 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-05-09T04:22:18.872 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-05-09T04:22:18.872 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-05-09T04:22:18.872 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-05-09T04:22:18.872 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-05-09T04:22:18.872 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-05-09T04:22:18.872 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-05-09T04:22:18.872 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-05-09T04:22:18.872 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-05-09T04:22:18.872 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-05-09T04:22:18.872 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-05-09T04:22:18.872 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-05-09T04:22:18.872 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-05-09T04:22:18.872 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-05-09T04:22:18.872 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-05-09T04:22:18.872 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-05-09T04:22:18.872 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-05-09T04:22:18.873 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-05-09T04:22:18.873 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-05-09T04:22:18.873 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-05-09T04:22:18.873 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-05-09T04:22:18.873 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-05-09T04:22:18.873 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-05-09T04:22:18.873 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-05-09T04:22:18.873 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-05-09T04:22:18.873 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-05-09T04:22:18.873 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-05-09T04:22:18.873 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-05-09T04:22:18.873 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-05-09T04:22:18.873 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-05-09T04:22:18.873 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-05-09T04:22:18.873 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-05-09T04:22:18.874 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-05-09T04:22:18.874 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-05-09T04:22:18.874 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-05-09T04:22:18.874 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-05-09T04:22:18.874 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-05-09T04:22:18.874 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-05-09T04:22:18.874 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-05-09T04:22:18.874 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-05-09T04:22:18.874 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-05-09T04:22:18.874 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-05-09T04:22:18.874 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-05-09T04:22:18.874 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-05-09T04:22:18.874 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-05-09T04:22:18.875 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-05-09T04:22:18.875 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-05-09T04:22:18.875 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-05-09T04:22:18.875 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-05-09T04:22:18.875 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-05-09T04:22:18.875 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-05-09T04:22:18.875 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-05-09T04:22:18.875 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-05-09T04:22:18.875 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-05-09T04:22:18.875 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-05-09T04:22:18.875 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-05-09T04:22:18.875 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-05-09T04:22:18.875 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-05-09T04:22:18.875 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-05-09T04:22:18.875 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-05-09T04:22:18.876 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-05-09T04:22:18.876 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-05-09T04:22:18.876 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-05-09T04:22:18.876 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-05-09T04:22:18.876 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-05-09T04:22:18.876 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-05-09T04:22:18.876 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-05-09T04:22:18.876 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-05-09T04:22:18.876 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-05-09T04:22:18.876 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-05-09T04:22:18.876 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-05-09T04:22:18.876 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-05-09T04:22:18.876 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-05-09T04:22:18.876 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-05-09T04:22:18.876 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-05-09T04:22:18.876 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-05-09T04:22:18.877 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-05-09T04:22:18.877 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-05-09T04:22:18.877 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-05-09T04:22:18.877 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-05-09T04:22:18.877 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-05-09T04:22:18.877 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-05-09T04:22:18.877 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-05-09T04:22:18.877 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-05-09T04:22:18.877 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-05-09T04:22:18.877 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-05-09T04:22:18.877 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-05-09T04:22:18.877 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-05-09T04:22:18.877 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-05-09T04:22:18.877 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-05-09T04:22:18.877 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-05-09T04:22:18.877 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-05-09T04:22:18.878 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-05-09T04:22:18.878 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-05-09T04:22:18.878 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-05-09T04:22:18.878 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-05-09T04:22:18.878 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-05-09T04:22:18.878 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-05-09T04:22:18.878 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-05-09T04:22:18.878 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-05-09T04:22:18.878 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-05-09T04:22:18.878 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-05-09T04:22:18.878 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-05-09T04:22:18.878 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-05-09T04:22:18.878 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-05-09T04:22:18.878 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-05-09T04:22:18.878 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-05-09T04:22:18.879 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-05-09T04:22:18.879 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-05-09T04:22:18.879 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-05-09T04:22:18.879 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-05-09T04:22:18.879 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-05-09T04:22:18.879 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-05-09T04:22:18.879 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-05-09T04:22:18.879 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-05-09T04:22:18.879 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-05-09T04:22:18.879 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-05-09T04:22:18.879 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-05-09T04:22:18.879 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-05-09T04:22:18.879 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-05-09T04:22:18.879 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-05-09T04:22:18.879 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-05-09T04:22:18.880 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-05-09T04:22:18.880 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-05-09T04:22:18.880 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-05-09T04:22:18.880 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-05-09T04:22:18.880 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-05-09T04:22:18.880 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-05-09T04:22:18.880 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-05-09T04:22:18.880 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-05-09T04:22:18.880 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-05-09T04:22:18.880 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-05-09T04:22:18.880 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-05-09T04:22:18.880 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-05-09T04:22:18.880 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-05-09T04:22:18.880 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-05-09T04:22:18.880 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-05-09T04:22:18.881 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-05-09T04:22:18.881 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-05-09T04:22:18.881 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-05-09T04:22:18.881 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-05-09T04:22:18.881 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-05-09T04:22:18.881 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-05-09T04:22:18.881 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-05-09T04:22:18.881 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-05-09T04:22:18.881 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-05-09T04:22:18.881 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-05-09T04:22:18.881 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-05-09T04:22:18.881 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-05-09T04:22:18.881 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-05-09T04:22:18.881 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-05-09T04:22:18.881 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-05-09T04:22:18.881 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-05-09T04:22:18.882 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-05-09T04:22:18.882 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-05-09T04:22:18.882 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-05-09T04:22:18.882 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-05-09T04:22:18.882 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-05-09T04:22:18.882 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-05-09T04:22:18.882 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-05-09T04:22:18.882 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-05-09T04:22:18.882 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-05-09T04:22:18.882 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-05-09T04:22:18.882 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-05-09T04:22:18.882 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-05-09T04:22:18.882 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-05-09T04:22:18.882 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-05-09T04:22:18.883 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-05-09T04:22:18.883 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-05-09T04:22:18.883 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-05-09T04:22:18.883 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-05-09T04:22:18.883 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-05-09T04:22:18.883 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-05-09T04:22:18.883 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-05-09T04:22:18.883 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-05-09T04:22:18.883 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-05-09T04:22:18.883 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-05-09T04:22:18.883 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-05-09T04:22:18.883 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-05-09T04:22:18.883 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-05-09T04:22:18.883 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-05-09T04:22:18.883 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-05-09T04:22:18.884 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-05-09T04:22:18.884 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-05-09T04:22:18.884 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-05-09T04:22:18.884 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-05-09T04:22:18.884 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-05-09T04:22:18.884 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-05-09T04:22:18.884 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-05-09T04:22:18.884 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-05-09T04:22:18.884 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-05-09T04:22:18.884 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-05-09T04:22:18.884 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-05-09T04:22:18.884 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-05-09T04:22:18.884 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-05-09T04:22:18.884 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-05-09T04:22:18.884 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-05-09T04:22:18.885 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-05-09T04:22:18.885 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-05-09T04:22:18.885 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-05-09T04:22:18.885 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-05-09T04:22:18.885 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-05-09T04:22:18.885 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-05-09T04:22:18.885 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-05-09T04:22:18.885 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-05-09T04:22:18.885 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-05-09T04:22:18.885 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-05-09T04:22:18.885 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-05-09T04:22:18.885 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-05-09T04:22:18.885 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-05-09T04:22:18.885 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-05-09T04:22:18.885 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-05-09T04:22:18.886 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-05-09T04:22:18.886 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-05-09T04:22:18.886 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-05-09T04:22:18.886 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-05-09T04:22:18.886 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-05-09T04:22:18.886 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-05-09T04:22:18.886 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-05-09T04:22:18.886 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-05-09T04:22:18.886 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-05-09T04:22:18.886 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-05-09T04:22:18.886 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-05-09T04:22:18.886 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-05-09T04:22:18.886 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-05-09T04:22:18.886 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-05-09T04:22:18.887 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-05-09T04:22:18.887 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-05-09T04:22:18.887 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-05-09T04:22:18.887 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-05-09T04:22:18.887 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-05-09T04:22:18.887 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-05-09T04:22:18.887 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-05-09T04:22:18.887 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-05-09T04:22:18.887 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-05-09T04:22:18.887 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-05-09T04:22:18.887 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-05-09T04:22:18.887 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-05-09T04:22:18.887 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-05-09T04:22:18.887 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-05-09T04:22:18.887 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-05-09T04:22:18.887 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-05-09T04:22:18.888 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-05-09T04:22:18.888 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-05-09T04:22:18.888 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-05-09T04:22:18.888 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-05-09T04:22:18.888 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-05-09T04:22:18.888 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-05-09T04:22:18.888 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-05-09T04:22:18.888 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-05-09T04:22:18.888 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-05-09T04:22:18.888 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-05-09T04:22:18.888 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-05-09T04:22:18.888 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-05-09T04:22:18.888 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-05-09T04:22:18.888 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-05-09T04:22:18.888 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-05-09T04:22:18.889 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-05-09T04:22:18.889 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-05-09T04:22:18.889 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-05-09T04:22:18.889 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-05-09T04:22:18.889 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-05-09T04:22:18.889 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-05-09T04:22:18.889 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-05-09T04:22:18.889 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-05-09T04:22:18.889 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-05-09T04:22:18.889 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-05-09T04:22:18.889 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-05-09T04:22:18.889 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-05-09T04:22:18.889 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-05-09T04:22:18.889 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-05-09T04:22:18.890 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-05-09T04:22:18.890 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-05-09T04:22:18.890 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-05-09T04:22:18.890 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-05-09T04:22:18.890 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-05-09T04:22:18.890 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-05-09T04:22:18.890 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-05-09T04:22:18.890 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-05-09T04:22:18.890 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-05-09T04:22:18.890 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-05-09T04:22:18.890 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-05-09T04:22:18.890 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-05-09T04:22:18.890 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-05-09T04:22:18.890 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-05-09T04:22:18.891 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-05-09T04:22:18.891 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-05-09T04:22:18.891 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-05-09T04:22:18.891 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-05-09T04:22:18.891 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-05-09T04:22:18.891 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-05-09T04:22:18.891 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-05-09T04:22:18.891 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-05-09T04:22:18.891 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-05-09T04:22:18.891 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-05-09T04:22:18.891 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-05-09T04:22:18.891 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-05-09T04:22:18.891 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-05-09T04:22:18.891 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-05-09T04:22:18.891 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-05-09T04:22:18.891 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-05-09T04:22:18.892 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-05-09T04:22:18.892 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-05-09T04:22:18.892 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-05-09T04:22:18.892 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-05-09T04:22:18.892 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-05-09T04:22:18.892 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-05-09T04:22:18.892 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-05-09T04:22:18.892 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-05-09T04:22:18.892 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-05-09T04:22:18.892 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-05-09T04:22:18.892 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-05-09T04:22:18.892 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-05-09T04:22:18.892 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-05-09T04:22:18.892 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-05-09T04:22:18.892 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-05-09T04:22:18.892 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-05-09T04:22:18.892 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-05-09T04:22:18.893 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-05-09T04:22:18.893 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-05-09T04:22:18.893 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-05-09T04:22:18.893 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-05-09T04:22:18.893 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-05-09T04:22:18.893 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-05-09T04:22:18.893 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-05-09T04:22:18.893 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-05-09T04:22:18.893 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-05-09T04:22:18.893 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-05-09T04:22:18.893 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-05-09T04:22:18.893 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-05-09T04:22:18.893 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-05-09T04:22:18.893 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-05-09T04:22:18.893 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-05-09T04:22:18.893 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-05-09T04:22:18.894 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-05-09T04:22:18.894 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-05-09T04:22:18.894 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-05-09T04:22:18.894 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-05-09T04:22:18.894 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-05-09T04:22:18.894 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-05-09T04:22:18.894 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-05-09T04:22:18.894 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-05-09T04:22:18.894 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-05-09T04:22:18.894 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-05-09T04:22:18.894 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-05-09T04:22:18.894 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-05-09T04:22:18.894 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-05-09T04:22:18.894 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-05-09T04:22:18.894 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-05-09T04:22:18.895 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-05-09T04:22:18.895 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-05-09T04:22:18.895 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-05-09T04:22:18.895 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-05-09T04:22:18.895 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-05-09T04:22:18.895 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-05-09T04:22:18.895 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-05-09T04:22:18.895 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-05-09T04:22:18.895 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-05-09T04:22:18.895 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-05-09T04:22:18.895 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-05-09T04:22:18.895 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-05-09T04:22:18.895 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-05-09T04:22:18.895 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-05-09T04:22:18.895 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-05-09T04:22:18.895 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-05-09T04:22:18.896 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-05-09T04:22:18.896 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-05-09T04:22:18.896 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-05-09T04:22:18.896 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-05-09T04:22:18.896 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-05-09T04:22:18.896 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-05-09T04:22:18.896 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-05-09T04:22:18.896 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-05-09T04:22:18.896 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-05-09T04:22:18.896 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-05-09T04:22:18.896 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-05-09T04:22:18.896 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-05-09T04:22:18.896 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-05-09T04:22:18.896 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-05-09T04:22:18.896 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-05-09T04:22:18.897 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-05-09T04:22:18.897 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-05-09T04:22:18.897 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-05-09T04:22:18.897 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-05-09T04:22:18.897 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-05-09T04:22:18.897 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-05-09T04:22:18.897 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-05-09T04:22:18.897 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-05-09T04:22:18.897 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-05-09T04:22:18.897 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-05-09T04:22:18.897 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-05-09T04:22:18.897 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-05-09T04:22:18.897 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-05-09T04:22:18.897 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-05-09T04:22:18.898 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-05-09T04:22:18.898 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-05-09T04:22:18.898 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-05-09T04:22:18.898 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-05-09T04:22:18.898 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-05-09T04:22:18.898 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-05-09T04:22:18.898 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-05-09T04:22:18.898 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-05-09T04:22:18.898 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-05-09T04:22:18.898 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-05-09T04:22:18.898 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-05-09T04:22:18.898 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-05-09T04:22:18.898 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-05-09T04:22:18.898 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-05-09T04:22:18.898 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-05-09T04:22:18.899 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-05-09T04:22:18.899 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-05-09T04:22:18.899 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-05-09T04:22:18.899 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-05-09T04:22:18.899 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-05-09T04:22:18.899 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-05-09T04:22:18.899 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-05-09T04:22:18.899 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-05-09T04:22:18.899 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-05-09T04:22:18.899 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-05-09T04:22:18.899 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-05-09T04:22:18.899 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-05-09T04:22:18.899 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-05-09T04:22:18.899 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-05-09T04:22:18.900 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-05-09T04:22:18.900 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-05-09T04:22:18.900 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-05-09T04:22:18.900 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-05-09T04:22:18.900 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-05-09T04:22:18.900 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-05-09T04:22:18.900 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-05-09T04:22:18.900 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-05-09T04:22:18.900 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-05-09T04:22:18.900 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-05-09T04:22:18.900 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-05-09T04:22:18.900 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-05-09T04:22:18.900 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-05-09T04:22:18.900 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-05-09T04:22:18.900 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-05-09T04:22:18.901 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-05-09T04:22:18.901 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-05-09T04:22:18.901 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-05-09T04:22:18.901 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-05-09T04:22:18.901 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-05-09T04:22:18.901 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-05-09T04:22:18.901 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-05-09T04:22:18.901 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-05-09T04:22:18.901 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-05-09T04:22:18.901 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-05-09T04:22:18.901 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-05-09T04:22:18.901 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-05-09T04:22:18.901 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-05-09T04:22:18.901 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-05-09T04:22:18.901 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-05-09T04:22:18.901 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-05-09T04:22:18.902 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-05-09T04:22:18.902 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-05-09T04:22:18.902 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-05-09T04:22:18.902 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-05-09T04:22:18.902 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-05-09T04:22:18.902 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-05-09T04:22:18.902 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-05-09T04:22:18.902 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-05-09T04:22:18.902 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-05-09T04:22:18.902 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-05-09T04:22:18.902 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-05-09T04:22:18.902 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-05-09T04:22:18.902 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-05-09T04:22:18.902 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-05-09T04:22:18.902 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-05-09T04:22:18.903 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-05-09T04:22:18.903 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-05-09T04:22:18.903 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-05-09T04:22:18.903 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-05-09T04:22:18.903 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-05-09T04:22:18.903 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-05-09T04:22:18.903 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-05-09T04:22:18.903 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-05-09T04:22:18.903 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-05-09T04:22:18.903 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-05-09T04:22:18.903 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-05-09T04:22:18.903 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-05-09T04:22:18.903 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-05-09T04:22:18.903 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-05-09T04:22:18.903 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-05-09T04:22:18.904 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-05-09T04:22:18.904 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-05-09T04:22:18.904 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-05-09T04:22:18.904 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-05-09T04:22:18.904 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-05-09T04:22:18.904 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-05-09T04:22:18.904 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-05-09T04:22:18.904 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-05-09T04:22:18.904 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-05-09T04:22:18.904 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-05-09T04:22:18.904 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-05-09T04:22:18.904 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-05-09T04:22:18.904 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-05-09T04:22:18.904 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-05-09T04:22:18.904 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-05-09T04:22:18.905 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-05-09T04:22:18.905 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-05-09T04:22:18.905 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-05-09T04:22:18.905 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-05-09T04:22:18.905 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-05-09T04:22:18.905 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-05-09T04:22:18.905 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-05-09T04:22:18.905 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-05-09T04:22:18.905 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-05-09T04:22:18.905 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-05-09T04:22:18.905 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-05-09T04:22:18.905 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-05-09T04:22:18.905 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-05-09T04:22:18.905 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-05-09T04:22:18.905 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-05-09T04:22:18.906 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-05-09T04:22:18.906 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-05-09T04:22:18.906 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-05-09T04:22:18.906 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-05-09T04:22:18.906 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-05-09T04:22:18.906 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-05-09T04:22:18.906 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-05-09T04:22:18.906 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-05-09T04:22:18.906 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-05-09T04:22:18.907 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-05-09T04:22:18.907 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-05-09T04:22:18.907 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-05-09T04:22:18.907 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-05-09T04:22:18.907 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-05-09T04:22:18.907 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-05-09T04:22:18.907 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-05-09T04:22:18.907 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-05-09T04:22:18.907 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-05-09T04:22:18.907 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-05-09T04:22:18.907 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-05-09T04:22:18.907 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-05-09T04:22:18.907 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-05-09T04:22:18.907 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-05-09T04:22:18.907 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-05-09T04:22:18.908 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-05-09T04:22:18.908 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-05-09T04:22:18.908 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-05-09T04:22:18.908 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-05-09T04:22:18.908 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-05-09T04:22:18.908 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-05-09T04:22:18.908 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-05-09T04:22:18.908 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-05-09T04:22:18.908 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-05-09T04:22:18.908 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-05-09T04:22:18.908 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-05-09T04:22:18.908 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-05-09T04:22:18.908 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-05-09T04:22:18.908 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-05-09T04:22:18.908 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-05-09T04:22:18.908 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-05-09T04:22:18.909 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-05-09T04:22:18.909 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-05-09T04:22:18.909 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-05-09T04:22:18.909 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-05-09T04:22:18.909 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-05-09T04:22:18.909 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-05-09T04:22:18.909 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-05-09T04:22:18.909 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-05-09T04:22:18.909 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-05-09T04:22:18.909 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-05-09T04:22:18.909 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-05-09T04:22:18.909 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-05-09T04:22:18.909 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-05-09T04:22:18.909 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-05-09T04:22:18.909 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-05-09T04:22:18.910 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-05-09T04:22:18.910 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-05-09T04:22:18.910 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-05-09T04:22:18.910 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-05-09T04:22:18.910 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-05-09T04:22:18.910 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-05-09T04:22:18.910 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-05-09T04:22:18.910 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-05-09T04:22:18.910 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-05-09T04:22:18.910 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-05-09T04:22:18.910 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-05-09T04:22:18.910 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-05-09T04:22:18.910 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-05-09T04:22:18.910 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-05-09T04:22:18.910 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-05-09T04:22:18.911 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-05-09T04:22:18.911 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-05-09T04:22:18.911 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-05-09T04:22:18.911 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-05-09T04:22:18.911 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-05-09T04:22:18.911 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-05-09T04:22:18.911 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-05-09T04:22:18.911 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-05-09T04:22:18.911 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-05-09T04:22:18.911 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-05-09T04:22:18.911 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-05-09T04:22:18.911 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-05-09T04:22:18.911 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-05-09T04:22:18.911 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-05-09T04:22:18.911 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-05-09T04:22:18.912 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-05-09T04:22:18.912 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-05-09T04:22:18.912 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-05-09T04:22:18.912 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-05-09T04:22:18.912 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-05-09T04:22:18.912 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-05-09T04:22:18.912 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-05-09T04:22:18.912 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-05-09T04:22:18.912 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-05-09T04:22:18.912 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-05-09T04:22:18.912 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-05-09T04:22:18.912 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-05-09T04:22:18.912 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-05-09T04:22:18.912 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-05-09T04:22:18.912 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-05-09T04:22:18.913 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-05-09T04:22:18.913 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-05-09T04:22:18.913 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-05-09T04:22:18.913 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-05-09T04:22:18.913 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-05-09T04:22:18.913 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-05-09T04:22:18.913 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-05-09T04:22:18.913 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-05-09T04:22:18.913 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-05-09T04:22:18.913 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-05-09T04:22:18.913 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-05-09T04:22:18.913 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-05-09T04:22:18.913 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-05-09T04:22:18.913 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-05-09T04:22:18.914 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-05-09T04:22:18.914 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-05-09T04:22:18.914 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-05-09T04:22:18.914 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-05-09T04:22:18.914 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-05-09T04:22:18.914 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-05-09T04:22:18.914 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-05-09T04:22:18.914 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-05-09T04:22:18.914 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-05-09T04:22:18.914 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-05-09T04:22:18.914 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-05-09T04:22:18.914 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-05-09T04:22:18.914 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-05-09T04:22:18.914 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-05-09T04:22:18.914 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-05-09T04:22:18.915 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-05-09T04:22:18.915 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-05-09T04:22:18.915 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-05-09T04:22:18.915 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-05-09T04:22:18.915 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-05-09T04:22:18.915 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-05-09T04:22:18.915 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-05-09T04:22:18.915 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-05-09T04:22:18.915 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-05-09T04:22:18.915 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-05-09T04:22:18.915 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-05-09T04:22:18.915 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-05-09T04:22:18.916 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-05-09T04:22:18.916 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-05-09T04:22:18.916 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-05-09T04:22:18.916 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-05-09T04:22:18.916 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-05-09T04:22:18.916 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-05-09T04:22:18.916 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-05-09T04:22:18.916 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-05-09T04:22:18.916 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-05-09T04:22:18.916 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-05-09T04:22:18.916 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-05-09T04:22:18.916 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-05-09T04:22:18.916 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-05-09T04:22:18.916 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-05-09T04:22:18.917 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-05-09T04:22:18.917 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-05-09T04:22:18.917 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-05-09T04:22:18.917 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-05-09T04:22:18.917 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-05-09T04:22:18.917 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-05-09T04:22:18.917 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-05-09T04:22:18.917 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-05-09T04:22:18.917 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-05-09T04:22:18.917 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-05-09T04:22:18.917 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-05-09T04:22:18.917 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-05-09T04:22:18.917 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-05-09T04:22:18.917 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-05-09T04:22:18.917 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-05-09T04:22:18.918 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-05-09T04:22:18.918 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-05-09T04:22:18.918 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-05-09T04:22:18.918 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-05-09T04:22:18.918 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-05-09T04:22:18.918 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-05-09T04:22:18.918 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-05-09T04:22:18.918 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-05-09T04:22:18.918 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-05-09T04:22:18.918 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-05-09T04:22:18.918 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-05-09T04:22:18.918 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-05-09T04:22:18.918 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-05-09T04:22:18.918 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-05-09T04:22:18.918 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-05-09T04:22:18.919 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-05-09T04:22:18.919 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-05-09T04:22:18.919 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-05-09T04:22:18.919 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-05-09T04:22:18.919 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-05-09T04:22:18.919 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-05-09T04:22:18.919 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-05-09T04:22:18.919 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-05-09T04:22:18.919 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-05-09T04:22:18.919 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-05-09T04:22:18.919 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-05-09T04:22:18.919 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-05-09T04:22:18.919 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-05-09T04:22:18.919 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-05-09T04:22:18.919 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-05-09T04:22:18.919 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-05-09T04:22:18.920 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-05-09T04:22:18.920 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-05-09T04:22:18.920 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-05-09T04:22:18.920 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-05-09T04:22:18.920 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-05-09T04:22:18.920 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-05-09T04:22:18.920 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-05-09T04:22:18.920 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-05-09T04:22:18.920 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-05-09T04:22:18.920 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-05-09T04:22:18.920 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-05-09T04:22:18.920 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-05-09T04:22:18.920 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-05-09T04:22:18.921 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-05-09T04:22:18.921 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-05-09T04:22:18.921 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-05-09T04:22:18.921 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-05-09T04:22:18.921 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-05-09T04:22:18.921 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-05-09T04:22:18.921 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-05-09T04:22:18.921 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-05-09T04:22:18.921 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-05-09T04:22:18.921 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-05-09T04:22:18.921 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-05-09T04:22:18.921 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-05-09T04:22:18.921 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-05-09T04:22:18.921 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-05-09T04:22:18.921 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-05-09T04:22:18.921 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-05-09T04:22:18.922 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-05-09T04:22:18.922 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-05-09T04:22:18.922 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-05-09T04:22:18.922 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-05-09T04:22:18.922 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-05-09T04:22:18.922 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-05-09T04:22:18.922 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-05-09T04:22:18.922 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-05-09T04:22:18.922 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-05-09T04:22:18.922 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-05-09T04:22:18.922 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-05-09T04:22:18.922 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-05-09T04:22:18.922 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-05-09T04:22:18.922 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-05-09T04:22:18.923 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-05-09T04:22:18.923 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-05-09T04:22:18.923 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-05-09T04:22:18.923 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-05-09T04:22:18.923 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-05-09T04:22:18.923 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-05-09T04:22:18.923 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-05-09T04:22:18.923 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-05-09T04:22:18.923 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-05-09T04:22:18.923 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-05-09T04:22:18.923 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-05-09T04:22:18.923 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-05-09T04:22:18.923 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-05-09T04:22:18.923 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-05-09T04:22:18.923 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-05-09T04:22:18.924 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-05-09T04:22:18.924 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-05-09T04:22:18.924 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-05-09T04:22:18.924 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-05-09T04:22:18.924 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-05-09T04:22:18.924 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-05-09T04:22:18.924 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-05-09T04:22:18.924 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-05-09T04:22:18.924 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-05-09T04:22:18.924 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-05-09T04:22:18.924 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-05-09T04:22:18.924 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-05-09T04:22:18.924 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-05-09T04:22:18.924 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-05-09T04:22:18.924 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-05-09T04:22:18.924 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-05-09T04:22:18.925 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-05-09T04:22:18.925 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-05-09T04:22:18.925 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-05-09T04:22:18.925 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-05-09T04:22:18.925 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-05-09T04:22:18.925 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-05-09T04:22:18.925 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-05-09T04:22:18.925 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-05-09T04:22:18.925 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-05-09T04:22:18.925 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-05-09T04:22:18.925 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-05-09T04:22:18.925 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-05-09T04:22:18.925 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-05-09T04:22:18.925 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-05-09T04:22:18.926 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-05-09T04:22:18.926 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-05-09T04:22:18.926 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-05-09T04:22:18.926 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-05-09T04:22:18.926 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-05-09T04:22:18.926 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-05-09T04:22:18.926 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-05-09T04:22:18.926 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-05-09T04:22:18.926 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-05-09T04:22:18.926 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-05-09T04:22:18.926 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-05-09T04:22:18.926 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-05-09T04:22:18.926 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-05-09T04:22:18.926 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-05-09T04:22:18.927 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-05-09T04:22:18.927 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-05-09T04:22:18.927 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-05-09T04:22:18.927 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-05-09T04:22:18.927 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-05-09T04:22:18.927 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-05-09T04:22:18.927 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-05-09T04:22:18.927 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-05-09T04:22:18.927 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-05-09T04:22:18.927 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-05-09T04:22:18.927 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-05-09T04:22:18.927 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-05-09T04:22:18.927 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-05-09T04:22:18.927 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-05-09T04:22:18.927 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-05-09T04:22:18.928 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-05-09T04:22:18.928 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-05-09T04:22:18.928 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-05-09T04:22:18.928 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-05-09T04:22:18.928 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-05-09T04:22:18.928 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-05-09T04:22:18.928 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-05-09T04:22:18.928 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-05-09T04:22:18.928 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-05-09T04:22:18.928 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-05-09T04:22:18.928 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-05-09T04:22:18.928 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-05-09T04:22:18.928 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-05-09T04:22:18.928 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-05-09T04:22:18.928 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-05-09T04:22:18.929 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-05-09T04:22:18.929 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-05-09T04:22:18.929 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-05-09T04:22:18.929 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-05-09T04:22:18.929 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-05-09T04:22:18.929 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-05-09T04:22:18.929 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-05-09T04:22:18.929 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-05-09T04:22:18.929 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-05-09T04:22:18.929 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-05-09T04:22:18.929 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-05-09T04:22:18.929 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-05-09T04:22:18.929 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-05-09T04:22:18.929 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-05-09T04:22:18.929 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-05-09T04:22:18.929 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-05-09T04:22:18.930 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-05-09T04:22:18.930 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-05-09T04:22:18.930 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-05-09T04:22:18.930 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-05-09T04:22:18.930 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-05-09T04:22:18.930 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-05-09T04:22:18.930 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-05-09T04:22:18.930 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-05-09T04:22:18.930 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-05-09T04:22:18.930 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-05-09T04:22:18.930 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-05-09T04:22:18.930 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-05-09T04:22:18.930 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-05-09T04:22:18.930 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-05-09T04:22:18.931 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-05-09T04:22:18.931 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-05-09T04:22:18.931 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-05-09T04:22:18.931 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-05-09T04:22:18.931 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-05-09T04:22:18.931 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-05-09T04:22:18.931 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-05-09T04:22:18.931 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-05-09T04:22:18.931 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-05-09T04:22:18.931 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-05-09T04:22:18.931 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-05-09T04:22:18.931 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-05-09T04:22:18.931 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-05-09T04:22:18.931 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-05-09T04:22:18.931 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-05-09T04:22:18.931 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-05-09T04:22:18.931 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-05-09T04:22:18.932 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-05-09T04:22:18.932 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-05-09T04:22:18.932 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-05-09T04:22:18.932 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-05-09T04:22:18.932 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-05-09T04:22:18.932 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-05-09T04:22:18.932 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-05-09T04:22:18.932 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-05-09T04:22:18.932 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-05-09T04:22:18.932 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-05-09T04:22:18.932 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-05-09T04:22:18.932 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-05-09T04:22:18.932 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-05-09T04:22:18.932 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-05-09T04:22:18.932 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-05-09T04:22:18.932 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-05-09T04:22:18.933 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-05-09T04:22:18.933 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-05-09T04:22:18.933 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-05-09T04:22:18.933 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-05-09T04:22:18.933 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-05-09T04:22:18.933 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-05-09T04:22:18.933 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-05-09T04:22:18.933 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-05-09T04:22:18.933 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-05-09T04:22:18.933 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-05-09T04:22:18.933 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-05-09T04:22:18.933 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-05-09T04:22:18.933 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-05-09T04:22:18.933 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-05-09T04:22:18.933 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-05-09T04:22:18.933 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-05-09T04:22:18.934 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-05-09T04:22:18.934 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-05-09T04:22:18.934 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-05-09T04:22:18.934 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-05-09T04:22:18.934 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-05-09T04:22:18.934 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-05-09T04:22:18.934 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-05-09T04:22:18.934 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-05-09T04:22:18.934 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-05-09T04:22:18.934 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-05-09T04:22:18.934 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-05-09T04:22:18.934 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-05-09T04:22:18.934 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-05-09T04:22:18.934 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-05-09T04:22:18.934 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-05-09T04:22:18.935 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-05-09T04:22:18.935 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-05-09T04:22:18.935 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-05-09T04:22:18.935 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-05-09T04:22:18.935 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-05-09T04:22:18.935 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-05-09T04:22:18.935 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-05-09T04:22:18.935 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-05-09T04:22:18.935 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-05-09T04:22:18.935 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-05-09T04:22:18.935 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-05-09T04:22:18.935 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-05-09T04:22:18.935 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-05-09T04:22:18.935 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-05-09T04:22:18.935 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-05-09T04:22:18.936 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-05-09T04:22:18.936 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-05-09T04:22:18.936 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-05-09T04:22:18.936 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-05-09T04:22:18.936 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-05-09T04:22:18.936 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-05-09T04:22:18.936 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-05-09T04:22:18.936 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-05-09T04:22:18.936 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-05-09T04:22:18.936 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-05-09T04:22:18.936 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-05-09T04:22:18.936 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-05-09T04:22:18.936 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-05-09T04:22:18.936 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-05-09T04:22:18.936 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-05-09T04:22:18.937 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-05-09T04:22:18.937 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-05-09T04:22:18.937 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-05-09T04:22:18.937 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-05-09T04:22:18.937 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-05-09T04:22:18.937 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-05-09T04:22:18.937 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-05-09T04:22:18.937 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-05-09T04:22:18.937 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-05-09T04:22:18.937 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-05-09T04:22:18.937 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-05-09T04:22:18.937 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-05-09T04:22:18.937 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-05-09T04:22:18.937 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-05-09T04:22:18.937 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-05-09T04:22:18.938 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-05-09T04:22:18.938 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-05-09T04:22:18.938 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-05-09T04:22:18.938 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-05-09T04:22:18.938 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-05-09T04:22:18.938 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-05-09T04:22:18.938 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-05-09T04:22:18.938 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-05-09T04:22:18.938 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-05-09T04:22:18.938 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-05-09T04:22:18.938 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-05-09T04:22:18.938 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-05-09T04:22:18.938 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-05-09T04:22:18.938 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-05-09T04:22:18.938 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-05-09T04:22:18.939 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-05-09T04:22:18.939 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-05-09T04:22:18.939 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-05-09T04:22:18.939 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-05-09T04:22:18.939 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-05-09T04:22:18.939 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-05-09T04:22:18.939 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-05-09T04:22:18.939 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-05-09T04:22:18.939 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-05-09T04:22:18.939 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-05-09T04:22:18.939 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-05-09T04:22:18.939 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-05-09T04:22:18.939 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-05-09T04:22:18.939 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-05-09T04:22:18.939 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-05-09T04:22:18.939 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-05-09T04:22:18.940 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-05-09T04:22:18.940 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-05-09T04:22:18.940 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-05-09T04:22:18.940 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-05-09T04:22:18.940 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-05-09T04:22:18.940 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-05-09T04:22:18.940 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-05-09T04:22:18.940 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-05-09T04:22:18.940 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-05-09T04:22:18.940 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-05-09T04:22:18.940 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-05-09T04:22:18.940 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-05-09T04:22:18.940 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-05-09T04:22:18.940 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-05-09T04:22:18.940 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-05-09T04:22:18.941 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-05-09T04:22:18.941 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-05-09T04:22:18.941 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-05-09T04:22:18.941 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-05-09T04:22:18.941 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-05-09T04:22:18.941 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-05-09T04:22:18.941 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-05-09T04:22:18.941 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-05-09T04:22:18.941 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-05-09T04:22:18.941 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-05-09T04:22:18.941 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-05-09T04:22:18.941 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-05-09T04:22:18.941 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-05-09T04:22:18.941 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-05-09T04:22:18.941 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-05-09T04:22:18.941 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-05-09T04:22:18.942 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-05-09T04:22:18.942 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-05-09T04:22:18.942 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-05-09T04:22:18.942 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-05-09T04:22:18.942 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-05-09T04:22:18.942 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-05-09T04:22:18.942 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-05-09T04:22:18.942 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-05-09T04:22:18.942 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-05-09T04:22:18.942 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-05-09T04:22:18.942 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-05-09T04:22:18.942 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-05-09T04:22:18.942 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-05-09T04:22:18.942 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-05-09T04:22:18.942 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-05-09T04:22:18.943 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-05-09T04:22:18.943 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-05-09T04:22:18.943 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-05-09T04:22:18.943 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-05-09T04:22:18.943 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-05-09T04:22:18.943 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-05-09T04:22:18.943 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-05-09T04:22:18.943 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-05-09T04:22:18.943 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-05-09T04:22:18.943 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-05-09T04:22:18.943 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-05-09T04:22:18.943 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-05-09T04:22:18.943 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-05-09T04:22:18.943 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-05-09T04:22:18.943 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-05-09T04:22:18.947 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-05-09T04:22:18.948 DEBUG:teuthology.orchestra.run.smithi192:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-05-09T04:22:18.970 INFO:teuthology.orchestra.run.smithi192.stdout:postinstall scriptlet (using /bin/sh): 2024-05-09T04:22:18.970 INFO:teuthology.orchestra.run.smithi192.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-09T04:22:18.970 INFO:teuthology.orchestra.run.smithi192.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-05-09T04:22:18.970 INFO:teuthology.orchestra.run.smithi192.stdout:fi 2024-05-09T04:22:18.970 INFO:teuthology.orchestra.run.smithi192.stdout:for file in vmlinuz System.map config; do 2024-05-09T04:22:18.970 INFO:teuthology.orchestra.run.smithi192.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-05-09T04:22:18.970 INFO:teuthology.orchestra.run.smithi192.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-05-09T04:22:18.970 INFO:teuthology.orchestra.run.smithi192.stdout: fi 2024-05-09T04:22:18.970 INFO:teuthology.orchestra.run.smithi192.stdout:done 2024-05-09T04:22:18.970 INFO:teuthology.orchestra.run.smithi192.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-05-09T04:22:18.971 INFO:teuthology.orchestra.run.smithi192.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-05-09T04:22:18.971 INFO:teuthology.orchestra.run.smithi192.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-05-09T04:22:18.971 INFO:teuthology.orchestra.run.smithi192.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-05-09T04:22:18.971 INFO:teuthology.orchestra.run.smithi192.stdout:fi 2024-05-09T04:22:18.971 INFO:teuthology.orchestra.run.smithi192.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-05-09T04:22:18.971 INFO:teuthology.orchestra.run.smithi192.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-05-09T04:22:18.971 INFO:teuthology.orchestra.run.smithi192.stdout:fi 2024-05-09T04:22:18.971 INFO:teuthology.orchestra.run.smithi192.stdout:preuninstall scriptlet (using /bin/sh): 2024-05-09T04:22:18.971 INFO:teuthology.orchestra.run.smithi192.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-09T04:22:18.971 INFO:teuthology.orchestra.run.smithi192.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-05-09T04:22:18.971 INFO:teuthology.orchestra.run.smithi192.stdout:fi 2024-05-09T04:22:18.972 DEBUG:teuthology.orchestra.run.smithi192:> sudo rpm -qi grub2-tools 2024-05-09T04:22:19.044 INFO:teuthology.orchestra.run.smithi192.stdout:Name : grub2-tools 2024-05-09T04:22:19.044 INFO:teuthology.orchestra.run.smithi192.stdout:Epoch : 1 2024-05-09T04:22:19.044 INFO:teuthology.orchestra.run.smithi192.stdout:Version : 2.06 2024-05-09T04:22:19.044 INFO:teuthology.orchestra.run.smithi192.stdout:Release : 68.el9 2024-05-09T04:22:19.044 INFO:teuthology.orchestra.run.smithi192.stdout:Architecture: x86_64 2024-05-09T04:22:19.044 INFO:teuthology.orchestra.run.smithi192.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-09T04:22:19.044 INFO:teuthology.orchestra.run.smithi192.stdout:Group : Unspecified 2024-05-09T04:22:19.044 INFO:teuthology.orchestra.run.smithi192.stdout:Size : 8264254 2024-05-09T04:22:19.044 INFO:teuthology.orchestra.run.smithi192.stdout:License : GPLv3+ 2024-05-09T04:22:19.045 INFO:teuthology.orchestra.run.smithi192.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-09T04:22:19.045 INFO:teuthology.orchestra.run.smithi192.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-09T04:22:19.045 INFO:teuthology.orchestra.run.smithi192.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-09T04:22:19.045 INFO:teuthology.orchestra.run.smithi192.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-09T04:22:19.045 INFO:teuthology.orchestra.run.smithi192.stdout:Packager : builder@centos.org 2024-05-09T04:22:19.045 INFO:teuthology.orchestra.run.smithi192.stdout:Vendor : CentOS 2024-05-09T04:22:19.045 INFO:teuthology.orchestra.run.smithi192.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-09T04:22:19.045 INFO:teuthology.orchestra.run.smithi192.stdout:Summary : Support tools for GRUB. 2024-05-09T04:22:19.045 INFO:teuthology.orchestra.run.smithi192.stdout:Description : 2024-05-09T04:22:19.045 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-09T04:22:19.045 INFO:teuthology.orchestra.run.smithi192.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-09T04:22:19.045 INFO:teuthology.orchestra.run.smithi192.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-09T04:22:19.045 INFO:teuthology.orchestra.run.smithi192.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-09T04:22:19.045 INFO:teuthology.orchestra.run.smithi192.stdout:hardware devices. 2024-05-09T04:22:19.045 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-09T04:22:19.045 INFO:teuthology.orchestra.run.smithi192.stdout:This subpackage provides tools for support of all platforms. 2024-05-09T04:22:19.046 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-09T04:22:19.046 INFO:teuthology.task.kernel:Updating grub on smithi192 to boot 6.9.0-rc3-g806ec36b2112 2024-05-09T04:22:19.047 DEBUG:teuthology.orchestra.run.smithi192:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-09T04:22:19.571 INFO:teuthology.orchestra.run.smithi192.stderr:Generating grub configuration file ... 2024-05-09T04:22:20.985 INFO:teuthology.orchestra.run.smithi192.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-09T04:22:21.011 INFO:teuthology.orchestra.run.smithi192.stderr:done 2024-05-09T04:22:21.013 DEBUG:teuthology.orchestra.run.smithi192:> mktemp 2024-05-09T04:22:21.030 INFO:teuthology.orchestra.run.smithi192.stdout:/tmp/tmp.bqt4JUEkOm 2024-05-09T04:22:21.030 DEBUG:teuthology.orchestra.run.smithi192:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.bqt4JUEkOm 2024-05-09T04:22:21.098 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 0666 /tmp/tmp.bqt4JUEkOm 2024-05-09T04:22:21.240 DEBUG:teuthology.orchestra.remote:smithi192:/tmp/tmp.bqt4JUEkOm is 6KB 2024-05-09T04:22:21.288 DEBUG:teuthology.orchestra.run.smithi192:> rm -fr /tmp/tmp.bqt4JUEkOm 2024-05-09T04:22:21.305 DEBUG:teuthology.orchestra.run.smithi192:> sudo /bin/ls /boot/loader/entries || true 2024-05-09T04:22:21.374 INFO:teuthology.orchestra.run.smithi192.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-09T04:22:21.374 INFO:teuthology.orchestra.run.smithi192.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-09T04:22:21.374 INFO:teuthology.orchestra.run.smithi192.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-09T04:22:21.374 INFO:teuthology.orchestra.run.smithi192.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-05-09T04:22:21.376 DEBUG:teuthology.orchestra.run.smithi192:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-05-09T04:22:21.524 DEBUG:teuthology.orchestra.run.smithi192:> sudo shutdown -r now 2024-05-09T04:22:32.455 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi145.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-05-09T04:22:32.455 DEBUG:teuthology.orchestra.run.smithi145:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-05-09T04:22:32.545 INFO:teuthology.orchestra.run.smithi145.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-05-09T04:22:32.545 INFO:teuthology.orchestra.run.smithi145.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-05-09T04:22:32.545 INFO:teuthology.orchestra.run.smithi145.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-05-09T04:22:32.545 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-05-09T04:22:32.545 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-05-09T04:22:32.545 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-05-09T04:22:32.545 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-05-09T04:22:32.545 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-05-09T04:22:32.545 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-05-09T04:22:32.545 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-05-09T04:22:32.545 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-05-09T04:22:32.546 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-05-09T04:22:32.546 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-05-09T04:22:32.546 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-05-09T04:22:32.546 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-05-09T04:22:32.546 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-05-09T04:22:32.546 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-05-09T04:22:32.546 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-05-09T04:22:32.546 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-05-09T04:22:32.546 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-05-09T04:22:32.546 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-05-09T04:22:32.546 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-05-09T04:22:32.546 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-05-09T04:22:32.546 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-05-09T04:22:32.547 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-05-09T04:22:32.547 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-05-09T04:22:32.547 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-05-09T04:22:32.547 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-05-09T04:22:32.547 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-05-09T04:22:32.547 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-05-09T04:22:32.547 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-05-09T04:22:32.547 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-05-09T04:22:32.547 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-05-09T04:22:32.547 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-05-09T04:22:32.547 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-05-09T04:22:32.547 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-05-09T04:22:32.547 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-05-09T04:22:32.547 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-05-09T04:22:32.547 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-05-09T04:22:32.548 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-05-09T04:22:32.548 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-05-09T04:22:32.548 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-05-09T04:22:32.548 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-05-09T04:22:32.548 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-05-09T04:22:32.548 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-05-09T04:22:32.548 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-05-09T04:22:32.548 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-05-09T04:22:32.548 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-05-09T04:22:32.548 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-05-09T04:22:32.548 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-05-09T04:22:32.548 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-05-09T04:22:32.548 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-05-09T04:22:32.548 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-05-09T04:22:32.549 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-05-09T04:22:32.549 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-05-09T04:22:32.549 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-05-09T04:22:32.549 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-05-09T04:22:32.549 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-05-09T04:22:32.549 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-05-09T04:22:32.549 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-05-09T04:22:32.549 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-05-09T04:22:32.549 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-05-09T04:22:32.549 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-05-09T04:22:32.549 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-05-09T04:22:32.549 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-05-09T04:22:32.549 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-05-09T04:22:32.549 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-05-09T04:22:32.550 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-05-09T04:22:32.550 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-05-09T04:22:32.550 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-05-09T04:22:32.550 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-05-09T04:22:32.550 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-05-09T04:22:32.550 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-05-09T04:22:32.550 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-05-09T04:22:32.550 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-05-09T04:22:32.550 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-05-09T04:22:32.550 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-05-09T04:22:32.550 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-05-09T04:22:32.550 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-05-09T04:22:32.550 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-05-09T04:22:32.550 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-05-09T04:22:32.551 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-05-09T04:22:32.551 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-05-09T04:22:32.551 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-05-09T04:22:32.551 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-05-09T04:22:32.551 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-05-09T04:22:32.551 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-05-09T04:22:32.551 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-05-09T04:22:32.551 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-05-09T04:22:32.551 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-05-09T04:22:32.551 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-05-09T04:22:32.551 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-05-09T04:22:32.551 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-05-09T04:22:32.551 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-05-09T04:22:32.552 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-05-09T04:22:32.552 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-05-09T04:22:32.552 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-05-09T04:22:32.552 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-05-09T04:22:32.552 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-05-09T04:22:32.552 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-05-09T04:22:32.552 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-05-09T04:22:32.552 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-05-09T04:22:32.552 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-05-09T04:22:32.552 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-05-09T04:22:32.552 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-05-09T04:22:32.552 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-05-09T04:22:32.552 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-05-09T04:22:32.552 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-05-09T04:22:32.552 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-05-09T04:22:32.553 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-05-09T04:22:32.553 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-05-09T04:22:32.553 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-05-09T04:22:32.553 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-05-09T04:22:32.553 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-05-09T04:22:32.553 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-05-09T04:22:32.553 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-05-09T04:22:32.553 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-05-09T04:22:32.553 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-05-09T04:22:32.553 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-05-09T04:22:32.553 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-05-09T04:22:32.553 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-05-09T04:22:32.553 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-05-09T04:22:32.553 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-05-09T04:22:32.554 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-05-09T04:22:32.554 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-05-09T04:22:32.554 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-05-09T04:22:32.554 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-05-09T04:22:32.554 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-05-09T04:22:32.554 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-05-09T04:22:32.554 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-05-09T04:22:32.554 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-05-09T04:22:32.554 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-05-09T04:22:32.554 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-05-09T04:22:32.554 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-05-09T04:22:32.554 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-05-09T04:22:32.554 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-05-09T04:22:32.554 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-05-09T04:22:32.555 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-05-09T04:22:32.555 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-05-09T04:22:32.555 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-05-09T04:22:32.555 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-05-09T04:22:32.555 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-05-09T04:22:32.555 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-05-09T04:22:32.555 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-05-09T04:22:32.555 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-05-09T04:22:32.555 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-05-09T04:22:32.555 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-05-09T04:22:32.555 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-05-09T04:22:32.555 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-05-09T04:22:32.555 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-05-09T04:22:32.555 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-05-09T04:22:32.556 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-05-09T04:22:32.556 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-05-09T04:22:32.556 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-05-09T04:22:32.556 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-05-09T04:22:32.556 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-05-09T04:22:32.556 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-05-09T04:22:32.556 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-05-09T04:22:32.556 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-05-09T04:22:32.556 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-05-09T04:22:32.556 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-05-09T04:22:32.556 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-05-09T04:22:32.556 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-05-09T04:22:32.556 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-05-09T04:22:32.557 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-05-09T04:22:32.557 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-05-09T04:22:32.557 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-05-09T04:22:32.557 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-05-09T04:22:32.557 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-05-09T04:22:32.557 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-05-09T04:22:32.557 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-05-09T04:22:32.557 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-05-09T04:22:32.557 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-05-09T04:22:32.557 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-05-09T04:22:32.557 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-05-09T04:22:32.557 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-05-09T04:22:32.557 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-05-09T04:22:32.557 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-05-09T04:22:32.557 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-05-09T04:22:32.558 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-05-09T04:22:32.558 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-05-09T04:22:32.558 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-05-09T04:22:32.558 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-05-09T04:22:32.558 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-05-09T04:22:32.558 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-05-09T04:22:32.558 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-05-09T04:22:32.558 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-05-09T04:22:32.558 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-05-09T04:22:32.558 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-05-09T04:22:32.558 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-05-09T04:22:32.558 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-05-09T04:22:32.558 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-05-09T04:22:32.558 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-05-09T04:22:32.558 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-05-09T04:22:32.559 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-05-09T04:22:32.559 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-05-09T04:22:32.559 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-05-09T04:22:32.559 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-05-09T04:22:32.559 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-05-09T04:22:32.559 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-05-09T04:22:32.559 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-05-09T04:22:32.559 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-05-09T04:22:32.559 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-05-09T04:22:32.559 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-05-09T04:22:32.559 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-05-09T04:22:32.559 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-05-09T04:22:32.559 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-05-09T04:22:32.560 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-05-09T04:22:32.560 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-05-09T04:22:32.560 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-05-09T04:22:32.560 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-05-09T04:22:32.560 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-05-09T04:22:32.560 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-05-09T04:22:32.560 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-05-09T04:22:32.560 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-05-09T04:22:32.560 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-05-09T04:22:32.560 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-05-09T04:22:32.560 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-05-09T04:22:32.560 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-05-09T04:22:32.560 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-05-09T04:22:32.560 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-05-09T04:22:32.560 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-05-09T04:22:32.561 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-05-09T04:22:32.561 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-05-09T04:22:32.561 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-05-09T04:22:32.561 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-05-09T04:22:32.561 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-05-09T04:22:32.561 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-05-09T04:22:32.561 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-05-09T04:22:32.561 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-05-09T04:22:32.561 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-05-09T04:22:32.561 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-05-09T04:22:32.561 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-05-09T04:22:32.561 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-05-09T04:22:32.561 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-05-09T04:22:32.561 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-05-09T04:22:32.561 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-05-09T04:22:32.562 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-05-09T04:22:32.562 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-05-09T04:22:32.562 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-05-09T04:22:32.562 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-05-09T04:22:32.562 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-05-09T04:22:32.562 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-05-09T04:22:32.562 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-05-09T04:22:32.562 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-05-09T04:22:32.562 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-05-09T04:22:32.562 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-05-09T04:22:32.562 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-05-09T04:22:32.562 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-05-09T04:22:32.562 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-05-09T04:22:32.562 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-05-09T04:22:32.563 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-05-09T04:22:32.563 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-05-09T04:22:32.563 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-05-09T04:22:32.563 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-05-09T04:22:32.563 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-05-09T04:22:32.563 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-05-09T04:22:32.563 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-05-09T04:22:32.563 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-05-09T04:22:32.563 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-05-09T04:22:32.563 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-05-09T04:22:32.563 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-05-09T04:22:32.563 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-05-09T04:22:32.563 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-05-09T04:22:32.563 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-05-09T04:22:32.564 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-05-09T04:22:32.564 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-05-09T04:22:32.564 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-05-09T04:22:32.564 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-05-09T04:22:32.564 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-05-09T04:22:32.564 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-05-09T04:22:32.564 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-05-09T04:22:32.564 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-05-09T04:22:32.564 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-05-09T04:22:32.564 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-05-09T04:22:32.564 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-05-09T04:22:32.564 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-05-09T04:22:32.564 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-05-09T04:22:32.565 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-05-09T04:22:32.565 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-05-09T04:22:32.565 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-05-09T04:22:32.565 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-05-09T04:22:32.565 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-05-09T04:22:32.565 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-05-09T04:22:32.565 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-05-09T04:22:32.565 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-05-09T04:22:32.565 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-05-09T04:22:32.565 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-05-09T04:22:32.565 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-05-09T04:22:32.565 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-05-09T04:22:32.565 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-05-09T04:22:32.565 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-05-09T04:22:32.566 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-05-09T04:22:32.566 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-05-09T04:22:32.566 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-05-09T04:22:32.566 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-05-09T04:22:32.566 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-05-09T04:22:32.566 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-05-09T04:22:32.566 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-05-09T04:22:32.566 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-05-09T04:22:32.566 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-05-09T04:22:32.566 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-05-09T04:22:32.566 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-05-09T04:22:32.566 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-05-09T04:22:32.566 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-05-09T04:22:32.567 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-05-09T04:22:32.567 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-05-09T04:22:32.567 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-05-09T04:22:32.567 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-05-09T04:22:32.567 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-05-09T04:22:32.567 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-05-09T04:22:32.567 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-05-09T04:22:32.567 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-05-09T04:22:32.567 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-05-09T04:22:32.567 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-05-09T04:22:32.567 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-05-09T04:22:32.567 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-05-09T04:22:32.567 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-05-09T04:22:32.567 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-05-09T04:22:32.567 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-05-09T04:22:32.567 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-05-09T04:22:32.568 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-05-09T04:22:32.568 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-05-09T04:22:32.568 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-05-09T04:22:32.568 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-05-09T04:22:32.568 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-05-09T04:22:32.568 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-05-09T04:22:32.568 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-05-09T04:22:32.568 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-05-09T04:22:32.568 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-05-09T04:22:32.568 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-05-09T04:22:32.568 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-05-09T04:22:32.568 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-05-09T04:22:32.568 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-05-09T04:22:32.568 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-05-09T04:22:32.568 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-05-09T04:22:32.568 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-05-09T04:22:32.568 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-05-09T04:22:32.569 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-05-09T04:22:32.569 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-05-09T04:22:32.569 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-05-09T04:22:32.569 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-05-09T04:22:32.569 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-05-09T04:22:32.569 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-05-09T04:22:32.569 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-05-09T04:22:32.569 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-05-09T04:22:32.569 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-05-09T04:22:32.569 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-05-09T04:22:32.569 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-05-09T04:22:32.569 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-05-09T04:22:32.569 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-05-09T04:22:32.569 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-05-09T04:22:32.569 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-05-09T04:22:32.569 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-05-09T04:22:32.570 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-05-09T04:22:32.570 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-05-09T04:22:32.570 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-05-09T04:22:32.570 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-05-09T04:22:32.570 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-05-09T04:22:32.570 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-05-09T04:22:32.570 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-05-09T04:22:32.570 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-05-09T04:22:32.570 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-05-09T04:22:32.570 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-05-09T04:22:32.570 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-05-09T04:22:32.570 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-05-09T04:22:32.570 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-05-09T04:22:32.570 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-05-09T04:22:32.571 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-05-09T04:22:32.571 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-05-09T04:22:32.571 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-05-09T04:22:32.571 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-05-09T04:22:32.571 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-05-09T04:22:32.571 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-05-09T04:22:32.571 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-05-09T04:22:32.571 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-05-09T04:22:32.571 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-05-09T04:22:32.571 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-05-09T04:22:32.571 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-05-09T04:22:32.571 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-05-09T04:22:32.571 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-05-09T04:22:32.571 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-05-09T04:22:32.571 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-05-09T04:22:32.571 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-05-09T04:22:32.572 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-05-09T04:22:32.572 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-05-09T04:22:32.572 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-05-09T04:22:32.572 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-05-09T04:22:32.572 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-05-09T04:22:32.572 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-05-09T04:22:32.572 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-05-09T04:22:32.572 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-05-09T04:22:32.572 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-05-09T04:22:32.572 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-05-09T04:22:32.572 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-05-09T04:22:32.572 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-05-09T04:22:32.572 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-05-09T04:22:32.572 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-05-09T04:22:32.572 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-05-09T04:22:32.572 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-05-09T04:22:32.573 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-05-09T04:22:32.573 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-05-09T04:22:32.573 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-05-09T04:22:32.573 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-05-09T04:22:32.573 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-05-09T04:22:32.573 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-05-09T04:22:32.573 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-05-09T04:22:32.573 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-05-09T04:22:32.573 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-05-09T04:22:32.573 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-05-09T04:22:32.573 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-05-09T04:22:32.573 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-05-09T04:22:32.573 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-05-09T04:22:32.573 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-05-09T04:22:32.573 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-05-09T04:22:32.573 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-05-09T04:22:32.574 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-05-09T04:22:32.574 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-05-09T04:22:32.574 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-05-09T04:22:32.574 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-05-09T04:22:32.574 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-05-09T04:22:32.574 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-05-09T04:22:32.574 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-05-09T04:22:32.574 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-05-09T04:22:32.574 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-05-09T04:22:32.574 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-05-09T04:22:32.574 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-05-09T04:22:32.574 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-05-09T04:22:32.574 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-05-09T04:22:32.574 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-05-09T04:22:32.574 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-05-09T04:22:32.575 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-05-09T04:22:32.575 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-05-09T04:22:32.575 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-05-09T04:22:32.575 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-05-09T04:22:32.575 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-05-09T04:22:32.575 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-05-09T04:22:32.575 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-05-09T04:22:32.575 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-05-09T04:22:32.575 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-05-09T04:22:32.575 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-05-09T04:22:32.575 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-05-09T04:22:32.575 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-05-09T04:22:32.575 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-05-09T04:22:32.575 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-05-09T04:22:32.575 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-05-09T04:22:32.575 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-05-09T04:22:32.575 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-05-09T04:22:32.576 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-05-09T04:22:32.576 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-05-09T04:22:32.576 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-05-09T04:22:32.576 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-05-09T04:22:32.576 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-05-09T04:22:32.576 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-05-09T04:22:32.576 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-05-09T04:22:32.576 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-05-09T04:22:32.576 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-05-09T04:22:32.576 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-05-09T04:22:32.576 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-05-09T04:22:32.576 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-05-09T04:22:32.576 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-05-09T04:22:32.576 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-05-09T04:22:32.576 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-05-09T04:22:32.576 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-05-09T04:22:32.576 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-05-09T04:22:32.576 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-05-09T04:22:32.577 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-05-09T04:22:32.577 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-05-09T04:22:32.577 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-05-09T04:22:32.577 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-05-09T04:22:32.577 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-05-09T04:22:32.577 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-05-09T04:22:32.577 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-05-09T04:22:32.577 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-05-09T04:22:32.577 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-05-09T04:22:32.577 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-05-09T04:22:32.577 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-05-09T04:22:32.577 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-05-09T04:22:32.577 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-05-09T04:22:32.578 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-05-09T04:22:32.578 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-05-09T04:22:32.578 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-05-09T04:22:32.578 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-05-09T04:22:32.578 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-05-09T04:22:32.578 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-05-09T04:22:32.578 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-05-09T04:22:32.578 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-05-09T04:22:32.578 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-05-09T04:22:32.578 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-05-09T04:22:32.578 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-05-09T04:22:32.578 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-05-09T04:22:32.578 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-05-09T04:22:32.578 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-05-09T04:22:32.578 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-05-09T04:22:32.579 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-05-09T04:22:32.579 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-05-09T04:22:32.579 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-05-09T04:22:32.579 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-05-09T04:22:32.579 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-05-09T04:22:32.579 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-05-09T04:22:32.579 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-05-09T04:22:32.579 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-05-09T04:22:32.579 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-05-09T04:22:32.579 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-05-09T04:22:32.579 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-05-09T04:22:32.579 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-05-09T04:22:32.579 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-05-09T04:22:32.579 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-05-09T04:22:32.579 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-05-09T04:22:32.580 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-05-09T04:22:32.580 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-05-09T04:22:32.580 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-05-09T04:22:32.580 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-05-09T04:22:32.580 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-05-09T04:22:32.580 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-05-09T04:22:32.580 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-05-09T04:22:32.580 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-05-09T04:22:32.580 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-05-09T04:22:32.580 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-05-09T04:22:32.580 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-05-09T04:22:32.580 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-05-09T04:22:32.580 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-05-09T04:22:32.580 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-05-09T04:22:32.580 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-05-09T04:22:32.580 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-05-09T04:22:32.580 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-05-09T04:22:32.581 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-05-09T04:22:32.581 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-05-09T04:22:32.581 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-05-09T04:22:32.581 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-05-09T04:22:32.581 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-05-09T04:22:32.581 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-05-09T04:22:32.581 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-05-09T04:22:32.581 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-05-09T04:22:32.581 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-05-09T04:22:32.581 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-05-09T04:22:32.581 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-05-09T04:22:32.581 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-05-09T04:22:32.581 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-05-09T04:22:32.581 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-05-09T04:22:32.581 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-05-09T04:22:32.581 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-05-09T04:22:32.581 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-05-09T04:22:32.582 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-05-09T04:22:32.582 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-05-09T04:22:32.582 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-05-09T04:22:32.582 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-05-09T04:22:32.582 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-05-09T04:22:32.582 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-05-09T04:22:32.582 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-05-09T04:22:32.582 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-05-09T04:22:32.582 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-05-09T04:22:32.582 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-05-09T04:22:32.582 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-05-09T04:22:32.582 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-05-09T04:22:32.582 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-05-09T04:22:32.582 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-05-09T04:22:32.582 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-05-09T04:22:32.582 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-05-09T04:22:32.583 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-05-09T04:22:32.583 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-05-09T04:22:32.583 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-05-09T04:22:32.583 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-05-09T04:22:32.583 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-05-09T04:22:32.583 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-05-09T04:22:32.583 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-05-09T04:22:32.583 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-05-09T04:22:32.583 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-05-09T04:22:32.583 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-05-09T04:22:32.583 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-05-09T04:22:32.583 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-05-09T04:22:32.583 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-05-09T04:22:32.583 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-05-09T04:22:32.583 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-05-09T04:22:32.584 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-05-09T04:22:32.584 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-05-09T04:22:32.584 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-05-09T04:22:32.584 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-05-09T04:22:32.584 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-05-09T04:22:32.584 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-05-09T04:22:32.584 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-05-09T04:22:32.584 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-05-09T04:22:32.584 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-05-09T04:22:32.584 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-05-09T04:22:32.584 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-05-09T04:22:32.584 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-05-09T04:22:32.584 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-05-09T04:22:32.584 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-05-09T04:22:32.584 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-05-09T04:22:32.584 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-05-09T04:22:32.585 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-05-09T04:22:32.585 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-05-09T04:22:32.585 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-05-09T04:22:32.585 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-05-09T04:22:32.585 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-05-09T04:22:32.585 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-05-09T04:22:32.585 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-05-09T04:22:32.585 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-05-09T04:22:32.585 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-05-09T04:22:32.585 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-05-09T04:22:32.585 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-05-09T04:22:32.585 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-05-09T04:22:32.585 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-05-09T04:22:32.585 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-05-09T04:22:32.585 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-05-09T04:22:32.585 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-05-09T04:22:32.586 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-05-09T04:22:32.586 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-05-09T04:22:32.586 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-05-09T04:22:32.586 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-05-09T04:22:32.586 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-05-09T04:22:32.586 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-05-09T04:22:32.586 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-05-09T04:22:32.586 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-05-09T04:22:32.586 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-05-09T04:22:32.586 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-05-09T04:22:32.586 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-05-09T04:22:32.586 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-05-09T04:22:32.586 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-05-09T04:22:32.586 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-05-09T04:22:32.586 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-05-09T04:22:32.587 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-05-09T04:22:32.587 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-05-09T04:22:32.587 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-05-09T04:22:32.587 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-05-09T04:22:32.587 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-05-09T04:22:32.587 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-05-09T04:22:32.587 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-05-09T04:22:32.587 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-05-09T04:22:32.587 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-05-09T04:22:32.587 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-05-09T04:22:32.587 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-05-09T04:22:32.587 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-05-09T04:22:32.589 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-05-09T04:22:32.589 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-05-09T04:22:32.589 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-05-09T04:22:32.589 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-05-09T04:22:32.589 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-05-09T04:22:32.589 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-05-09T04:22:32.590 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-05-09T04:22:32.590 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-05-09T04:22:32.590 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-05-09T04:22:32.590 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-05-09T04:22:32.590 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-05-09T04:22:32.590 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-05-09T04:22:32.590 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-05-09T04:22:32.590 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-05-09T04:22:32.590 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-05-09T04:22:32.590 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-05-09T04:22:32.590 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-05-09T04:22:32.590 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-05-09T04:22:32.590 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-05-09T04:22:32.591 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-05-09T04:22:32.591 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-05-09T04:22:32.591 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-05-09T04:22:32.591 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-05-09T04:22:32.591 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-05-09T04:22:32.591 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-05-09T04:22:32.591 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-05-09T04:22:32.591 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-05-09T04:22:32.591 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-05-09T04:22:32.591 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-05-09T04:22:32.591 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-05-09T04:22:32.591 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-05-09T04:22:32.591 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-05-09T04:22:32.591 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-05-09T04:22:32.592 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-05-09T04:22:32.592 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-05-09T04:22:32.592 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-05-09T04:22:32.592 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-05-09T04:22:32.592 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-05-09T04:22:32.592 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-05-09T04:22:32.592 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-05-09T04:22:32.592 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-05-09T04:22:32.592 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-05-09T04:22:32.592 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-05-09T04:22:32.592 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-05-09T04:22:32.592 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-05-09T04:22:32.592 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-05-09T04:22:32.592 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-05-09T04:22:32.592 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-05-09T04:22:32.593 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-05-09T04:22:32.593 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-05-09T04:22:32.593 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-05-09T04:22:32.593 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-05-09T04:22:32.593 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-05-09T04:22:32.593 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-05-09T04:22:32.593 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-05-09T04:22:32.593 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-05-09T04:22:32.593 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-05-09T04:22:32.593 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-05-09T04:22:32.593 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-05-09T04:22:32.593 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-05-09T04:22:32.593 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-05-09T04:22:32.593 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-05-09T04:22:32.594 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-05-09T04:22:32.594 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-05-09T04:22:32.594 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-05-09T04:22:32.594 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-05-09T04:22:32.594 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-05-09T04:22:32.594 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-05-09T04:22:32.594 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-05-09T04:22:32.594 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-05-09T04:22:32.594 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-05-09T04:22:32.594 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-05-09T04:22:32.594 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-05-09T04:22:32.594 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-05-09T04:22:32.594 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-05-09T04:22:32.594 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-05-09T04:22:32.594 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-05-09T04:22:32.595 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-05-09T04:22:32.595 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-05-09T04:22:32.595 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-05-09T04:22:32.595 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-05-09T04:22:32.595 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-05-09T04:22:32.595 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-05-09T04:22:32.595 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-05-09T04:22:32.595 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-05-09T04:22:32.595 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-05-09T04:22:32.595 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-05-09T04:22:32.595 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-05-09T04:22:32.595 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-05-09T04:22:32.595 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-05-09T04:22:32.595 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-05-09T04:22:32.595 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-05-09T04:22:32.596 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-05-09T04:22:32.596 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-05-09T04:22:32.596 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-05-09T04:22:32.596 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-05-09T04:22:32.596 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-05-09T04:22:32.596 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-05-09T04:22:32.596 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-05-09T04:22:32.596 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-05-09T04:22:32.596 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-05-09T04:22:32.596 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-05-09T04:22:32.596 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-05-09T04:22:32.596 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-05-09T04:22:32.596 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-05-09T04:22:32.596 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-05-09T04:22:32.596 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-05-09T04:22:32.596 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-05-09T04:22:32.597 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-05-09T04:22:32.597 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-05-09T04:22:32.597 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-05-09T04:22:32.597 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-05-09T04:22:32.597 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-05-09T04:22:32.597 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-05-09T04:22:32.597 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-05-09T04:22:32.597 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-05-09T04:22:32.597 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-05-09T04:22:32.597 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-05-09T04:22:32.597 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-05-09T04:22:32.597 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-05-09T04:22:32.597 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-05-09T04:22:32.597 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-05-09T04:22:32.597 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-05-09T04:22:32.597 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-05-09T04:22:32.598 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-05-09T04:22:32.598 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-05-09T04:22:32.598 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-05-09T04:22:32.598 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-05-09T04:22:32.598 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-05-09T04:22:32.598 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-05-09T04:22:32.598 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-05-09T04:22:32.598 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-05-09T04:22:32.598 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-05-09T04:22:32.598 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-05-09T04:22:32.598 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-05-09T04:22:32.598 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-05-09T04:22:32.598 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-05-09T04:22:32.598 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-05-09T04:22:32.598 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-05-09T04:22:32.599 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-05-09T04:22:32.599 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-05-09T04:22:32.599 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-05-09T04:22:32.599 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-05-09T04:22:32.599 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-05-09T04:22:32.599 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-05-09T04:22:32.599 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-05-09T04:22:32.599 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-05-09T04:22:32.599 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-05-09T04:22:32.599 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-05-09T04:22:32.599 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-05-09T04:22:32.599 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-05-09T04:22:32.599 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-05-09T04:22:32.599 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-05-09T04:22:32.600 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-05-09T04:22:32.600 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-05-09T04:22:32.600 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-05-09T04:22:32.600 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-05-09T04:22:32.600 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-05-09T04:22:32.600 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-05-09T04:22:32.600 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-05-09T04:22:32.600 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-05-09T04:22:32.600 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-05-09T04:22:32.600 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-05-09T04:22:32.600 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-05-09T04:22:32.600 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-05-09T04:22:32.600 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-05-09T04:22:32.601 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-05-09T04:22:32.601 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-05-09T04:22:32.601 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-05-09T04:22:32.601 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-05-09T04:22:32.601 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-05-09T04:22:32.601 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-05-09T04:22:32.601 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-05-09T04:22:32.601 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-05-09T04:22:32.601 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-05-09T04:22:32.601 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-05-09T04:22:32.601 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-05-09T04:22:32.601 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-05-09T04:22:32.601 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-05-09T04:22:32.601 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-05-09T04:22:32.601 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-05-09T04:22:32.602 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-05-09T04:22:32.602 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-05-09T04:22:32.602 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-05-09T04:22:32.602 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-05-09T04:22:32.602 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-05-09T04:22:32.602 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-05-09T04:22:32.602 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-05-09T04:22:32.602 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-05-09T04:22:32.602 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-05-09T04:22:32.602 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-05-09T04:22:32.602 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-05-09T04:22:32.602 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-05-09T04:22:32.602 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-05-09T04:22:32.602 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-05-09T04:22:32.602 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-05-09T04:22:32.603 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-05-09T04:22:32.603 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-05-09T04:22:32.603 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-05-09T04:22:32.603 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-05-09T04:22:32.603 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-05-09T04:22:32.603 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-05-09T04:22:32.603 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-05-09T04:22:32.603 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-05-09T04:22:32.603 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-05-09T04:22:32.603 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-05-09T04:22:32.603 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-05-09T04:22:32.603 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-05-09T04:22:32.603 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-05-09T04:22:32.603 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-05-09T04:22:32.603 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-05-09T04:22:32.603 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-05-09T04:22:32.604 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-05-09T04:22:32.604 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-05-09T04:22:32.604 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-05-09T04:22:32.604 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-05-09T04:22:32.604 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-05-09T04:22:32.604 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-05-09T04:22:32.604 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-05-09T04:22:32.604 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-05-09T04:22:32.604 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-05-09T04:22:32.604 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-05-09T04:22:32.604 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-05-09T04:22:32.604 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-05-09T04:22:32.604 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-05-09T04:22:32.604 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-05-09T04:22:32.605 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-05-09T04:22:32.605 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-05-09T04:22:32.605 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-05-09T04:22:32.605 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-05-09T04:22:32.605 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-05-09T04:22:32.605 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-05-09T04:22:32.605 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-05-09T04:22:32.605 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-05-09T04:22:32.605 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-05-09T04:22:32.605 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-05-09T04:22:32.605 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-05-09T04:22:32.605 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-05-09T04:22:32.605 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-05-09T04:22:32.605 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-05-09T04:22:32.605 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-05-09T04:22:32.605 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-05-09T04:22:32.606 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-05-09T04:22:32.606 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-05-09T04:22:32.606 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-05-09T04:22:32.606 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-05-09T04:22:32.606 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-05-09T04:22:32.606 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-05-09T04:22:32.606 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-05-09T04:22:32.606 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-05-09T04:22:32.606 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-05-09T04:22:32.606 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-05-09T04:22:32.606 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-05-09T04:22:32.606 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-05-09T04:22:32.606 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-05-09T04:22:32.606 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-05-09T04:22:32.607 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-05-09T04:22:32.607 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-05-09T04:22:32.607 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-05-09T04:22:32.607 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-05-09T04:22:32.607 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-05-09T04:22:32.607 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-05-09T04:22:32.607 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-05-09T04:22:32.607 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-05-09T04:22:32.607 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-05-09T04:22:32.607 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-05-09T04:22:32.607 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-05-09T04:22:32.607 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-05-09T04:22:32.607 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-05-09T04:22:32.607 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-05-09T04:22:32.607 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-05-09T04:22:32.607 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-05-09T04:22:32.608 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-05-09T04:22:32.608 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-05-09T04:22:32.608 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-05-09T04:22:32.608 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-05-09T04:22:32.608 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-05-09T04:22:32.608 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-05-09T04:22:32.608 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-05-09T04:22:32.608 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-05-09T04:22:32.608 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-05-09T04:22:32.608 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-05-09T04:22:32.608 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-05-09T04:22:32.608 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-05-09T04:22:32.608 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-05-09T04:22:32.608 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-05-09T04:22:32.608 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-05-09T04:22:32.609 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-05-09T04:22:32.609 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-05-09T04:22:32.609 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-05-09T04:22:32.609 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-05-09T04:22:32.609 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-05-09T04:22:32.609 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-05-09T04:22:32.609 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-05-09T04:22:32.609 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-05-09T04:22:32.609 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-05-09T04:22:32.609 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-05-09T04:22:32.609 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-05-09T04:22:32.609 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-05-09T04:22:32.609 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-05-09T04:22:32.609 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-05-09T04:22:32.609 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-05-09T04:22:32.610 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-05-09T04:22:32.610 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-05-09T04:22:32.610 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-05-09T04:22:32.610 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-05-09T04:22:32.610 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-05-09T04:22:32.610 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-05-09T04:22:32.610 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-05-09T04:22:32.610 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-05-09T04:22:32.610 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-05-09T04:22:32.610 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-05-09T04:22:32.610 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-05-09T04:22:32.610 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-05-09T04:22:32.610 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-05-09T04:22:32.610 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-05-09T04:22:32.611 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-05-09T04:22:32.611 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-05-09T04:22:32.611 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-05-09T04:22:32.611 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-05-09T04:22:32.611 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-05-09T04:22:32.611 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-05-09T04:22:32.611 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-05-09T04:22:32.611 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-05-09T04:22:32.611 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-05-09T04:22:32.611 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-05-09T04:22:32.611 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-05-09T04:22:32.611 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-05-09T04:22:32.611 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-05-09T04:22:32.611 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-05-09T04:22:32.611 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-05-09T04:22:32.612 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-05-09T04:22:32.612 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-05-09T04:22:32.612 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-05-09T04:22:32.612 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-05-09T04:22:32.612 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-05-09T04:22:32.612 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-05-09T04:22:32.612 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-05-09T04:22:32.612 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-05-09T04:22:32.612 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-05-09T04:22:32.612 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-05-09T04:22:32.612 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-05-09T04:22:32.612 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-05-09T04:22:32.612 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-05-09T04:22:32.612 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-05-09T04:22:32.613 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-05-09T04:22:32.613 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-05-09T04:22:32.613 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-05-09T04:22:32.613 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-05-09T04:22:32.613 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-05-09T04:22:32.613 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-05-09T04:22:32.613 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-05-09T04:22:32.613 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-05-09T04:22:32.613 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-05-09T04:22:32.613 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-05-09T04:22:32.613 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-05-09T04:22:32.613 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-05-09T04:22:32.613 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-05-09T04:22:32.613 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-05-09T04:22:32.614 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-05-09T04:22:32.614 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-05-09T04:22:32.614 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-05-09T04:22:32.614 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-05-09T04:22:32.614 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-05-09T04:22:32.614 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-05-09T04:22:32.614 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-05-09T04:22:32.614 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-05-09T04:22:32.614 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-05-09T04:22:32.614 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-05-09T04:22:32.614 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-05-09T04:22:32.614 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-05-09T04:22:32.614 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-05-09T04:22:32.614 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-05-09T04:22:32.614 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-05-09T04:22:32.615 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-05-09T04:22:32.615 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-05-09T04:22:32.615 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-05-09T04:22:32.615 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-05-09T04:22:32.615 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-05-09T04:22:32.615 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-05-09T04:22:32.615 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-05-09T04:22:32.615 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-05-09T04:22:32.615 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-05-09T04:22:32.615 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-05-09T04:22:32.615 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-05-09T04:22:32.615 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-05-09T04:22:32.615 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-05-09T04:22:32.615 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-05-09T04:22:32.616 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-05-09T04:22:32.616 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-05-09T04:22:32.616 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-05-09T04:22:32.616 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-05-09T04:22:32.616 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-05-09T04:22:32.616 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-05-09T04:22:32.616 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-05-09T04:22:32.616 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-05-09T04:22:32.616 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-05-09T04:22:32.616 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-05-09T04:22:32.616 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-05-09T04:22:32.616 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-05-09T04:22:32.616 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-05-09T04:22:32.617 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-05-09T04:22:32.617 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-05-09T04:22:32.617 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-05-09T04:22:32.617 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-05-09T04:22:32.617 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-05-09T04:22:32.617 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-05-09T04:22:32.617 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-05-09T04:22:32.617 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-05-09T04:22:32.617 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-05-09T04:22:32.617 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-05-09T04:22:32.617 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-05-09T04:22:32.617 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-05-09T04:22:32.617 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-05-09T04:22:32.617 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-05-09T04:22:32.618 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-05-09T04:22:32.618 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-05-09T04:22:32.618 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-05-09T04:22:32.618 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-05-09T04:22:32.618 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-05-09T04:22:32.618 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-05-09T04:22:32.618 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-05-09T04:22:32.618 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-05-09T04:22:32.618 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-05-09T04:22:32.618 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-05-09T04:22:32.618 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-05-09T04:22:32.618 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-05-09T04:22:32.618 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-05-09T04:22:32.618 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-05-09T04:22:32.618 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-05-09T04:22:32.619 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-05-09T04:22:32.619 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-05-09T04:22:32.619 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-05-09T04:22:32.619 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-05-09T04:22:32.619 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-05-09T04:22:32.619 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-05-09T04:22:32.619 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-05-09T04:22:32.619 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-05-09T04:22:32.619 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-05-09T04:22:32.619 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-05-09T04:22:32.619 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-05-09T04:22:32.619 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-05-09T04:22:32.619 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-05-09T04:22:32.619 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-05-09T04:22:32.620 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-05-09T04:22:32.620 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-05-09T04:22:32.620 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-05-09T04:22:32.620 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-05-09T04:22:32.620 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-05-09T04:22:32.620 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-05-09T04:22:32.620 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-05-09T04:22:32.620 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-05-09T04:22:32.620 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-05-09T04:22:32.620 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-05-09T04:22:32.620 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-05-09T04:22:32.620 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-05-09T04:22:32.620 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-05-09T04:22:32.620 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-05-09T04:22:32.620 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-05-09T04:22:32.620 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-05-09T04:22:32.621 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-05-09T04:22:32.621 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-05-09T04:22:32.621 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-05-09T04:22:32.621 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-05-09T04:22:32.621 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-05-09T04:22:32.621 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-05-09T04:22:32.621 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-05-09T04:22:32.621 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-05-09T04:22:32.621 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-05-09T04:22:32.621 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-05-09T04:22:32.621 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-05-09T04:22:32.621 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-05-09T04:22:32.621 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-05-09T04:22:32.621 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-05-09T04:22:32.622 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-05-09T04:22:32.622 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-05-09T04:22:32.622 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-05-09T04:22:32.622 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-05-09T04:22:32.622 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-05-09T04:22:32.622 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-05-09T04:22:32.622 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-05-09T04:22:32.622 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-05-09T04:22:32.622 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-05-09T04:22:32.622 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-05-09T04:22:32.622 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-05-09T04:22:32.622 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-05-09T04:22:32.622 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-05-09T04:22:32.622 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-05-09T04:22:32.623 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-05-09T04:22:32.623 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-05-09T04:22:32.623 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-05-09T04:22:32.623 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-05-09T04:22:32.623 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-05-09T04:22:32.623 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-05-09T04:22:32.623 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-05-09T04:22:32.623 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-05-09T04:22:32.623 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-05-09T04:22:32.623 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-05-09T04:22:32.623 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-05-09T04:22:32.623 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-05-09T04:22:32.623 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-05-09T04:22:32.623 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-05-09T04:22:32.623 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-05-09T04:22:32.624 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-05-09T04:22:32.624 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-05-09T04:22:32.624 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-05-09T04:22:32.624 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-05-09T04:22:32.624 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-05-09T04:22:32.624 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-05-09T04:22:32.624 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-05-09T04:22:32.624 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-05-09T04:22:32.624 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-05-09T04:22:32.624 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-05-09T04:22:32.624 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-05-09T04:22:32.624 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-05-09T04:22:32.624 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-05-09T04:22:32.624 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-05-09T04:22:32.624 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-05-09T04:22:32.625 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-05-09T04:22:32.625 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-05-09T04:22:32.625 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-05-09T04:22:32.625 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-05-09T04:22:32.625 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-05-09T04:22:32.625 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-05-09T04:22:32.625 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-05-09T04:22:32.625 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-05-09T04:22:32.625 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-05-09T04:22:32.625 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-05-09T04:22:32.625 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-05-09T04:22:32.625 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-05-09T04:22:32.625 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-05-09T04:22:32.625 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-05-09T04:22:32.625 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-05-09T04:22:32.626 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-05-09T04:22:32.626 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-05-09T04:22:32.626 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-05-09T04:22:32.626 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-05-09T04:22:32.626 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-05-09T04:22:32.626 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-05-09T04:22:32.626 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-05-09T04:22:32.626 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-05-09T04:22:32.626 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-05-09T04:22:32.626 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-05-09T04:22:32.626 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-05-09T04:22:32.626 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-05-09T04:22:32.626 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-05-09T04:22:32.626 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-05-09T04:22:32.626 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-05-09T04:22:32.627 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-05-09T04:22:32.627 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-05-09T04:22:32.627 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-05-09T04:22:32.627 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-05-09T04:22:32.627 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-05-09T04:22:32.627 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-05-09T04:22:32.627 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-05-09T04:22:32.627 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-05-09T04:22:32.627 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-05-09T04:22:32.627 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-05-09T04:22:32.627 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-05-09T04:22:32.627 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-05-09T04:22:32.627 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-05-09T04:22:32.627 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-05-09T04:22:32.627 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-05-09T04:22:32.627 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-05-09T04:22:32.628 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-05-09T04:22:32.628 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-05-09T04:22:32.628 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-05-09T04:22:32.628 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-05-09T04:22:32.628 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-05-09T04:22:32.628 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-05-09T04:22:32.628 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-05-09T04:22:32.628 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-05-09T04:22:32.628 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-05-09T04:22:32.628 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-05-09T04:22:32.628 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-05-09T04:22:32.628 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-05-09T04:22:32.628 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-05-09T04:22:32.628 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-05-09T04:22:32.628 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-05-09T04:22:32.628 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-05-09T04:22:32.629 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-05-09T04:22:32.629 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-05-09T04:22:32.629 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-05-09T04:22:32.629 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-05-09T04:22:32.629 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-05-09T04:22:32.629 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-05-09T04:22:32.629 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-05-09T04:22:32.629 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-05-09T04:22:32.629 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-05-09T04:22:32.629 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-05-09T04:22:32.629 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-05-09T04:22:32.629 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-05-09T04:22:32.629 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-05-09T04:22:32.629 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-05-09T04:22:32.629 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-05-09T04:22:32.629 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-05-09T04:22:32.630 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-05-09T04:22:32.630 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-05-09T04:22:32.630 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-05-09T04:22:32.630 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-05-09T04:22:32.630 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-05-09T04:22:32.630 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-05-09T04:22:32.630 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-05-09T04:22:32.630 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-05-09T04:22:32.630 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-05-09T04:22:32.630 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-05-09T04:22:32.630 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-05-09T04:22:32.630 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-05-09T04:22:32.630 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-05-09T04:22:32.630 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-05-09T04:22:32.631 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-05-09T04:22:32.631 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-05-09T04:22:32.631 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-05-09T04:22:32.631 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-05-09T04:22:32.631 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-05-09T04:22:32.631 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-05-09T04:22:32.631 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-05-09T04:22:32.631 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-05-09T04:22:32.631 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-05-09T04:22:32.631 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-05-09T04:22:32.631 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-05-09T04:22:32.631 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-05-09T04:22:32.631 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-05-09T04:22:32.631 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-05-09T04:22:32.631 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-05-09T04:22:32.632 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-05-09T04:22:32.632 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-05-09T04:22:32.632 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-05-09T04:22:32.632 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-05-09T04:22:32.632 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-05-09T04:22:32.632 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-05-09T04:22:32.632 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-05-09T04:22:32.632 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-05-09T04:22:32.632 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-05-09T04:22:32.632 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-05-09T04:22:32.632 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-05-09T04:22:32.632 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-05-09T04:22:32.632 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-05-09T04:22:32.632 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-05-09T04:22:32.632 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-05-09T04:22:32.633 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-05-09T04:22:32.633 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-05-09T04:22:32.633 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-05-09T04:22:32.633 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-05-09T04:22:32.633 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-05-09T04:22:32.633 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-05-09T04:22:32.633 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-05-09T04:22:32.633 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-05-09T04:22:32.633 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-05-09T04:22:32.633 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-05-09T04:22:32.633 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-05-09T04:22:32.633 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-05-09T04:22:32.633 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-05-09T04:22:32.633 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-05-09T04:22:32.633 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-05-09T04:22:32.634 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-05-09T04:22:32.634 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-05-09T04:22:32.634 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-05-09T04:22:32.634 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-05-09T04:22:32.634 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-05-09T04:22:32.634 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-05-09T04:22:32.634 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-05-09T04:22:32.634 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-05-09T04:22:32.634 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-05-09T04:22:32.634 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-05-09T04:22:32.634 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-05-09T04:22:32.634 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-05-09T04:22:32.634 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-05-09T04:22:32.634 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-05-09T04:22:32.634 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-05-09T04:22:32.635 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-05-09T04:22:32.635 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-05-09T04:22:32.635 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-05-09T04:22:32.635 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-05-09T04:22:32.635 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-05-09T04:22:32.635 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-05-09T04:22:32.635 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-05-09T04:22:32.635 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-05-09T04:22:32.635 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-05-09T04:22:32.635 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-05-09T04:22:32.635 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-05-09T04:22:32.635 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-05-09T04:22:32.635 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-05-09T04:22:32.635 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-05-09T04:22:32.636 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-05-09T04:22:32.636 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-05-09T04:22:32.636 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-05-09T04:22:32.636 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-05-09T04:22:32.636 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-05-09T04:22:32.636 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-05-09T04:22:32.636 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-05-09T04:22:32.636 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-05-09T04:22:32.636 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-05-09T04:22:32.636 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-05-09T04:22:32.636 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-05-09T04:22:32.636 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-05-09T04:22:32.636 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-05-09T04:22:32.636 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-05-09T04:22:32.636 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-05-09T04:22:32.637 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-05-09T04:22:32.637 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-05-09T04:22:32.637 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-05-09T04:22:32.637 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-05-09T04:22:32.637 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-05-09T04:22:32.637 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-05-09T04:22:32.637 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-05-09T04:22:32.637 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-05-09T04:22:32.637 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-05-09T04:22:32.637 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-05-09T04:22:32.637 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-05-09T04:22:32.637 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-05-09T04:22:32.637 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-05-09T04:22:32.637 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-05-09T04:22:32.638 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-05-09T04:22:32.638 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-05-09T04:22:32.638 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-05-09T04:22:32.638 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-05-09T04:22:32.638 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-05-09T04:22:32.638 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-05-09T04:22:32.638 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-05-09T04:22:32.638 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-05-09T04:22:32.638 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-05-09T04:22:32.638 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-05-09T04:22:32.638 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-05-09T04:22:32.638 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-05-09T04:22:32.638 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-05-09T04:22:32.638 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-05-09T04:22:32.639 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-05-09T04:22:32.639 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-05-09T04:22:32.639 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-05-09T04:22:32.639 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-05-09T04:22:32.639 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-05-09T04:22:32.639 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-05-09T04:22:32.639 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-05-09T04:22:32.639 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-05-09T04:22:32.639 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-05-09T04:22:32.639 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-05-09T04:22:32.639 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-05-09T04:22:32.639 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-05-09T04:22:32.639 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-05-09T04:22:32.639 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-05-09T04:22:32.640 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-05-09T04:22:32.640 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-05-09T04:22:32.640 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-05-09T04:22:32.640 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-05-09T04:22:32.640 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-05-09T04:22:32.640 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-05-09T04:22:32.640 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-05-09T04:22:32.640 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-05-09T04:22:32.640 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-05-09T04:22:32.640 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-05-09T04:22:32.640 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-05-09T04:22:32.640 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-05-09T04:22:32.640 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-05-09T04:22:32.640 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-05-09T04:22:32.640 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-05-09T04:22:32.640 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-05-09T04:22:32.641 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-05-09T04:22:32.641 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-05-09T04:22:32.641 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-05-09T04:22:32.641 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-05-09T04:22:32.641 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-05-09T04:22:32.641 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-05-09T04:22:32.641 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-05-09T04:22:32.641 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-05-09T04:22:32.641 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-05-09T04:22:32.641 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-05-09T04:22:32.641 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-05-09T04:22:32.641 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-05-09T04:22:32.641 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-05-09T04:22:32.641 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-05-09T04:22:32.642 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-05-09T04:22:32.642 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-05-09T04:22:32.642 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-05-09T04:22:32.642 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-05-09T04:22:32.642 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-05-09T04:22:32.642 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-05-09T04:22:32.642 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-05-09T04:22:32.642 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-05-09T04:22:32.642 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-05-09T04:22:32.642 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-05-09T04:22:32.642 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-05-09T04:22:32.642 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-05-09T04:22:32.642 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-05-09T04:22:32.642 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-05-09T04:22:32.643 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-05-09T04:22:32.643 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-05-09T04:22:32.643 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-05-09T04:22:32.643 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-05-09T04:22:32.643 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-05-09T04:22:32.643 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-05-09T04:22:32.643 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-05-09T04:22:32.643 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-05-09T04:22:32.643 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-05-09T04:22:32.643 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-05-09T04:22:32.643 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-05-09T04:22:32.643 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-05-09T04:22:32.643 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-05-09T04:22:32.643 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-05-09T04:22:32.643 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-05-09T04:22:32.643 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-05-09T04:22:32.644 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-05-09T04:22:32.644 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-05-09T04:22:32.644 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-05-09T04:22:32.644 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-05-09T04:22:32.644 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-05-09T04:22:32.644 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-05-09T04:22:32.644 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-05-09T04:22:32.644 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-05-09T04:22:32.644 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-05-09T04:22:32.644 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-05-09T04:22:32.644 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-05-09T04:22:32.644 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-05-09T04:22:32.644 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-05-09T04:22:32.644 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-05-09T04:22:32.644 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-05-09T04:22:32.644 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-05-09T04:22:32.644 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-05-09T04:22:32.645 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-05-09T04:22:32.645 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-05-09T04:22:32.645 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-05-09T04:22:32.645 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-05-09T04:22:32.645 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-05-09T04:22:32.645 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-05-09T04:22:32.645 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-05-09T04:22:32.645 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-05-09T04:22:32.645 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-05-09T04:22:32.645 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-05-09T04:22:32.645 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-05-09T04:22:32.645 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-05-09T04:22:32.645 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-05-09T04:22:32.645 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-05-09T04:22:32.646 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-05-09T04:22:32.646 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-05-09T04:22:32.646 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-05-09T04:22:32.646 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-05-09T04:22:32.646 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-05-09T04:22:32.646 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-05-09T04:22:32.646 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-05-09T04:22:32.646 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-05-09T04:22:32.646 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-05-09T04:22:32.646 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-05-09T04:22:32.646 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-05-09T04:22:32.646 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-05-09T04:22:32.647 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-05-09T04:22:32.647 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-05-09T04:22:32.647 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-05-09T04:22:32.647 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-05-09T04:22:32.647 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-05-09T04:22:32.647 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-05-09T04:22:32.647 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-05-09T04:22:32.647 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-05-09T04:22:32.647 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-05-09T04:22:32.647 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-05-09T04:22:32.647 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-05-09T04:22:32.647 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-05-09T04:22:32.647 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-05-09T04:22:32.647 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-05-09T04:22:32.647 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-05-09T04:22:32.648 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-05-09T04:22:32.648 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-05-09T04:22:32.648 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-05-09T04:22:32.648 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-05-09T04:22:32.648 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-05-09T04:22:32.648 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-05-09T04:22:32.648 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-05-09T04:22:32.648 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-05-09T04:22:32.648 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-05-09T04:22:32.648 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-05-09T04:22:32.648 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-05-09T04:22:32.648 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-05-09T04:22:32.648 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-05-09T04:22:32.648 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-05-09T04:22:32.648 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-05-09T04:22:32.649 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-05-09T04:22:32.649 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-05-09T04:22:32.649 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-05-09T04:22:32.649 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-05-09T04:22:32.649 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-05-09T04:22:32.649 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-05-09T04:22:32.649 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-05-09T04:22:32.649 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-05-09T04:22:32.649 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-05-09T04:22:32.649 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-05-09T04:22:32.649 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-05-09T04:22:32.649 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-05-09T04:22:32.649 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-05-09T04:22:32.649 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-05-09T04:22:32.649 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-05-09T04:22:32.650 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-05-09T04:22:32.650 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-05-09T04:22:32.650 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-05-09T04:22:32.650 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-05-09T04:22:32.650 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-05-09T04:22:32.650 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-05-09T04:22:32.650 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-05-09T04:22:32.650 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-05-09T04:22:32.650 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-05-09T04:22:32.650 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-05-09T04:22:32.650 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-05-09T04:22:32.650 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-05-09T04:22:32.650 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-05-09T04:22:32.650 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-05-09T04:22:32.650 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-05-09T04:22:32.651 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-05-09T04:22:32.651 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-05-09T04:22:32.651 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-05-09T04:22:32.651 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-05-09T04:22:32.651 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-05-09T04:22:32.651 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-05-09T04:22:32.651 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-05-09T04:22:32.651 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-05-09T04:22:32.651 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-05-09T04:22:32.651 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-05-09T04:22:32.651 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-05-09T04:22:32.651 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-05-09T04:22:32.651 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-05-09T04:22:32.651 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-05-09T04:22:32.651 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-05-09T04:22:32.651 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-05-09T04:22:32.652 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-05-09T04:22:32.652 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-05-09T04:22:32.652 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-05-09T04:22:32.652 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-05-09T04:22:32.652 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-05-09T04:22:32.652 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-05-09T04:22:32.652 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-05-09T04:22:32.652 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-05-09T04:22:32.652 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-05-09T04:22:32.652 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-05-09T04:22:32.652 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-05-09T04:22:32.652 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-05-09T04:22:32.652 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-05-09T04:22:32.652 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-05-09T04:22:32.652 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-05-09T04:22:32.653 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-05-09T04:22:32.653 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-05-09T04:22:32.653 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-05-09T04:22:32.653 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-05-09T04:22:32.653 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-05-09T04:22:32.653 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-05-09T04:22:32.653 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-05-09T04:22:32.653 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-05-09T04:22:32.653 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-05-09T04:22:32.653 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-05-09T04:22:32.653 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-05-09T04:22:32.653 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-05-09T04:22:32.653 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-05-09T04:22:32.653 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-05-09T04:22:32.654 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-05-09T04:22:32.654 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-05-09T04:22:32.654 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-05-09T04:22:32.654 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-05-09T04:22:32.654 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-05-09T04:22:32.654 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-05-09T04:22:32.654 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-05-09T04:22:32.654 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-05-09T04:22:32.654 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-05-09T04:22:32.654 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-05-09T04:22:32.654 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-05-09T04:22:32.654 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-05-09T04:22:32.654 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-05-09T04:22:32.654 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-05-09T04:22:32.655 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-05-09T04:22:32.655 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-05-09T04:22:32.655 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-05-09T04:22:32.655 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-05-09T04:22:32.655 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-05-09T04:22:32.655 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-05-09T04:22:32.655 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-05-09T04:22:32.655 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-05-09T04:22:32.655 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-05-09T04:22:32.655 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-05-09T04:22:32.655 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-05-09T04:22:32.655 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-05-09T04:22:32.655 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-05-09T04:22:32.655 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-05-09T04:22:32.655 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-05-09T04:22:32.655 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-05-09T04:22:32.656 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-05-09T04:22:32.656 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-05-09T04:22:32.656 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-05-09T04:22:32.656 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-05-09T04:22:32.656 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-05-09T04:22:32.656 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-05-09T04:22:32.656 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-05-09T04:22:32.656 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-05-09T04:22:32.656 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-05-09T04:22:32.656 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-05-09T04:22:32.656 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-05-09T04:22:32.656 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-05-09T04:22:32.656 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-05-09T04:22:32.656 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-05-09T04:22:32.656 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-05-09T04:22:32.657 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-05-09T04:22:32.657 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-05-09T04:22:32.657 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-05-09T04:22:32.657 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-05-09T04:22:32.657 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-05-09T04:22:32.657 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-05-09T04:22:32.657 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-05-09T04:22:32.657 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-05-09T04:22:32.657 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-05-09T04:22:32.657 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-05-09T04:22:32.657 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-05-09T04:22:32.657 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-05-09T04:22:32.658 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-05-09T04:22:32.658 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-05-09T04:22:32.658 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-05-09T04:22:32.658 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-05-09T04:22:32.658 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-05-09T04:22:32.658 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-05-09T04:22:32.658 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-05-09T04:22:32.658 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-05-09T04:22:32.658 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-05-09T04:22:32.658 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-05-09T04:22:32.658 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-05-09T04:22:32.658 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-05-09T04:22:32.659 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-05-09T04:22:32.659 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-05-09T04:22:32.659 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-05-09T04:22:32.659 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-05-09T04:22:32.659 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-05-09T04:22:32.659 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-05-09T04:22:32.659 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-05-09T04:22:32.659 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-05-09T04:22:32.659 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-05-09T04:22:32.659 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-05-09T04:22:32.659 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-05-09T04:22:32.659 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-05-09T04:22:32.659 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-05-09T04:22:32.659 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-05-09T04:22:32.659 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-05-09T04:22:32.660 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-05-09T04:22:32.660 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-05-09T04:22:32.660 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-05-09T04:22:32.660 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-05-09T04:22:32.660 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-05-09T04:22:32.660 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-05-09T04:22:32.660 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-05-09T04:22:32.660 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-05-09T04:22:32.660 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-05-09T04:22:32.660 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-05-09T04:22:32.660 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-05-09T04:22:32.660 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-05-09T04:22:32.660 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-05-09T04:22:32.660 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-05-09T04:22:32.661 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-05-09T04:22:32.661 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-05-09T04:22:32.661 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-05-09T04:22:32.661 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-05-09T04:22:32.661 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-05-09T04:22:32.661 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-05-09T04:22:32.661 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-05-09T04:22:32.661 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-05-09T04:22:32.661 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-05-09T04:22:32.661 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-05-09T04:22:32.661 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-05-09T04:22:32.661 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-05-09T04:22:32.661 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-05-09T04:22:32.661 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-05-09T04:22:32.661 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-05-09T04:22:32.662 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-05-09T04:22:32.662 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-05-09T04:22:32.662 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-05-09T04:22:32.662 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-05-09T04:22:32.662 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-05-09T04:22:32.662 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-05-09T04:22:32.662 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-05-09T04:22:32.662 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-05-09T04:22:32.662 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-05-09T04:22:32.662 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-05-09T04:22:32.662 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-05-09T04:22:32.662 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-05-09T04:22:32.662 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-05-09T04:22:32.662 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-05-09T04:22:32.663 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-05-09T04:22:32.663 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-05-09T04:22:32.663 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-05-09T04:22:32.663 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-05-09T04:22:32.663 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-05-09T04:22:32.663 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-05-09T04:22:32.663 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-05-09T04:22:32.663 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-05-09T04:22:32.663 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-05-09T04:22:32.663 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-05-09T04:22:32.663 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-05-09T04:22:32.663 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-05-09T04:22:32.663 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-05-09T04:22:32.663 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-05-09T04:22:32.663 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-05-09T04:22:32.663 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-05-09T04:22:32.664 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-05-09T04:22:32.664 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-05-09T04:22:32.664 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-05-09T04:22:32.664 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-05-09T04:22:32.664 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-05-09T04:22:32.664 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-05-09T04:22:32.664 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-05-09T04:22:32.664 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-05-09T04:22:32.664 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-05-09T04:22:32.664 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-05-09T04:22:32.664 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-05-09T04:22:32.664 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-05-09T04:22:32.664 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-05-09T04:22:32.664 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-05-09T04:22:32.665 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-05-09T04:22:32.665 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-05-09T04:22:32.665 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-05-09T04:22:32.665 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-05-09T04:22:32.665 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-05-09T04:22:32.665 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-05-09T04:22:32.665 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-05-09T04:22:32.665 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-05-09T04:22:32.665 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-05-09T04:22:32.665 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-05-09T04:22:32.665 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-05-09T04:22:32.665 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-05-09T04:22:32.665 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-05-09T04:22:32.665 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-05-09T04:22:32.666 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-05-09T04:22:32.666 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-05-09T04:22:32.666 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-05-09T04:22:32.666 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-05-09T04:22:32.666 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-05-09T04:22:32.666 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-05-09T04:22:32.666 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-05-09T04:22:32.666 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-05-09T04:22:32.666 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-05-09T04:22:32.666 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-05-09T04:22:32.666 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-05-09T04:22:32.666 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-05-09T04:22:32.667 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-05-09T04:22:32.667 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-05-09T04:22:32.667 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-05-09T04:22:32.667 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-05-09T04:22:32.667 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-05-09T04:22:32.667 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-05-09T04:22:32.667 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-05-09T04:22:32.667 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-05-09T04:22:32.667 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-05-09T04:22:32.667 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-05-09T04:22:32.667 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-05-09T04:22:32.667 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-05-09T04:22:32.667 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-05-09T04:22:32.667 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-05-09T04:22:32.667 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-05-09T04:22:32.667 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-05-09T04:22:32.668 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-05-09T04:22:32.668 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-05-09T04:22:32.668 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-05-09T04:22:32.668 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-05-09T04:22:32.668 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-05-09T04:22:32.668 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-05-09T04:22:32.668 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-05-09T04:22:32.668 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-05-09T04:22:32.668 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-05-09T04:22:32.668 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-05-09T04:22:32.668 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-05-09T04:22:32.668 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-05-09T04:22:32.668 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-05-09T04:22:32.668 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-05-09T04:22:32.668 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-05-09T04:22:32.669 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-05-09T04:22:32.669 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-05-09T04:22:32.669 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-05-09T04:22:32.669 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-05-09T04:22:32.669 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-05-09T04:22:32.669 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-05-09T04:22:32.669 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-05-09T04:22:32.669 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-05-09T04:22:32.669 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-05-09T04:22:32.669 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-05-09T04:22:32.669 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-05-09T04:22:32.669 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-05-09T04:22:32.669 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-05-09T04:22:32.669 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-05-09T04:22:32.670 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-05-09T04:22:32.670 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-05-09T04:22:32.670 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-05-09T04:22:32.670 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-05-09T04:22:32.670 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-05-09T04:22:32.670 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-05-09T04:22:32.670 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-05-09T04:22:32.670 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-05-09T04:22:32.670 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-05-09T04:22:32.670 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-05-09T04:22:32.670 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-05-09T04:22:32.670 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-05-09T04:22:32.670 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-05-09T04:22:32.670 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-05-09T04:22:32.671 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-05-09T04:22:32.671 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-05-09T04:22:32.671 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-05-09T04:22:32.671 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-05-09T04:22:32.671 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-05-09T04:22:32.671 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-05-09T04:22:32.671 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-05-09T04:22:32.671 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-05-09T04:22:32.671 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-05-09T04:22:32.671 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-05-09T04:22:32.671 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-05-09T04:22:32.671 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-05-09T04:22:32.671 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-05-09T04:22:32.671 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-05-09T04:22:32.671 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-05-09T04:22:32.671 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-05-09T04:22:32.672 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-05-09T04:22:32.672 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-05-09T04:22:32.672 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-05-09T04:22:32.672 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-05-09T04:22:32.672 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-05-09T04:22:32.672 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-05-09T04:22:32.672 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-05-09T04:22:32.672 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-05-09T04:22:32.672 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-05-09T04:22:32.672 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-05-09T04:22:32.672 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-05-09T04:22:32.672 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-05-09T04:22:32.672 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-05-09T04:22:32.672 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-05-09T04:22:32.672 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-05-09T04:22:32.673 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-05-09T04:22:32.673 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-05-09T04:22:32.673 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-05-09T04:22:32.673 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-05-09T04:22:32.673 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-05-09T04:22:32.673 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-05-09T04:22:32.673 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-05-09T04:22:32.673 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-05-09T04:22:32.673 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-05-09T04:22:32.673 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-05-09T04:22:32.673 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-05-09T04:22:32.673 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-05-09T04:22:32.673 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-05-09T04:22:32.673 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-05-09T04:22:32.673 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-05-09T04:22:32.673 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-05-09T04:22:32.674 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-05-09T04:22:32.674 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-05-09T04:22:32.674 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-05-09T04:22:32.674 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-05-09T04:22:32.674 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-05-09T04:22:32.674 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-05-09T04:22:32.674 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-05-09T04:22:32.674 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-05-09T04:22:32.674 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-05-09T04:22:32.674 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-05-09T04:22:32.674 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-05-09T04:22:32.674 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-05-09T04:22:32.674 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-05-09T04:22:32.674 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-05-09T04:22:32.675 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-05-09T04:22:32.675 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-05-09T04:22:32.675 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-05-09T04:22:32.675 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-05-09T04:22:32.675 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-05-09T04:22:32.675 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-05-09T04:22:32.675 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-05-09T04:22:32.675 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-05-09T04:22:32.675 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-05-09T04:22:32.675 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-05-09T04:22:32.675 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-05-09T04:22:32.675 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-05-09T04:22:32.675 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-05-09T04:22:32.675 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-05-09T04:22:32.676 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-05-09T04:22:32.676 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-05-09T04:22:32.676 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-05-09T04:22:32.676 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-05-09T04:22:32.676 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-05-09T04:22:32.676 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-05-09T04:22:32.676 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-05-09T04:22:32.676 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-05-09T04:22:32.676 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-05-09T04:22:32.676 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-05-09T04:22:32.676 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-05-09T04:22:32.676 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-05-09T04:22:32.676 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-05-09T04:22:32.676 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-05-09T04:22:32.676 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-05-09T04:22:32.677 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-05-09T04:22:32.677 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-05-09T04:22:32.677 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-05-09T04:22:32.677 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-05-09T04:22:32.677 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-05-09T04:22:32.677 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-05-09T04:22:32.677 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-05-09T04:22:32.677 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-05-09T04:22:32.677 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-05-09T04:22:32.677 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-05-09T04:22:32.677 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-05-09T04:22:32.677 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-05-09T04:22:32.677 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-05-09T04:22:32.677 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-05-09T04:22:32.677 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-05-09T04:22:32.677 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-05-09T04:22:32.678 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-05-09T04:22:32.678 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-05-09T04:22:32.678 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-05-09T04:22:32.678 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-05-09T04:22:32.678 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-05-09T04:22:32.678 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-05-09T04:22:32.678 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-05-09T04:22:32.678 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-05-09T04:22:32.678 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-05-09T04:22:32.678 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-05-09T04:22:32.678 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-05-09T04:22:32.678 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-05-09T04:22:32.678 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-05-09T04:22:32.678 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-05-09T04:22:32.678 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-05-09T04:22:32.679 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-05-09T04:22:32.679 INFO:teuthology.orchestra.run.smithi145.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-05-09T04:22:32.682 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-05-09T04:22:32.682 DEBUG:teuthology.orchestra.run.smithi145:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-05-09T04:22:32.704 INFO:teuthology.orchestra.run.smithi145.stdout:postinstall scriptlet (using /bin/sh): 2024-05-09T04:22:32.704 INFO:teuthology.orchestra.run.smithi145.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-09T04:22:32.704 INFO:teuthology.orchestra.run.smithi145.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-05-09T04:22:32.704 INFO:teuthology.orchestra.run.smithi145.stdout:fi 2024-05-09T04:22:32.704 INFO:teuthology.orchestra.run.smithi145.stdout:for file in vmlinuz System.map config; do 2024-05-09T04:22:32.704 INFO:teuthology.orchestra.run.smithi145.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-05-09T04:22:32.704 INFO:teuthology.orchestra.run.smithi145.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-05-09T04:22:32.705 INFO:teuthology.orchestra.run.smithi145.stdout: fi 2024-05-09T04:22:32.705 INFO:teuthology.orchestra.run.smithi145.stdout:done 2024-05-09T04:22:32.705 INFO:teuthology.orchestra.run.smithi145.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-05-09T04:22:32.705 INFO:teuthology.orchestra.run.smithi145.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-05-09T04:22:32.705 INFO:teuthology.orchestra.run.smithi145.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-05-09T04:22:32.705 INFO:teuthology.orchestra.run.smithi145.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-05-09T04:22:32.705 INFO:teuthology.orchestra.run.smithi145.stdout:fi 2024-05-09T04:22:32.705 INFO:teuthology.orchestra.run.smithi145.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-05-09T04:22:32.705 INFO:teuthology.orchestra.run.smithi145.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-05-09T04:22:32.705 INFO:teuthology.orchestra.run.smithi145.stdout:fi 2024-05-09T04:22:32.705 INFO:teuthology.orchestra.run.smithi145.stdout:preuninstall scriptlet (using /bin/sh): 2024-05-09T04:22:32.705 INFO:teuthology.orchestra.run.smithi145.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-05-09T04:22:32.705 INFO:teuthology.orchestra.run.smithi145.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-05-09T04:22:32.705 INFO:teuthology.orchestra.run.smithi145.stdout:fi 2024-05-09T04:22:32.706 DEBUG:teuthology.orchestra.run.smithi145:> sudo rpm -qi grub2-tools 2024-05-09T04:22:32.779 INFO:teuthology.orchestra.run.smithi145.stdout:Name : grub2-tools 2024-05-09T04:22:32.779 INFO:teuthology.orchestra.run.smithi145.stdout:Epoch : 1 2024-05-09T04:22:32.779 INFO:teuthology.orchestra.run.smithi145.stdout:Version : 2.06 2024-05-09T04:22:32.779 INFO:teuthology.orchestra.run.smithi145.stdout:Release : 68.el9 2024-05-09T04:22:32.779 INFO:teuthology.orchestra.run.smithi145.stdout:Architecture: x86_64 2024-05-09T04:22:32.779 INFO:teuthology.orchestra.run.smithi145.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-09T04:22:32.779 INFO:teuthology.orchestra.run.smithi145.stdout:Group : Unspecified 2024-05-09T04:22:32.779 INFO:teuthology.orchestra.run.smithi145.stdout:Size : 8264254 2024-05-09T04:22:32.779 INFO:teuthology.orchestra.run.smithi145.stdout:License : GPLv3+ 2024-05-09T04:22:32.779 INFO:teuthology.orchestra.run.smithi145.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-09T04:22:32.779 INFO:teuthology.orchestra.run.smithi145.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-09T04:22:32.779 INFO:teuthology.orchestra.run.smithi145.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-09T04:22:32.780 INFO:teuthology.orchestra.run.smithi145.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-09T04:22:32.780 INFO:teuthology.orchestra.run.smithi145.stdout:Packager : builder@centos.org 2024-05-09T04:22:32.780 INFO:teuthology.orchestra.run.smithi145.stdout:Vendor : CentOS 2024-05-09T04:22:32.780 INFO:teuthology.orchestra.run.smithi145.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-09T04:22:32.780 INFO:teuthology.orchestra.run.smithi145.stdout:Summary : Support tools for GRUB. 2024-05-09T04:22:32.780 INFO:teuthology.orchestra.run.smithi145.stdout:Description : 2024-05-09T04:22:32.780 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:22:32.780 INFO:teuthology.orchestra.run.smithi145.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-09T04:22:32.780 INFO:teuthology.orchestra.run.smithi145.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-09T04:22:32.780 INFO:teuthology.orchestra.run.smithi145.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-09T04:22:32.780 INFO:teuthology.orchestra.run.smithi145.stdout:hardware devices. 2024-05-09T04:22:32.780 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:22:32.780 INFO:teuthology.orchestra.run.smithi145.stdout:This subpackage provides tools for support of all platforms. 2024-05-09T04:22:32.781 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-09T04:22:32.781 INFO:teuthology.task.kernel:Updating grub on smithi145 to boot 6.9.0-rc3-g806ec36b2112 2024-05-09T04:22:32.781 DEBUG:teuthology.orchestra.run.smithi145:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-09T04:22:33.325 INFO:teuthology.orchestra.run.smithi145.stderr:Generating grub configuration file ... 2024-05-09T04:22:34.770 INFO:teuthology.orchestra.run.smithi145.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-09T04:22:34.787 INFO:teuthology.orchestra.run.smithi145.stderr:done 2024-05-09T04:22:34.789 DEBUG:teuthology.orchestra.run.smithi145:> mktemp 2024-05-09T04:22:34.810 INFO:teuthology.orchestra.run.smithi145.stdout:/tmp/tmp.Jojzft52Il 2024-05-09T04:22:34.810 DEBUG:teuthology.orchestra.run.smithi145:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Jojzft52Il 2024-05-09T04:22:34.885 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod 0666 /tmp/tmp.Jojzft52Il 2024-05-09T04:22:35.027 DEBUG:teuthology.orchestra.remote:smithi145:/tmp/tmp.Jojzft52Il is 6KB 2024-05-09T04:22:35.075 DEBUG:teuthology.orchestra.run.smithi145:> rm -fr /tmp/tmp.Jojzft52Il 2024-05-09T04:22:35.089 DEBUG:teuthology.orchestra.run.smithi145:> sudo /bin/ls /boot/loader/entries || true 2024-05-09T04:22:35.150 INFO:teuthology.orchestra.run.smithi145.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-09T04:22:35.150 INFO:teuthology.orchestra.run.smithi145.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-09T04:22:35.150 INFO:teuthology.orchestra.run.smithi145.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-09T04:22:35.150 INFO:teuthology.orchestra.run.smithi145.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-05-09T04:22:35.152 DEBUG:teuthology.orchestra.run.smithi145:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-05-09T04:22:35.289 DEBUG:teuthology.orchestra.run.smithi145:> sudo shutdown -r now 2024-05-09T04:22:51.544 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-05-09T04:22:51.544 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-05-09T04:22:51.546 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T04:23:05.298 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-05-09T04:23:05.299 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com' 2024-05-09T04:23:05.300 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T04:23:10.110 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.192 2024-05-09T04:23:19.120 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-05-09T04:23:19.120 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T04:23:22.174 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.192 2024-05-09T04:23:23.678 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.145 2024-05-09T04:23:32.686 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com' 2024-05-09T04:23:32.687 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T04:23:34.177 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-05-09T04:23:34.178 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T04:23:35.746 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.145 2024-05-09T04:23:47.758 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com' 2024-05-09T04:23:47.759 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T04:24:06.035 DEBUG:teuthology.orchestra.run.smithi192:> true 2024-05-09T04:24:06.174 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.145 2024-05-09T04:24:06.678 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-05-09T04:24:06.678 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.9.0-rc3-g806ec36b2112"... 2024-05-09T04:24:06.678 DEBUG:teuthology.orchestra.run.smithi192:> uname -r 2024-05-09T04:24:06.720 INFO:teuthology.orchestra.run.smithi192.stdout:6.9.0-rc3-g806ec36b2112 2024-05-09T04:24:06.720 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-05-09T04:24:06.720 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-09T04:24:06.720 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-09T04:24:07.722 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-05-09T04:24:07.722 DEBUG:teuthology.orchestra.run.smithi192:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-09T04:24:07.838 INFO:teuthology.orchestra.run.smithi192.stdout:ttyS1 2024-05-09T04:24:07.893 DEBUG:teuthology.parallel:result is None 2024-05-09T04:24:21.176 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com' 2024-05-09T04:24:21.177 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T04:24:21.483 DEBUG:teuthology.orchestra.run.smithi145:> true 2024-05-09T04:24:22.079 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi145.front.sepia.ceph.com' 2024-05-09T04:24:22.079 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.9.0-rc3-g806ec36b2112"... 2024-05-09T04:24:22.079 DEBUG:teuthology.orchestra.run.smithi145:> uname -r 2024-05-09T04:24:22.118 INFO:teuthology.orchestra.run.smithi145.stdout:6.9.0-rc3-g806ec36b2112 2024-05-09T04:24:22.118 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-05-09T04:24:22.119 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-09T04:24:22.119 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-09T04:24:23.120 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-05-09T04:24:23.120 DEBUG:teuthology.orchestra.run.smithi145:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-09T04:24:23.273 INFO:teuthology.orchestra.run.smithi145.stdout:ttyS1 2024-05-09T04:24:23.335 DEBUG:teuthology.parallel:result is None 2024-05-09T04:24:23.335 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-09T04:24:23.347 INFO:teuthology.task.internal:Creating test directory... 2024-05-09T04:24:23.348 DEBUG:teuthology.orchestra.run.smithi145:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-09T04:24:23.353 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-09T04:24:23.404 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-09T04:24:23.410 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-09T04:24:23.435 INFO:teuthology.task.internal:Creating archive directory... 2024-05-09T04:24:23.435 DEBUG:teuthology.orchestra.run.smithi145:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-09T04:24:23.440 DEBUG:teuthology.orchestra.run.smithi192:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-09T04:24:23.753 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-09T04:24:23.760 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-09T04:24:23.760 DEBUG:teuthology.orchestra.run.smithi145:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-09T04:24:23.765 DEBUG:teuthology.orchestra.run.smithi192:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-09T04:24:23.847 INFO:teuthology.orchestra.run.smithi145.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T04:24:23.855 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T04:24:23.876 INFO:teuthology.orchestra.run.smithi145.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T04:24:23.885 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T04:24:23.889 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-09T04:24:23.896 INFO:teuthology.task.internal:Configuring sudo... 2024-05-09T04:24:23.896 DEBUG:teuthology.orchestra.run.smithi145:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-09T04:24:23.902 DEBUG:teuthology.orchestra.run.smithi192:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-09T04:24:23.983 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-09T04:24:23.992 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-09T04:24:23.992 DEBUG:teuthology.orchestra.run.smithi145:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-09T04:24:23.999 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-09T04:24:24.041 DEBUG:teuthology.orchestra.run.smithi145:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T04:24:24.150 DEBUG:teuthology.orchestra.run.smithi145:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T04:24:24.243 DEBUG:teuthology.orchestra.run.smithi145:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T04:24:24.313 DEBUG:teuthology.orchestra.run.smithi145:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T04:24:24.392 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-05-09T04:24:24.392 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-09T04:24:24.455 DEBUG:teuthology.orchestra.run.smithi192:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T04:24:24.545 DEBUG:teuthology.orchestra.run.smithi192:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T04:24:24.640 DEBUG:teuthology.orchestra.run.smithi192:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T04:24:24.723 DEBUG:teuthology.orchestra.run.smithi192:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T04:24:24.811 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-09T04:24:24.811 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-09T04:24:24.893 DEBUG:teuthology.orchestra.run.smithi145:> sudo service rsyslog restart 2024-05-09T04:24:24.899 DEBUG:teuthology.orchestra.run.smithi192:> sudo service rsyslog restart 2024-05-09T04:24:25.010 INFO:teuthology.orchestra.run.smithi145.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T04:24:25.065 INFO:teuthology.orchestra.run.smithi192.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T04:24:25.404 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-09T04:24:25.413 INFO:teuthology.task.internal:Starting timer... 2024-05-09T04:24:25.413 INFO:teuthology.run_tasks:Running task pcp... 2024-05-09T04:24:25.456 INFO:teuthology.run_tasks:Running task selinux... 2024-05-09T04:24:25.496 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-05-09T04:24:25.497 DEBUG:teuthology.orchestra.run.smithi145:> sudo service auditd rotate 2024-05-09T04:24:25.604 INFO:teuthology.orchestra.run.smithi145.stdout:Rotating logs: 2024-05-09T04:24:25.608 DEBUG:teuthology.orchestra.run.smithi192:> sudo service auditd rotate 2024-05-09T04:24:25.739 INFO:teuthology.orchestra.run.smithi192.stdout:Rotating logs: 2024-05-09T04:24:25.744 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-09T04:24:25.744 DEBUG:teuthology.orchestra.run.smithi145:> /usr/sbin/getenforce 2024-05-09T04:24:25.797 INFO:teuthology.orchestra.run.smithi145.stdout:Permissive 2024-05-09T04:24:25.797 DEBUG:teuthology.orchestra.run.smithi192:> /usr/sbin/getenforce 2024-05-09T04:24:25.847 INFO:teuthology.orchestra.run.smithi192.stdout:Permissive 2024-05-09T04:24:25.848 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi145.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi192.front.sepia.ceph.com': 'permissive'} 2024-05-09T04:24:25.848 DEBUG:teuthology.orchestra.run.smithi145:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T04:24:25.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T04:24:25.922 DEBUG:teuthology.orchestra.run.smithi192:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T04:24:25.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T04:24:26.000 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-09T04:24:26.000 DEBUG:teuthology.orchestra.run.smithi145:> sudo /usr/sbin/setenforce permissive 2024-05-09T04:24:26.076 DEBUG:teuthology.orchestra.run.smithi192:> sudo /usr/sbin/setenforce permissive 2024-05-09T04:24:26.162 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-09T04:24:26.172 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-09T04:24:26.189 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-09T04:24:26.191 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi145.front.sepia.ceph.com,smithi192.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-09T04:30:11.271 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi145.front.sepia.ceph.com'), Remote(name='ubuntu@smithi192.front.sepia.ceph.com')] 2024-05-09T04:30:11.273 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com' 2024-05-09T04:30:11.274 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T04:30:11.386 DEBUG:teuthology.orchestra.run.smithi145:> true 2024-05-09T04:30:11.534 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi145.front.sepia.ceph.com' 2024-05-09T04:30:11.534 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-05-09T04:30:11.535 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T04:30:11.644 DEBUG:teuthology.orchestra.run.smithi192:> true 2024-05-09T04:30:11.791 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-05-09T04:30:11.792 INFO:teuthology.run_tasks:Running task clock... 2024-05-09T04:30:11.804 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-09T04:30:11.804 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-09T04:30:11.805 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T04:30:11.812 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-09T04:30:11.812 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T04:30:11.897 INFO:teuthology.orchestra.run.smithi145.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-09T04:30:11.899 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-09T04:30:11.935 INFO:teuthology.orchestra.run.smithi145.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-09T04:30:11.938 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-09T04:30:12.002 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: ntpd: command not found 2024-05-09T04:30:12.004 INFO:teuthology.orchestra.run.smithi192.stderr:sudo: ntpd: command not found 2024-05-09T04:30:12.031 INFO:teuthology.orchestra.run.smithi145.stdout:506 Cannot talk to daemon 2024-05-09T04:30:12.035 INFO:teuthology.orchestra.run.smithi192.stdout:506 Cannot talk to daemon 2024-05-09T04:30:12.070 INFO:teuthology.orchestra.run.smithi145.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-09T04:30:12.072 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-09T04:30:12.111 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-09T04:30:12.112 INFO:teuthology.orchestra.run.smithi145.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-09T04:30:12.238 INFO:teuthology.orchestra.run.smithi145.stderr:bash: line 1: ntpq: command not found 2024-05-09T04:30:12.244 INFO:teuthology.orchestra.run.smithi192.stderr:bash: line 1: ntpq: command not found 2024-05-09T04:30:12.246 INFO:teuthology.orchestra.run.smithi145.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T04:30:12.246 INFO:teuthology.orchestra.run.smithi145.stdout:=============================================================================== 2024-05-09T04:30:12.246 INFO:teuthology.orchestra.run.smithi145.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T04:30:12.246 INFO:teuthology.orchestra.run.smithi145.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T04:30:12.246 INFO:teuthology.orchestra.run.smithi145.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T04:30:12.247 INFO:teuthology.orchestra.run.smithi145.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T04:30:12.252 INFO:teuthology.orchestra.run.smithi192.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T04:30:12.252 INFO:teuthology.orchestra.run.smithi192.stdout:=============================================================================== 2024-05-09T04:30:12.253 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T04:30:12.253 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T04:30:12.253 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T04:30:12.253 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T04:30:12.255 INFO:teuthology.run_tasks:Running task install... 2024-05-09T04:30:12.263 DEBUG:teuthology.task.install:project ceph 2024-05-09T04:30:12.263 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '165a15e51d08073b5317c443415b0d8a7141b1ab'}} 2024-05-09T04:30:12.263 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '165a15e51d08073b5317c443415b0d8a7141b1ab'} 2024-05-09T04:30:12.263 INFO:teuthology.task.install:Using flavor: default 2024-05-09T04:30:12.269 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-09T04:30:12.269 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-09T04:30:12.270 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '165a15e51d08073b5317c443415b0d8a7141b1ab', 'tag': None, 'wait_for_package': False} 2024-05-09T04:30:12.270 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=165a15e51d08073b5317c443415b0d8a7141b1ab 2024-05-09T04:30:12.272 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '165a15e51d08073b5317c443415b0d8a7141b1ab', 'tag': None, 'wait_for_package': False} 2024-05-09T04:30:12.272 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=165a15e51d08073b5317c443415b0d8a7141b1ab 2024-05-09T04:30:12.431 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-63259-3/165a15e51d08073b5317c443415b0d8a7141b1ab/centos/9/flavors/default/ 2024-05-09T04:30:12.431 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3568.g165a15e5 2024-05-09T04:30:12.592 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-63259-3/165a15e51d08073b5317c443415b0d8a7141b1ab/centos/9/flavors/default/ 2024-05-09T04:30:12.592 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3568.g165a15e5 2024-05-09T04:30:12.733 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-63259-3/165a15e51d08073b5317c443415b0d8a7141b1ab/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-63259-3/165a15e51d08073b5317c443415b0d8a7141b1ab/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-63259-3/165a15e51d08073b5317c443415b0d8a7141b1ab/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-09T04:30:12.733 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-09T04:30:12.733 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-09T04:30:12.823 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-09T04:30:12.823 DEBUG:teuthology.orchestra.run.smithi192:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/165a15e51d08073b5317c443415b0d8a7141b1ab/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-09T04:30:12.894 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-63259-3/165a15e51d08073b5317c443415b0d8a7141b1ab/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-63259-3/165a15e51d08073b5317c443415b0d8a7141b1ab/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-63259-3/165a15e51d08073b5317c443415b0d8a7141b1ab/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-09T04:30:12.894 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-05-09T04:30:12.894 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-09T04:30:12.954 DEBUG:teuthology.orchestra.run.smithi192:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-09T04:30:12.984 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-09T04:30:12.984 DEBUG:teuthology.orchestra.run.smithi145:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/165a15e51d08073b5317c443415b0d8a7141b1ab/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-09T04:30:13.070 DEBUG:teuthology.orchestra.run.smithi192:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-09T04:30:13.113 DEBUG:teuthology.orchestra.run.smithi145:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-09T04:30:13.149 INFO:teuthology.orchestra.run.smithi192.stdout:check_obsoletes = 1 2024-05-09T04:30:13.153 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum clean all 2024-05-09T04:30:13.228 DEBUG:teuthology.orchestra.run.smithi145:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-09T04:30:13.303 INFO:teuthology.orchestra.run.smithi145.stdout:check_obsoletes = 1 2024-05-09T04:30:13.307 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum clean all 2024-05-09T04:30:13.499 INFO:teuthology.orchestra.run.smithi192.stdout:54 files removed 2024-05-09T04:30:13.541 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-09T04:30:13.641 INFO:teuthology.orchestra.run.smithi145.stdout:54 files removed 2024-05-09T04:30:13.680 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-09T04:30:14.716 INFO:teuthology.orchestra.run.smithi192.stdout:ceph packages for x86_64 113 kB/s | 84 kB 00:00 2024-05-09T04:30:14.867 INFO:teuthology.orchestra.run.smithi145.stdout:ceph packages for x86_64 111 kB/s | 84 kB 00:00 2024-05-09T04:30:15.292 INFO:teuthology.orchestra.run.smithi192.stdout:ceph noarch packages 25 kB/s | 12 kB 00:00 2024-05-09T04:30:15.300 INFO:teuthology.orchestra.run.smithi145.stdout:ceph noarch packages 35 kB/s | 12 kB 00:00 2024-05-09T04:30:15.767 INFO:teuthology.orchestra.run.smithi192.stdout:ceph source packages 4.9 kB/s | 1.9 kB 00:00 2024-05-09T04:30:16.002 INFO:teuthology.orchestra.run.smithi145.stdout:ceph source packages 3.0 kB/s | 1.9 kB 00:00 2024-05-09T04:30:17.866 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS Stream 9 - BaseOS 4.5 MB/s | 8.1 MB 00:01 2024-05-09T04:30:17.949 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - BaseOS 3.8 MB/s | 8.1 MB 00:02 2024-05-09T04:30:21.622 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS Stream 9 - AppStream 7.6 MB/s | 19 MB 00:02 2024-05-09T04:30:22.317 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - AppStream 6.1 MB/s | 19 MB 00:03 2024-05-09T04:30:28.209 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - CRB 3.1 MB/s | 6.2 MB 00:01 2024-05-09T04:30:29.718 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS Stream 9 - CRB 1.5 MB/s | 6.2 MB 00:04 2024-05-09T04:30:30.676 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - Extras packages 15 kB/s | 16 kB 00:01 2024-05-09T04:30:31.543 INFO:teuthology.orchestra.run.smithi145.stdout:CentOS Stream 9 - Extras packages 35 kB/s | 16 kB 00:00 2024-05-09T04:30:32.562 INFO:teuthology.orchestra.run.smithi192.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-05-09T04:30:34.261 INFO:teuthology.orchestra.run.smithi145.stdout:Extra Packages for Enterprise Linux 8.4 MB/s | 21 MB 00:02 2024-05-09T04:30:38.703 INFO:teuthology.orchestra.run.smithi192.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-05-09T04:30:40.436 INFO:teuthology.orchestra.run.smithi145.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-05-09T04:30:41.254 INFO:teuthology.orchestra.run.smithi192.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T04:30:41.256 INFO:teuthology.orchestra.run.smithi192.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T04:30:41.276 INFO:teuthology.orchestra.run.smithi192.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-09T04:30:41.277 INFO:teuthology.orchestra.run.smithi192.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-09T04:30:41.278 INFO:teuthology.orchestra.run.smithi192.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-09T04:30:41.283 INFO:teuthology.orchestra.run.smithi192.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-09T04:30:41.283 INFO:teuthology.orchestra.run.smithi192.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-09T04:30:41.285 INFO:teuthology.orchestra.run.smithi192.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-09T04:30:41.286 INFO:teuthology.orchestra.run.smithi192.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-09T04:30:41.360 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-05-09T04:30:41.392 INFO:teuthology.orchestra.run.smithi192.stdout:======================================================================================= 2024-05-09T04:30:41.392 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-05-09T04:30:41.392 INFO:teuthology.orchestra.run.smithi192.stdout:======================================================================================= 2024-05-09T04:30:41.392 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-05-09T04:30:41.392 INFO:teuthology.orchestra.run.smithi192.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-09T04:30:41.392 INFO:teuthology.orchestra.run.smithi192.stdout: ceph x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 6.6 k 2024-05-09T04:30:41.392 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-base x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 5.4 M 2024-05-09T04:30:41.393 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-fuse x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 890 k 2024-05-09T04:30:41.393 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 145 k 2024-05-09T04:30:41.393 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 1.5 M 2024-05-09T04:30:41.393 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3568.g165a15e5.el9 ceph-noarch 142 k 2024-05-09T04:30:41.393 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3568.g165a15e5.el9 ceph-noarch 3.5 M 2024-05-09T04:30:41.393 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3568.g165a15e5.el9 ceph-noarch 7.4 M 2024-05-09T04:30:41.393 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-rook noarch 2:19.0.0-3568.g165a15e5.el9 ceph-noarch 49 k 2024-05-09T04:30:41.393 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-radosgw x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 11 M 2024-05-09T04:30:41.393 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-test x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 49 M 2024-05-09T04:30:41.393 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-volume noarch 2:19.0.0-3568.g165a15e5.el9 ceph-noarch 273 k 2024-05-09T04:30:41.394 INFO:teuthology.orchestra.run.smithi192.stdout: cephadm noarch 2:19.0.0-3568.g165a15e5.el9 ceph-noarch 764 k 2024-05-09T04:30:41.394 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-mirror x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 225 k 2024-05-09T04:30:41.394 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-shell x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 27 k 2024-05-09T04:30:41.394 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-top noarch 2:19.0.0-3568.g165a15e5.el9 ceph-noarch 18 k 2024-05-09T04:30:41.394 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-09T04:30:41.394 INFO:teuthology.orchestra.run.smithi192.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-09T04:30:41.394 INFO:teuthology.orchestra.run.smithi192.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-09T04:30:41.394 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs-devel x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 32 k 2024-05-09T04:30:41.394 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs2 x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 747 k 2024-05-09T04:30:41.394 INFO:teuthology.orchestra.run.smithi192.stdout: librados-devel x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 127 k 2024-05-09T04:30:41.394 INFO:teuthology.orchestra.run.smithi192.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-09T04:30:41.394 INFO:teuthology.orchestra.run.smithi192.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-09T04:30:41.394 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-09T04:30:41.394 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-09T04:30:41.395 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-09T04:30:41.395 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-09T04:30:41.395 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cephfs x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 162 k 2024-05-09T04:30:41.395 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rados x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 321 k 2024-05-09T04:30:41.395 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rbd x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 299 k 2024-05-09T04:30:41.395 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rgw x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 100 k 2024-05-09T04:30:41.395 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-fuse x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 85 k 2024-05-09T04:30:41.395 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-mirror x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 3.1 M 2024-05-09T04:30:41.395 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-nbd x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 171 k 2024-05-09T04:30:41.395 INFO:teuthology.orchestra.run.smithi192.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-09T04:30:41.395 INFO:teuthology.orchestra.run.smithi192.stdout:Upgrading: 2024-05-09T04:30:41.395 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-09T04:30:41.395 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-09T04:30:41.395 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-09T04:30:41.396 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-09T04:30:41.396 INFO:teuthology.orchestra.run.smithi192.stdout: librados2 x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 3.4 M 2024-05-09T04:30:41.396 INFO:teuthology.orchestra.run.smithi192.stdout: librbd1 x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 3.1 M 2024-05-09T04:30:41.396 INFO:teuthology.orchestra.run.smithi192.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-09T04:30:41.396 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-09T04:30:41.396 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-05-09T04:30:41.396 INFO:teuthology.orchestra.run.smithi192.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-09T04:30:41.396 INFO:teuthology.orchestra.run.smithi192.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-09T04:30:41.396 INFO:teuthology.orchestra.run.smithi192.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-09T04:30:41.396 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-common x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 21 M 2024-05-09T04:30:41.396 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3568.g165a15e5.el9 ceph-noarch 34 k 2024-05-09T04:30:41.396 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mds x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 2.4 M 2024-05-09T04:30:41.396 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3568.g165a15e5.el9 ceph-noarch 274 k 2024-05-09T04:30:41.396 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mon x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 4.7 M 2024-05-09T04:30:41.396 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-osd x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 17 M 2024-05-09T04:30:41.397 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3568.g165a15e5.el9 ceph-noarch 16 k 2024-05-09T04:30:41.397 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-selinux x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 25 k 2024-05-09T04:30:41.397 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-09T04:30:41.397 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-09T04:30:41.397 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-09T04:30:41.397 INFO:teuthology.orchestra.run.smithi192.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-09T04:30:41.397 INFO:teuthology.orchestra.run.smithi192.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-09T04:30:41.397 INFO:teuthology.orchestra.run.smithi192.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-09T04:30:41.398 INFO:teuthology.orchestra.run.smithi192.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-09T04:30:41.398 INFO:teuthology.orchestra.run.smithi192.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-09T04:30:41.398 INFO:teuthology.orchestra.run.smithi192.stdout: libcephsqlite x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 163 k 2024-05-09T04:30:41.398 INFO:teuthology.orchestra.run.smithi192.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-09T04:30:41.398 INFO:teuthology.orchestra.run.smithi192.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-09T04:30:41.398 INFO:teuthology.orchestra.run.smithi192.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-09T04:30:41.398 INFO:teuthology.orchestra.run.smithi192.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-09T04:30:41.398 INFO:teuthology.orchestra.run.smithi192.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-09T04:30:41.398 INFO:teuthology.orchestra.run.smithi192.stdout: libradosstriper1 x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 500 k 2024-05-09T04:30:41.398 INFO:teuthology.orchestra.run.smithi192.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-09T04:30:41.398 INFO:teuthology.orchestra.run.smithi192.stdout: librgw2 x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 5.4 M 2024-05-09T04:30:41.398 INFO:teuthology.orchestra.run.smithi192.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-09T04:30:41.399 INFO:teuthology.orchestra.run.smithi192.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-09T04:30:41.399 INFO:teuthology.orchestra.run.smithi192.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-09T04:30:41.399 INFO:teuthology.orchestra.run.smithi192.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-09T04:30:41.399 INFO:teuthology.orchestra.run.smithi192.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-09T04:30:41.399 INFO:teuthology.orchestra.run.smithi192.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-09T04:30:41.399 INFO:teuthology.orchestra.run.smithi192.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-09T04:30:41.399 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-09T04:30:41.399 INFO:teuthology.orchestra.run.smithi192.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-09T04:30:41.399 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-09T04:30:41.399 INFO:teuthology.orchestra.run.smithi192.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-09T04:30:41.399 INFO:teuthology.orchestra.run.smithi192.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-09T04:30:41.399 INFO:teuthology.orchestra.run.smithi192.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-09T04:30:41.399 INFO:teuthology.orchestra.run.smithi192.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-09T04:30:41.399 INFO:teuthology.orchestra.run.smithi192.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-09T04:30:41.400 INFO:teuthology.orchestra.run.smithi192.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-09T04:30:41.400 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-09T04:30:41.400 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-argparse x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 45 k 2024-05-09T04:30:41.400 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-common x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 135 k 2024-05-09T04:30:41.400 INFO:teuthology.orchestra.run.smithi192.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-09T04:30:41.400 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-09T04:30:41.400 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-09T04:30:41.400 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-09T04:30:41.400 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-09T04:30:41.400 INFO:teuthology.orchestra.run.smithi192.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-09T04:30:41.400 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-09T04:30:41.400 INFO:teuthology.orchestra.run.smithi192.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-09T04:30:41.400 INFO:teuthology.orchestra.run.smithi192.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-09T04:30:41.400 INFO:teuthology.orchestra.run.smithi192.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-09T04:30:41.401 INFO:teuthology.orchestra.run.smithi192.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-09T04:30:41.401 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-09T04:30:41.401 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-09T04:30:41.401 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-09T04:30:41.401 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-09T04:30:41.401 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-09T04:30:41.401 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-09T04:30:41.401 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-09T04:30:41.401 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-09T04:30:41.401 INFO:teuthology.orchestra.run.smithi192.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-09T04:30:41.401 INFO:teuthology.orchestra.run.smithi192.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-09T04:30:41.401 INFO:teuthology.orchestra.run.smithi192.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-09T04:30:41.401 INFO:teuthology.orchestra.run.smithi192.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-09T04:30:41.401 INFO:teuthology.orchestra.run.smithi192.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-09T04:30:41.402 INFO:teuthology.orchestra.run.smithi192.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-09T04:30:41.402 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-09T04:30:41.402 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-09T04:30:41.402 INFO:teuthology.orchestra.run.smithi192.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-09T04:30:41.402 INFO:teuthology.orchestra.run.smithi192.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-09T04:30:41.402 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-09T04:30:41.402 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-09T04:30:41.402 INFO:teuthology.orchestra.run.smithi192.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-09T04:30:41.402 INFO:teuthology.orchestra.run.smithi192.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-09T04:30:41.402 INFO:teuthology.orchestra.run.smithi192.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-09T04:30:41.402 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-09T04:30:41.402 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-09T04:30:41.402 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-09T04:30:41.402 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-09T04:30:41.403 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-09T04:30:41.403 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-09T04:30:41.403 INFO:teuthology.orchestra.run.smithi192.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-09T04:30:41.403 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-09T04:30:41.403 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-09T04:30:41.403 INFO:teuthology.orchestra.run.smithi192.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-09T04:30:41.403 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-09T04:30:41.403 INFO:teuthology.orchestra.run.smithi192.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-09T04:30:41.403 INFO:teuthology.orchestra.run.smithi192.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-09T04:30:41.403 INFO:teuthology.orchestra.run.smithi192.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-09T04:30:41.403 INFO:teuthology.orchestra.run.smithi192.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-09T04:30:41.403 INFO:teuthology.orchestra.run.smithi192.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-09T04:30:41.403 INFO:teuthology.orchestra.run.smithi192.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-09T04:30:41.403 INFO:teuthology.orchestra.run.smithi192.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-09T04:30:41.404 INFO:teuthology.orchestra.run.smithi192.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-09T04:30:41.404 INFO:teuthology.orchestra.run.smithi192.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-09T04:30:41.404 INFO:teuthology.orchestra.run.smithi192.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-09T04:30:41.404 INFO:teuthology.orchestra.run.smithi192.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-09T04:30:41.404 INFO:teuthology.orchestra.run.smithi192.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-09T04:30:41.404 INFO:teuthology.orchestra.run.smithi192.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-09T04:30:41.404 INFO:teuthology.orchestra.run.smithi192.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-09T04:30:41.404 INFO:teuthology.orchestra.run.smithi192.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-09T04:30:41.404 INFO:teuthology.orchestra.run.smithi192.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-09T04:30:41.404 INFO:teuthology.orchestra.run.smithi192.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-09T04:30:41.404 INFO:teuthology.orchestra.run.smithi192.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-09T04:30:41.405 INFO:teuthology.orchestra.run.smithi192.stdout:Installing weak dependencies: 2024-05-09T04:30:41.405 INFO:teuthology.orchestra.run.smithi192.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-09T04:30:41.405 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-09T04:30:41.405 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-05-09T04:30:41.405 INFO:teuthology.orchestra.run.smithi192.stdout:======================================================================================= 2024-05-09T04:30:41.405 INFO:teuthology.orchestra.run.smithi192.stdout:Install 148 Packages 2024-05-09T04:30:41.405 INFO:teuthology.orchestra.run.smithi192.stdout:Upgrade 8 Packages 2024-05-09T04:30:41.405 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-09T04:30:41.406 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 226 M 2024-05-09T04:30:41.406 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-05-09T04:30:42.755 INFO:teuthology.orchestra.run.smithi192.stdout:(1/156): ceph-19.0.0-3568.g165a15e5.el9.x86_64. 13 kB/s | 6.6 kB 00:00 2024-05-09T04:30:42.988 INFO:teuthology.orchestra.run.smithi145.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T04:30:42.990 INFO:teuthology.orchestra.run.smithi145.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T04:30:43.010 INFO:teuthology.orchestra.run.smithi145.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-09T04:30:43.011 INFO:teuthology.orchestra.run.smithi145.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-09T04:30:43.012 INFO:teuthology.orchestra.run.smithi145.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-09T04:30:43.013 INFO:teuthology.orchestra.run.smithi145.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-09T04:30:43.016 INFO:teuthology.orchestra.run.smithi145.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-09T04:30:43.019 INFO:teuthology.orchestra.run.smithi145.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-09T04:30:43.020 INFO:teuthology.orchestra.run.smithi145.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-09T04:30:43.091 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2024-05-09T04:30:43.124 INFO:teuthology.orchestra.run.smithi145.stdout:======================================================================================= 2024-05-09T04:30:43.124 INFO:teuthology.orchestra.run.smithi145.stdout: Package Arch Version Repository Size 2024-05-09T04:30:43.124 INFO:teuthology.orchestra.run.smithi145.stdout:======================================================================================= 2024-05-09T04:30:43.124 INFO:teuthology.orchestra.run.smithi145.stdout:Installing: 2024-05-09T04:30:43.124 INFO:teuthology.orchestra.run.smithi145.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-09T04:30:43.124 INFO:teuthology.orchestra.run.smithi145.stdout: ceph x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 6.6 k 2024-05-09T04:30:43.125 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-base x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 5.4 M 2024-05-09T04:30:43.125 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-fuse x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 890 k 2024-05-09T04:30:43.125 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 145 k 2024-05-09T04:30:43.125 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 1.5 M 2024-05-09T04:30:43.125 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3568.g165a15e5.el9 ceph-noarch 142 k 2024-05-09T04:30:43.125 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3568.g165a15e5.el9 ceph-noarch 3.5 M 2024-05-09T04:30:43.125 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3568.g165a15e5.el9 ceph-noarch 7.4 M 2024-05-09T04:30:43.125 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-rook noarch 2:19.0.0-3568.g165a15e5.el9 ceph-noarch 49 k 2024-05-09T04:30:43.125 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-radosgw x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 11 M 2024-05-09T04:30:43.125 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-test x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 49 M 2024-05-09T04:30:43.125 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-volume noarch 2:19.0.0-3568.g165a15e5.el9 ceph-noarch 273 k 2024-05-09T04:30:43.125 INFO:teuthology.orchestra.run.smithi145.stdout: cephadm noarch 2:19.0.0-3568.g165a15e5.el9 ceph-noarch 764 k 2024-05-09T04:30:43.125 INFO:teuthology.orchestra.run.smithi145.stdout: cephfs-mirror x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 225 k 2024-05-09T04:30:43.126 INFO:teuthology.orchestra.run.smithi145.stdout: cephfs-shell x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 27 k 2024-05-09T04:30:43.126 INFO:teuthology.orchestra.run.smithi145.stdout: cephfs-top noarch 2:19.0.0-3568.g165a15e5.el9 ceph-noarch 18 k 2024-05-09T04:30:43.126 INFO:teuthology.orchestra.run.smithi145.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-09T04:30:43.126 INFO:teuthology.orchestra.run.smithi145.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-09T04:30:43.126 INFO:teuthology.orchestra.run.smithi145.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-09T04:30:43.126 INFO:teuthology.orchestra.run.smithi145.stdout: libcephfs-devel x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 32 k 2024-05-09T04:30:43.126 INFO:teuthology.orchestra.run.smithi145.stdout: libcephfs2 x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 747 k 2024-05-09T04:30:43.126 INFO:teuthology.orchestra.run.smithi145.stdout: librados-devel x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 127 k 2024-05-09T04:30:43.126 INFO:teuthology.orchestra.run.smithi145.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-09T04:30:43.126 INFO:teuthology.orchestra.run.smithi145.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-09T04:30:43.126 INFO:teuthology.orchestra.run.smithi145.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-09T04:30:43.126 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-09T04:30:43.126 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-09T04:30:43.126 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-09T04:30:43.127 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cephfs x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 162 k 2024-05-09T04:30:43.127 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rados x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 321 k 2024-05-09T04:30:43.127 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rbd x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 299 k 2024-05-09T04:30:43.127 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rgw x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 100 k 2024-05-09T04:30:43.127 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-fuse x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 85 k 2024-05-09T04:30:43.127 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-mirror x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 3.1 M 2024-05-09T04:30:43.127 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-nbd x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 171 k 2024-05-09T04:30:43.127 INFO:teuthology.orchestra.run.smithi145.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-09T04:30:43.127 INFO:teuthology.orchestra.run.smithi145.stdout:Upgrading: 2024-05-09T04:30:43.127 INFO:teuthology.orchestra.run.smithi145.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-09T04:30:43.127 INFO:teuthology.orchestra.run.smithi145.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-09T04:30:43.127 INFO:teuthology.orchestra.run.smithi145.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-09T04:30:43.127 INFO:teuthology.orchestra.run.smithi145.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-09T04:30:43.127 INFO:teuthology.orchestra.run.smithi145.stdout: librados2 x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 3.4 M 2024-05-09T04:30:43.127 INFO:teuthology.orchestra.run.smithi145.stdout: librbd1 x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 3.1 M 2024-05-09T04:30:43.128 INFO:teuthology.orchestra.run.smithi145.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-09T04:30:43.128 INFO:teuthology.orchestra.run.smithi145.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-09T04:30:43.128 INFO:teuthology.orchestra.run.smithi145.stdout:Installing dependencies: 2024-05-09T04:30:43.128 INFO:teuthology.orchestra.run.smithi145.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-09T04:30:43.128 INFO:teuthology.orchestra.run.smithi145.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-09T04:30:43.128 INFO:teuthology.orchestra.run.smithi145.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-09T04:30:43.128 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-common x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 21 M 2024-05-09T04:30:43.128 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3568.g165a15e5.el9 ceph-noarch 34 k 2024-05-09T04:30:43.128 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mds x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 2.4 M 2024-05-09T04:30:43.128 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3568.g165a15e5.el9 ceph-noarch 274 k 2024-05-09T04:30:43.128 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mon x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 4.7 M 2024-05-09T04:30:43.128 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-osd x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 17 M 2024-05-09T04:30:43.128 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3568.g165a15e5.el9 ceph-noarch 16 k 2024-05-09T04:30:43.128 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-selinux x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 25 k 2024-05-09T04:30:43.129 INFO:teuthology.orchestra.run.smithi145.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-09T04:30:43.129 INFO:teuthology.orchestra.run.smithi145.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-09T04:30:43.129 INFO:teuthology.orchestra.run.smithi145.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-09T04:30:43.129 INFO:teuthology.orchestra.run.smithi145.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-09T04:30:43.129 INFO:teuthology.orchestra.run.smithi145.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-09T04:30:43.129 INFO:teuthology.orchestra.run.smithi145.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-09T04:30:43.129 INFO:teuthology.orchestra.run.smithi145.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-09T04:30:43.129 INFO:teuthology.orchestra.run.smithi145.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-09T04:30:43.129 INFO:teuthology.orchestra.run.smithi145.stdout: libcephsqlite x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 163 k 2024-05-09T04:30:43.129 INFO:teuthology.orchestra.run.smithi145.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-09T04:30:43.129 INFO:teuthology.orchestra.run.smithi145.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-09T04:30:43.129 INFO:teuthology.orchestra.run.smithi145.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-09T04:30:43.129 INFO:teuthology.orchestra.run.smithi145.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-09T04:30:43.129 INFO:teuthology.orchestra.run.smithi145.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-09T04:30:43.130 INFO:teuthology.orchestra.run.smithi145.stdout: libradosstriper1 x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 500 k 2024-05-09T04:30:43.130 INFO:teuthology.orchestra.run.smithi145.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-09T04:30:43.130 INFO:teuthology.orchestra.run.smithi145.stdout: librgw2 x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 5.4 M 2024-05-09T04:30:43.130 INFO:teuthology.orchestra.run.smithi145.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-09T04:30:43.130 INFO:teuthology.orchestra.run.smithi145.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-09T04:30:43.130 INFO:teuthology.orchestra.run.smithi145.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-09T04:30:43.130 INFO:teuthology.orchestra.run.smithi145.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-09T04:30:43.130 INFO:teuthology.orchestra.run.smithi145.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-09T04:30:43.130 INFO:teuthology.orchestra.run.smithi145.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-09T04:30:43.130 INFO:teuthology.orchestra.run.smithi145.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-09T04:30:43.130 INFO:teuthology.orchestra.run.smithi145.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-09T04:30:43.130 INFO:teuthology.orchestra.run.smithi145.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-09T04:30:43.130 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-09T04:30:43.130 INFO:teuthology.orchestra.run.smithi145.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-09T04:30:43.131 INFO:teuthology.orchestra.run.smithi145.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-09T04:30:43.131 INFO:teuthology.orchestra.run.smithi145.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-09T04:30:43.131 INFO:teuthology.orchestra.run.smithi145.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-09T04:30:43.131 INFO:teuthology.orchestra.run.smithi145.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-09T04:30:43.131 INFO:teuthology.orchestra.run.smithi145.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-09T04:30:43.131 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-09T04:30:43.131 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ceph-argparse x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 45 k 2024-05-09T04:30:43.131 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ceph-common x86_64 2:19.0.0-3568.g165a15e5.el9 ceph 135 k 2024-05-09T04:30:43.131 INFO:teuthology.orchestra.run.smithi145.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-09T04:30:43.131 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-09T04:30:43.131 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-09T04:30:43.131 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-09T04:30:43.131 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-09T04:30:43.131 INFO:teuthology.orchestra.run.smithi145.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-09T04:30:43.131 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-09T04:30:43.132 INFO:teuthology.orchestra.run.smithi145.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-09T04:30:43.132 INFO:teuthology.orchestra.run.smithi145.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-09T04:30:43.132 INFO:teuthology.orchestra.run.smithi145.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-09T04:30:43.132 INFO:teuthology.orchestra.run.smithi145.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-09T04:30:43.132 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-09T04:30:43.132 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-09T04:30:43.132 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-09T04:30:43.132 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-09T04:30:43.132 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-09T04:30:43.132 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-09T04:30:43.132 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-09T04:30:43.132 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-09T04:30:43.132 INFO:teuthology.orchestra.run.smithi145.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-09T04:30:43.132 INFO:teuthology.orchestra.run.smithi145.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-09T04:30:43.133 INFO:teuthology.orchestra.run.smithi145.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-09T04:30:43.133 INFO:teuthology.orchestra.run.smithi145.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-09T04:30:43.133 INFO:teuthology.orchestra.run.smithi145.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-09T04:30:43.133 INFO:teuthology.orchestra.run.smithi145.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-09T04:30:43.133 INFO:teuthology.orchestra.run.smithi145.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-09T04:30:43.133 INFO:teuthology.orchestra.run.smithi145.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-09T04:30:43.133 INFO:teuthology.orchestra.run.smithi145.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-09T04:30:43.133 INFO:teuthology.orchestra.run.smithi145.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-09T04:30:43.133 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-09T04:30:43.133 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-09T04:30:43.133 INFO:teuthology.orchestra.run.smithi145.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-09T04:30:43.133 INFO:teuthology.orchestra.run.smithi145.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-09T04:30:43.133 INFO:teuthology.orchestra.run.smithi145.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-09T04:30:43.133 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-09T04:30:43.133 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-09T04:30:43.134 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-09T04:30:43.134 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-09T04:30:43.134 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-09T04:30:43.134 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-09T04:30:43.134 INFO:teuthology.orchestra.run.smithi145.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-09T04:30:43.134 INFO:teuthology.orchestra.run.smithi145.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-09T04:30:43.134 INFO:teuthology.orchestra.run.smithi145.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-09T04:30:43.134 INFO:teuthology.orchestra.run.smithi145.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-09T04:30:43.134 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-09T04:30:43.134 INFO:teuthology.orchestra.run.smithi145.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-09T04:30:43.134 INFO:teuthology.orchestra.run.smithi145.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-09T04:30:43.134 INFO:teuthology.orchestra.run.smithi145.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-09T04:30:43.134 INFO:teuthology.orchestra.run.smithi145.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-09T04:30:43.134 INFO:teuthology.orchestra.run.smithi145.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-09T04:30:43.135 INFO:teuthology.orchestra.run.smithi145.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-09T04:30:43.135 INFO:teuthology.orchestra.run.smithi145.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-09T04:30:43.135 INFO:teuthology.orchestra.run.smithi145.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-09T04:30:43.135 INFO:teuthology.orchestra.run.smithi145.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-09T04:30:43.135 INFO:teuthology.orchestra.run.smithi145.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-09T04:30:43.135 INFO:teuthology.orchestra.run.smithi145.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-09T04:30:43.135 INFO:teuthology.orchestra.run.smithi145.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-09T04:30:43.135 INFO:teuthology.orchestra.run.smithi145.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-09T04:30:43.135 INFO:teuthology.orchestra.run.smithi145.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-09T04:30:43.135 INFO:teuthology.orchestra.run.smithi145.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-09T04:30:43.135 INFO:teuthology.orchestra.run.smithi145.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-09T04:30:43.135 INFO:teuthology.orchestra.run.smithi145.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-09T04:30:43.135 INFO:teuthology.orchestra.run.smithi145.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-09T04:30:43.135 INFO:teuthology.orchestra.run.smithi145.stdout:Installing weak dependencies: 2024-05-09T04:30:43.136 INFO:teuthology.orchestra.run.smithi145.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-09T04:30:43.136 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:30:43.136 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction Summary 2024-05-09T04:30:43.136 INFO:teuthology.orchestra.run.smithi145.stdout:======================================================================================= 2024-05-09T04:30:43.136 INFO:teuthology.orchestra.run.smithi145.stdout:Install 148 Packages 2024-05-09T04:30:43.136 INFO:teuthology.orchestra.run.smithi145.stdout:Upgrade 8 Packages 2024-05-09T04:30:43.136 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:30:43.136 INFO:teuthology.orchestra.run.smithi145.stdout:Total download size: 226 M 2024-05-09T04:30:43.137 INFO:teuthology.orchestra.run.smithi145.stdout:Downloading Packages: 2024-05-09T04:30:43.405 INFO:teuthology.orchestra.run.smithi192.stdout:(2/156): ceph-fuse-19.0.0-3568.g165a15e5.el9.x8 1.3 MB/s | 890 kB 00:00 2024-05-09T04:30:43.580 INFO:teuthology.orchestra.run.smithi192.stdout:(3/156): ceph-immutable-object-cache-19.0.0-356 832 kB/s | 145 kB 00:00 2024-05-09T04:30:43.731 INFO:teuthology.orchestra.run.smithi192.stdout:(4/156): ceph-base-19.0.0-3568.g165a15e5.el9.x8 3.6 MB/s | 5.4 MB 00:01 2024-05-09T04:30:44.048 INFO:teuthology.orchestra.run.smithi192.stdout:(5/156): ceph-mgr-19.0.0-3568.g165a15e5.el9.x86 4.8 MB/s | 1.5 MB 00:00 2024-05-09T04:30:44.296 INFO:teuthology.orchestra.run.smithi145.stdout:(1/156): ceph-19.0.0-3568.g165a15e5.el9.x86_64. 19 kB/s | 6.6 kB 00:00 2024-05-09T04:30:44.481 INFO:teuthology.orchestra.run.smithi192.stdout:(6/156): ceph-mds-19.0.0-3568.g165a15e5.el9.x86 2.6 MB/s | 2.4 MB 00:00 2024-05-09T04:30:44.696 INFO:teuthology.orchestra.run.smithi145.stdout:(2/156): ceph-fuse-19.0.0-3568.g165a15e5.el9.x8 2.2 MB/s | 890 kB 00:00 2024-05-09T04:30:44.771 INFO:teuthology.orchestra.run.smithi145.stdout:(3/156): ceph-immutable-object-cache-19.0.0-356 1.9 MB/s | 145 kB 00:00 2024-05-09T04:30:44.890 INFO:teuthology.orchestra.run.smithi192.stdout:(7/156): ceph-common-19.0.0-3568.g165a15e5.el9. 8.0 MB/s | 21 MB 00:02 2024-05-09T04:30:44.998 INFO:teuthology.orchestra.run.smithi192.stdout:(8/156): ceph-mon-19.0.0-3568.g165a15e5.el9.x86 4.9 MB/s | 4.7 MB 00:00 2024-05-09T04:30:45.132 INFO:teuthology.orchestra.run.smithi192.stdout:(9/156): ceph-selinux-19.0.0-3568.g165a15e5.el9 190 kB/s | 25 kB 00:00 2024-05-09T04:30:45.171 INFO:teuthology.orchestra.run.smithi145.stdout:(4/156): ceph-mds-19.0.0-3568.g165a15e5.el9.x86 5.9 MB/s | 2.4 MB 00:00 2024-05-09T04:30:45.530 INFO:teuthology.orchestra.run.smithi145.stdout:(5/156): ceph-mgr-19.0.0-3568.g165a15e5.el9.x86 4.2 MB/s | 1.5 MB 00:00 2024-05-09T04:30:46.149 INFO:teuthology.orchestra.run.smithi192.stdout:(10/156): ceph-radosgw-19.0.0-3568.g165a15e5.el 8.4 MB/s | 11 MB 00:01 2024-05-09T04:30:46.231 INFO:teuthology.orchestra.run.smithi145.stdout:(6/156): ceph-mon-19.0.0-3568.g165a15e5.el9.x86 6.7 MB/s | 4.7 MB 00:00 2024-05-09T04:30:46.316 INFO:teuthology.orchestra.run.smithi192.stdout:(11/156): cephfs-mirror-19.0.0-3568.g165a15e5.e 1.3 MB/s | 225 kB 00:00 2024-05-09T04:30:46.524 INFO:teuthology.orchestra.run.smithi192.stdout:(12/156): cephfs-shell-19.0.0-3568.g165a15e5.el 130 kB/s | 27 kB 00:00 2024-05-09T04:30:46.958 INFO:teuthology.orchestra.run.smithi192.stdout:(13/156): libcephfs-devel-19.0.0-3568.g165a15e5 73 kB/s | 32 kB 00:00 2024-05-09T04:30:47.475 INFO:teuthology.orchestra.run.smithi192.stdout:(14/156): ceph-osd-19.0.0-3568.g165a15e5.el9.x8 5.6 MB/s | 17 MB 00:02 2024-05-09T04:30:47.564 INFO:teuthology.orchestra.run.smithi145.stdout:(7/156): ceph-osd-19.0.0-3568.g165a15e5.el9.x86 13 MB/s | 17 MB 00:01 2024-05-09T04:30:47.617 INFO:teuthology.orchestra.run.smithi192.stdout:(15/156): libcephsqlite-19.0.0-3568.g165a15e5.e 1.1 MB/s | 163 kB 00:00 2024-05-09T04:30:47.759 INFO:teuthology.orchestra.run.smithi192.stdout:(16/156): librados-devel-19.0.0-3568.g165a15e5. 897 kB/s | 127 kB 00:00 2024-05-09T04:30:47.951 INFO:teuthology.orchestra.run.smithi192.stdout:(17/156): libcephfs2-19.0.0-3568.g165a15e5.el9. 753 kB/s | 747 kB 00:00 2024-05-09T04:30:47.985 INFO:teuthology.orchestra.run.smithi192.stdout:(18/156): libradosstriper1-19.0.0-3568.g165a15e 2.2 MB/s | 500 kB 00:00 2024-05-09T04:30:48.135 INFO:teuthology.orchestra.run.smithi192.stdout:(19/156): python3-ceph-argparse-19.0.0-3568.g16 302 kB/s | 45 kB 00:00 2024-05-09T04:30:48.252 INFO:teuthology.orchestra.run.smithi192.stdout:(20/156): python3-ceph-common-19.0.0-3568.g165a 1.1 MB/s | 135 kB 00:00 2024-05-09T04:30:48.289 INFO:teuthology.orchestra.run.smithi145.stdout:(8/156): ceph-radosgw-19.0.0-3568.g165a15e5.el9 15 MB/s | 11 MB 00:00 2024-05-09T04:30:48.339 INFO:teuthology.orchestra.run.smithi145.stdout:(9/156): ceph-selinux-19.0.0-3568.g165a15e5.el9 514 kB/s | 25 kB 00:00 2024-05-09T04:30:48.377 INFO:teuthology.orchestra.run.smithi192.stdout:(21/156): python3-cephfs-19.0.0-3568.g165a15e5. 1.3 MB/s | 162 kB 00:00 2024-05-09T04:30:48.585 INFO:teuthology.orchestra.run.smithi192.stdout:(22/156): python3-rados-19.0.0-3568.g165a15e5.e 1.5 MB/s | 321 kB 00:00 2024-05-09T04:30:48.744 INFO:teuthology.orchestra.run.smithi192.stdout:(23/156): python3-rbd-19.0.0-3568.g165a15e5.el9 1.9 MB/s | 299 kB 00:00 2024-05-09T04:30:48.886 INFO:teuthology.orchestra.run.smithi192.stdout:(24/156): python3-rgw-19.0.0-3568.g165a15e5.el9 707 kB/s | 100 kB 00:00 2024-05-09T04:30:48.995 INFO:teuthology.orchestra.run.smithi192.stdout:(25/156): librgw2-19.0.0-3568.g165a15e5.el9.x86 5.1 MB/s | 5.4 MB 00:01 2024-05-09T04:30:49.063 INFO:teuthology.orchestra.run.smithi192.stdout:(26/156): rbd-fuse-19.0.0-3568.g165a15e5.el9.x8 486 kB/s | 85 kB 00:00 2024-05-09T04:30:49.267 INFO:teuthology.orchestra.run.smithi192.stdout:(27/156): rbd-nbd-19.0.0-3568.g165a15e5.el9.x86 860 kB/s | 171 kB 00:00 2024-05-09T04:30:49.446 INFO:teuthology.orchestra.run.smithi192.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3568.g 187 kB/s | 34 kB 00:00 2024-05-09T04:30:49.713 INFO:teuthology.orchestra.run.smithi192.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3568.g165a15e 535 kB/s | 142 kB 00:00 2024-05-09T04:30:49.797 INFO:teuthology.orchestra.run.smithi192.stdout:(30/156): rbd-mirror-19.0.0-3568.g165a15e5.el9. 3.9 MB/s | 3.1 MB 00:00 2024-05-09T04:30:50.805 INFO:teuthology.orchestra.run.smithi192.stdout:(31/156): ceph-mgr-dashboard-19.0.0-3568.g165a1 3.2 MB/s | 3.5 MB 00:01 2024-05-09T04:30:50.955 INFO:teuthology.orchestra.run.smithi192.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3568.g16 1.8 MB/s | 274 kB 00:00 2024-05-09T04:30:51.072 INFO:teuthology.orchestra.run.smithi192.stdout:(33/156): ceph-mgr-rook-19.0.0-3568.g165a15e5.e 427 kB/s | 49 kB 00:00 2024-05-09T04:30:51.189 INFO:teuthology.orchestra.run.smithi192.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3568.g1 142 kB/s | 16 kB 00:00 2024-05-09T04:30:51.331 INFO:teuthology.orchestra.run.smithi192.stdout:(35/156): ceph-volume-19.0.0-3568.g165a15e5.el9 1.9 MB/s | 273 kB 00:00 2024-05-09T04:30:51.515 INFO:teuthology.orchestra.run.smithi192.stdout:(36/156): ceph-mgr-diskprediction-local-19.0.0- 4.3 MB/s | 7.4 MB 00:01 2024-05-09T04:30:51.573 INFO:teuthology.orchestra.run.smithi192.stdout:(37/156): cephadm-19.0.0-3568.g165a15e5.el9.noa 3.1 MB/s | 764 kB 00:00 2024-05-09T04:30:51.599 INFO:teuthology.orchestra.run.smithi192.stdout:(38/156): cephfs-top-19.0.0-3568.g165a15e5.el9. 221 kB/s | 18 kB 00:00 2024-05-09T04:30:51.773 INFO:teuthology.orchestra.run.smithi145.stdout:(10/156): ceph-test-19.0.0-3568.g165a15e5.el9.x 14 MB/s | 49 MB 00:03 2024-05-09T04:30:51.831 INFO:teuthology.orchestra.run.smithi145.stdout:(11/156): cephfs-mirror-19.0.0-3568.g165a15e5.e 3.8 MB/s | 225 kB 00:00 2024-05-09T04:30:51.882 INFO:teuthology.orchestra.run.smithi145.stdout:(12/156): cephfs-shell-19.0.0-3568.g165a15e5.el 551 kB/s | 27 kB 00:00 2024-05-09T04:30:51.934 INFO:teuthology.orchestra.run.smithi145.stdout:(13/156): libcephfs-devel-19.0.0-3568.g165a15e5 651 kB/s | 32 kB 00:00 2024-05-09T04:30:52.024 INFO:teuthology.orchestra.run.smithi145.stdout:(14/156): libcephfs2-19.0.0-3568.g165a15e5.el9. 8.0 MB/s | 747 kB 00:00 2024-05-09T04:30:52.074 INFO:teuthology.orchestra.run.smithi145.stdout:(15/156): libcephsqlite-19.0.0-3568.g165a15e5.e 3.3 MB/s | 163 kB 00:00 2024-05-09T04:30:52.090 INFO:teuthology.orchestra.run.smithi192.stdout:(39/156): fuse-2.9.9-15.el9.x86_64.rpm 156 kB/s | 80 kB 00:00 2024-05-09T04:30:52.416 INFO:teuthology.orchestra.run.smithi192.stdout:(40/156): libquadmath-11.4.1-3.el9.x86_64.rpm 602 kB/s | 195 kB 00:00 2024-05-09T04:30:52.491 INFO:teuthology.orchestra.run.smithi192.stdout:(41/156): libgfortran-11.4.1-3.el9.x86_64.rpm 901 kB/s | 803 kB 00:00 2024-05-09T04:30:52.558 INFO:teuthology.orchestra.run.smithi145.stdout:(16/156): ceph-common-19.0.0-3568.g165a15e5.el9 2.5 MB/s | 21 MB 00:08 2024-05-09T04:30:52.591 INFO:teuthology.orchestra.run.smithi145.stdout:(17/156): librados-devel-19.0.0-3568.g165a15e5. 245 kB/s | 127 kB 00:00 2024-05-09T04:30:52.625 INFO:teuthology.orchestra.run.smithi192.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.2 MB/s | 253 kB 00:00 2024-05-09T04:30:52.733 INFO:teuthology.orchestra.run.smithi192.stdout:(43/156): python3-cryptography-36.0.1-4.el9.x86 5.1 MB/s | 1.2 MB 00:00 2024-05-09T04:30:52.767 INFO:teuthology.orchestra.run.smithi192.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 752 kB/s | 106 kB 00:00 2024-05-09T04:30:52.842 INFO:teuthology.orchestra.run.smithi145.stdout:(18/156): ceph-base-19.0.0-3568.g165a15e5.el9.x 619 kB/s | 5.4 MB 00:08 2024-05-09T04:30:52.850 INFO:teuthology.orchestra.run.smithi192.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-05-09T04:30:52.875 INFO:teuthology.orchestra.run.smithi145.stdout:(19/156): libradosstriper1-19.0.0-3568.g165a15e 1.5 MB/s | 500 kB 00:00 2024-05-09T04:30:52.884 INFO:teuthology.orchestra.run.smithi192.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.0 MB/s | 125 kB 00:00 2024-05-09T04:30:52.917 INFO:teuthology.orchestra.run.smithi145.stdout:(20/156): python3-ceph-argparse-19.0.0-3568.g16 606 kB/s | 45 kB 00:00 2024-05-09T04:30:52.954 INFO:teuthology.orchestra.run.smithi145.stdout:(21/156): python3-ceph-common-19.0.0-3568.g165a 1.8 MB/s | 135 kB 00:00 2024-05-09T04:30:52.984 INFO:teuthology.orchestra.run.smithi192.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 1.6 MB/s | 215 kB 00:00 2024-05-09T04:30:53.026 INFO:teuthology.orchestra.run.smithi192.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 4.8 MB/s | 689 kB 00:00 2024-05-09T04:30:53.068 INFO:teuthology.orchestra.run.smithi145.stdout:(22/156): librgw2-19.0.0-3568.g165a15e5.el9.x86 11 MB/s | 5.4 MB 00:00 2024-05-09T04:30:53.093 INFO:teuthology.orchestra.run.smithi192.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-05-09T04:30:53.101 INFO:teuthology.orchestra.run.smithi145.stdout:(23/156): python3-cephfs-19.0.0-3568.g165a15e5. 883 kB/s | 162 kB 00:00 2024-05-09T04:30:53.135 INFO:teuthology.orchestra.run.smithi145.stdout:(24/156): python3-rados-19.0.0-3568.g165a15e5.e 1.7 MB/s | 321 kB 00:00 2024-05-09T04:30:53.144 INFO:teuthology.orchestra.run.smithi192.stdout:(50/156): bison-3.7.4-5.el9.x86_64.rpm 5.8 MB/s | 943 kB 00:00 2024-05-09T04:30:53.169 INFO:teuthology.orchestra.run.smithi145.stdout:(25/156): python3-rbd-19.0.0-3568.g165a15e5.el9 2.9 MB/s | 299 kB 00:00 2024-05-09T04:30:53.170 INFO:teuthology.orchestra.run.smithi192.stdout:(51/156): elfutils-libelf-devel-0.190-2.el9.x86 289 kB/s | 21 kB 00:00 2024-05-09T04:30:53.202 INFO:teuthology.orchestra.run.smithi145.stdout:(26/156): rbd-fuse-19.0.0-3568.g165a15e5.el9.x8 1.3 MB/s | 85 kB 00:00 2024-05-09T04:30:53.220 INFO:teuthology.orchestra.run.smithi192.stdout:(52/156): flexiblas-3.0.4-8.el9.x86_64.rpm 666 kB/s | 33 kB 00:00 2024-05-09T04:30:53.244 INFO:teuthology.orchestra.run.smithi145.stdout:(27/156): python3-rgw-19.0.0-3568.g165a15e5.el9 703 kB/s | 100 kB 00:00 2024-05-09T04:30:53.253 INFO:teuthology.orchestra.run.smithi192.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-05-09T04:30:53.304 INFO:teuthology.orchestra.run.smithi192.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 370 kB/s | 18 kB 00:00 2024-05-09T04:30:53.311 INFO:teuthology.orchestra.run.smithi145.stdout:(28/156): rbd-nbd-19.0.0-3568.g165a15e5.el9.x86 1.5 MB/s | 171 kB 00:00 2024-05-09T04:30:53.337 INFO:teuthology.orchestra.run.smithi145.stdout:(29/156): ceph-grafana-dashboards-19.0.0-3568.g 375 kB/s | 34 kB 00:00 2024-05-09T04:30:53.346 INFO:teuthology.orchestra.run.smithi192.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 273 kB/s | 11 kB 00:00 2024-05-09T04:30:53.420 INFO:teuthology.orchestra.run.smithi145.stdout:(30/156): rbd-mirror-19.0.0-3568.g165a15e5.el9. 12 MB/s | 3.1 MB 00:00 2024-05-09T04:30:53.454 INFO:teuthology.orchestra.run.smithi145.stdout:(31/156): ceph-mgr-cephadm-19.0.0-3568.g165a15e 1.0 MB/s | 142 kB 00:00 2024-05-09T04:30:53.454 INFO:teuthology.orchestra.run.smithi192.stdout:(56/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 13 MB/s | 3.0 MB 00:00 2024-05-09T04:30:53.479 INFO:teuthology.orchestra.run.smithi192.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-05-09T04:30:53.530 INFO:teuthology.orchestra.run.smithi192.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 609 kB/s | 45 kB 00:00 2024-05-09T04:30:53.605 INFO:teuthology.orchestra.run.smithi192.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 5.2 MB/s | 662 kB 00:00 2024-05-09T04:30:53.629 INFO:teuthology.orchestra.run.smithi145.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3568.g16 1.5 MB/s | 274 kB 00:00 2024-05-09T04:30:53.655 INFO:teuthology.orchestra.run.smithi192.stdout:(60/156): libtool-2.4.6-45.el9.x86_64.rpm 4.6 MB/s | 582 kB 00:00 2024-05-09T04:30:53.681 INFO:teuthology.orchestra.run.smithi192.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 343 kB/s | 25 kB 00:00 2024-05-09T04:30:53.729 INFO:teuthology.orchestra.run.smithi145.stdout:(33/156): ceph-mgr-rook-19.0.0-3568.g165a15e5.e 498 kB/s | 49 kB 00:00 2024-05-09T04:30:53.739 INFO:teuthology.orchestra.run.smithi192.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 573 kB/s | 47 kB 00:00 2024-05-09T04:30:53.790 INFO:teuthology.orchestra.run.smithi192.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.7 MB/s | 292 kB 00:00 2024-05-09T04:30:53.831 INFO:teuthology.orchestra.run.smithi192.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-05-09T04:30:53.848 INFO:teuthology.orchestra.run.smithi192.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 610 kB/s | 35 kB 00:00 2024-05-09T04:30:53.946 INFO:teuthology.orchestra.run.smithi145.stdout:(34/156): ceph-mgr-diskprediction-local-19.0.0- 14 MB/s | 7.4 MB 00:00 2024-05-09T04:30:53.972 INFO:teuthology.orchestra.run.smithi145.stdout:(35/156): ceph-prometheus-alerts-19.0.0-3568.g1 68 kB/s | 16 kB 00:00 2024-05-09T04:30:54.005 INFO:teuthology.orchestra.run.smithi145.stdout:(36/156): ceph-volume-19.0.0-3568.g165a15e5.el9 4.6 MB/s | 273 kB 00:00 2024-05-09T04:30:54.065 INFO:teuthology.orchestra.run.smithi192.stdout:(66/156): openblas-openmp-0.3.21-2.el9.x86_64.r 20 MB/s | 4.7 MB 00:00 2024-05-09T04:30:54.072 INFO:teuthology.orchestra.run.smithi145.stdout:(37/156): cephfs-top-19.0.0-3568.g165a15e5.el9. 276 kB/s | 18 kB 00:00 2024-05-09T04:30:54.216 INFO:teuthology.orchestra.run.smithi192.stdout:(67/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-05-09T04:30:54.256 INFO:teuthology.orchestra.run.smithi145.stdout:(38/156): cephadm-19.0.0-3568.g165a15e5.el9.noa 2.6 MB/s | 764 kB 00:00 2024-05-09T04:30:54.324 INFO:teuthology.orchestra.run.smithi192.stdout:(68/156): postgresql-contrib-13.14-1.el9.x86_64 8.3 MB/s | 911 kB 00:00 2024-05-09T04:30:54.391 INFO:teuthology.orchestra.run.smithi192.stdout:(69/156): postgresql-13.14-1.el9.x86_64.rpm 4.9 MB/s | 1.6 MB 00:00 2024-05-09T04:30:54.433 INFO:teuthology.orchestra.run.smithi192.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 1.2 MB/s | 133 kB 00:00 2024-05-09T04:30:54.550 INFO:teuthology.orchestra.run.smithi192.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 8.7 MB/s | 1.0 MB 00:00 2024-05-09T04:30:54.642 INFO:teuthology.orchestra.run.smithi192.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 964 kB/s | 87 kB 00:00 2024-05-09T04:30:54.767 INFO:teuthology.orchestra.run.smithi192.stdout:(73/156): postgresql-server-13.14-1.el9.x86_64. 15 MB/s | 5.8 MB 00:00 2024-05-09T04:30:54.851 INFO:teuthology.orchestra.run.smithi192.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 2.9 MB/s | 244 kB 00:00 2024-05-09T04:30:54.934 INFO:teuthology.orchestra.run.smithi192.stdout:(75/156): python3-jinja2-2.11.3-5.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-05-09T04:30:54.994 INFO:teuthology.orchestra.run.smithi192.stdout:(76/156): python3-jsonpatch-1.21-16.el9.noarch. 445 kB/s | 26 kB 00:00 2024-05-09T04:30:55.073 INFO:teuthology.orchestra.run.smithi145.stdout:(39/156): fuse-2.9.9-15.el9.x86_64.rpm 80 kB/s | 80 kB 00:00 2024-05-09T04:30:55.119 INFO:teuthology.orchestra.run.smithi192.stdout:(77/156): python3-babel-2.9.1-2.el9.noarch.rpm 13 MB/s | 6.0 MB 00:00 2024-05-09T04:30:55.161 INFO:teuthology.orchestra.run.smithi192.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 115 kB/s | 19 kB 00:00 2024-05-09T04:30:55.203 INFO:teuthology.orchestra.run.smithi192.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-05-09T04:30:55.228 INFO:teuthology.orchestra.run.smithi192.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 525 kB/s | 35 kB 00:00 2024-05-09T04:30:55.312 INFO:teuthology.orchestra.run.smithi192.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.0 MB/s | 250 kB 00:00 2024-05-09T04:30:55.395 INFO:teuthology.orchestra.run.smithi192.stdout:(82/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 222 kB 00:00 2024-05-09T04:30:55.562 INFO:teuthology.orchestra.run.smithi192.stdout:(83/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 14 MB/s | 5.1 MB 00:00 2024-05-09T04:30:55.615 INFO:teuthology.orchestra.run.smithi145.stdout:(40/156): libgfortran-11.4.1-3.el9.x86_64.rpm 591 kB/s | 803 kB 00:01 2024-05-09T04:30:55.622 INFO:teuthology.orchestra.run.smithi192.stdout:(84/156): python3-prettytable-0.7.2-27.el9.noar 731 kB/s | 42 kB 00:00 2024-05-09T04:30:55.705 INFO:teuthology.orchestra.run.smithi192.stdout:(85/156): python3-protobuf-3.14.0-13.el9.noarch 3.2 MB/s | 269 kB 00:00 2024-05-09T04:30:55.772 INFO:teuthology.orchestra.run.smithi192.stdout:(86/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.4 MB/s | 159 kB 00:00 2024-05-09T04:30:55.855 INFO:teuthology.orchestra.run.smithi192.stdout:(87/156): python3-pyasn1-modules-0.4.8-6.el9.no 3.3 MB/s | 279 kB 00:00 2024-05-09T04:30:55.914 INFO:teuthology.orchestra.run.smithi192.stdout:(88/156): python3-pytz-2021.1-5.el9.noarch.rpm 888 kB/s | 51 kB 00:00 2024-05-09T04:30:55.972 INFO:teuthology.orchestra.run.smithi192.stdout:(89/156): python3-requests-oauthlib-1.3.0-12.el 935 kB/s | 54 kB 00:00 2024-05-09T04:30:56.106 INFO:teuthology.orchestra.run.smithi145.stdout:(41/156): libquadmath-11.4.1-3.el9.x86_64.rpm 189 kB/s | 195 kB 00:01 2024-05-09T04:30:56.498 INFO:teuthology.orchestra.run.smithi192.stdout:(90/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 31 MB/s | 16 MB 00:00 2024-05-09T04:30:56.556 INFO:teuthology.orchestra.run.smithi192.stdout:(91/156): python3-toml-0.10.2-6.el9.noarch.rpm 722 kB/s | 42 kB 00:00 2024-05-09T04:30:56.573 INFO:teuthology.orchestra.run.smithi145.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 265 kB/s | 253 kB 00:00 2024-05-09T04:30:56.581 INFO:teuthology.orchestra.run.smithi192.stdout:(92/156): python3-packaging-20.9-5.el9.noarch.r 65 kB/s | 77 kB 00:01 2024-05-09T04:30:56.632 INFO:teuthology.orchestra.run.smithi192.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 595 kB/s | 44 kB 00:00 2024-05-09T04:30:56.682 INFO:teuthology.orchestra.run.smithi192.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.2 MB/s | 221 kB 00:00 2024-05-09T04:30:56.707 INFO:teuthology.orchestra.run.smithi192.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 483 kB/s | 36 kB 00:00 2024-05-09T04:30:56.758 INFO:teuthology.orchestra.run.smithi192.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 638 kB/s | 47 kB 00:00 2024-05-09T04:30:56.817 INFO:teuthology.orchestra.run.smithi192.stdout:(97/156): socat-1.7.4.1-5.el9.x86_64.rpm 2.8 MB/s | 305 kB 00:00 2024-05-09T04:30:56.842 INFO:teuthology.orchestra.run.smithi192.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 700 kB/s | 58 kB 00:00 2024-05-09T04:30:56.900 INFO:teuthology.orchestra.run.smithi192.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.1 MB/s | 94 kB 00:00 2024-05-09T04:30:56.926 INFO:teuthology.orchestra.run.smithi192.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 771 kB/s | 64 kB 00:00 2024-05-09T04:30:56.974 INFO:teuthology.orchestra.run.smithi145.stdout:(43/156): ceph-mgr-dashboard-19.0.0-3568.g165a1 997 kB/s | 3.5 MB 00:03 2024-05-09T04:30:56.976 INFO:teuthology.orchestra.run.smithi192.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 602 kB/s | 45 kB 00:00 2024-05-09T04:30:57.118 INFO:teuthology.orchestra.run.smithi192.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 117 kB/s | 22 kB 00:00 2024-05-09T04:30:57.265 INFO:teuthology.orchestra.run.smithi145.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 1.1 MB/s | 1.2 MB 00:01 2024-05-09T04:30:57.432 INFO:teuthology.orchestra.run.smithi145.stdout:(45/156): python3-ply-3.11-14.el9.noarch.rpm 124 kB/s | 106 kB 00:00 2024-05-09T04:30:57.743 INFO:teuthology.orchestra.run.smithi192.stdout:(103/156): abseil-cpp-20211102.0-3.el9.x86_64.r 879 kB/s | 548 kB 00:00 2024-05-09T04:30:57.785 INFO:teuthology.orchestra.run.smithi192.stdout:(104/156): protobuf-compiler-3.14.0-13.el9.x86_ 1.0 MB/s | 863 kB 00:00 2024-05-09T04:30:57.869 INFO:teuthology.orchestra.run.smithi192.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 2.4 MB/s | 309 kB 00:00 2024-05-09T04:30:57.894 INFO:teuthology.orchestra.run.smithi192.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 180 kB/s | 19 kB 00:00 2024-05-09T04:30:58.008 INFO:teuthology.orchestra.run.smithi145.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 131 kB/s | 135 kB 00:01 2024-05-09T04:30:58.053 INFO:teuthology.orchestra.run.smithi192.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 165 kB/s | 26 kB 00:00 2024-05-09T04:30:58.236 INFO:teuthology.orchestra.run.smithi192.stdout:(108/156): liboath-2.6.7-2.el9.x86_64.rpm 267 kB/s | 49 kB 00:00 2024-05-09T04:30:58.258 INFO:teuthology.orchestra.run.smithi145.stdout:(47/156): python3-requests-2.25.1-8.el9.noarch. 135 kB/s | 125 kB 00:00 2024-05-09T04:30:58.367 INFO:teuthology.orchestra.run.smithi145.stdout:(48/156): python3-urllib3-1.26.5-5.el9.noarch.r 231 kB/s | 215 kB 00:00 2024-05-09T04:30:58.495 INFO:teuthology.orchestra.run.smithi192.stdout:(109/156): libunwind-1.6.2-1.el9.x86_64.rpm 263 kB/s | 67 kB 00:00 2024-05-09T04:30:58.729 INFO:teuthology.orchestra.run.smithi192.stdout:(110/156): libarrow-9.0.0-11.el9.x86_64.rpm 5.1 MB/s | 4.4 MB 00:00 2024-05-09T04:30:58.754 INFO:teuthology.orchestra.run.smithi192.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 594 kB/s | 151 kB 00:00 2024-05-09T04:30:58.834 INFO:teuthology.orchestra.run.smithi145.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 836 kB/s | 689 kB 00:00 2024-05-09T04:30:58.859 INFO:teuthology.orchestra.run.smithi145.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 218 kB/s | 107 kB 00:00 2024-05-09T04:30:58.879 INFO:teuthology.orchestra.run.smithi192.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 5.5 MB/s | 839 kB 00:00 2024-05-09T04:30:58.938 INFO:teuthology.orchestra.run.smithi192.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 2.9 MB/s | 548 kB 00:00 2024-05-09T04:30:58.943 INFO:teuthology.orchestra.run.smithi145.stdout:(51/156): elfutils-libelf-devel-0.190-2.el9.x86 199 kB/s | 21 kB 00:00 2024-05-09T04:30:58.963 INFO:teuthology.orchestra.run.smithi192.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 556 kB/s | 43 kB 00:00 2024-05-09T04:30:59.022 INFO:teuthology.orchestra.run.smithi192.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 391 kB/s | 32 kB 00:00 2024-05-09T04:30:59.047 INFO:teuthology.orchestra.run.smithi192.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 181 kB/s | 15 kB 00:00 2024-05-09T04:30:59.114 INFO:teuthology.orchestra.run.smithi192.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 1.8 MB/s | 172 kB 00:00 2024-05-09T04:30:59.118 INFO:teuthology.orchestra.run.smithi145.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 1.1 MB/s | 943 kB 00:00 2024-05-09T04:30:59.148 INFO:teuthology.orchestra.run.smithi192.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-05-09T04:30:59.226 INFO:teuthology.orchestra.run.smithi145.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 115 kB/s | 33 kB 00:00 2024-05-09T04:30:59.260 INFO:teuthology.orchestra.run.smithi145.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 783 kB/s | 313 kB 00:00 2024-05-09T04:30:59.327 INFO:teuthology.orchestra.run.smithi145.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 183 kB/s | 18 kB 00:00 2024-05-09T04:30:59.360 INFO:teuthology.orchestra.run.smithi145.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 112 kB/s | 11 kB 00:00 2024-05-09T04:30:59.461 INFO:teuthology.orchestra.run.smithi145.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-05-09T04:30:59.561 INFO:teuthology.orchestra.run.smithi145.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 226 kB/s | 45 kB 00:00 2024-05-09T04:30:59.637 INFO:teuthology.orchestra.run.smithi145.stdout:(59/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.8 MB/s | 3.0 MB 00:00 2024-05-09T04:30:59.665 INFO:teuthology.orchestra.run.smithi192.stdout:(119/156): ceph-test-19.0.0-3568.g165a15e5.el9. 3.4 MB/s | 49 MB 00:14 2024-05-09T04:30:59.698 INFO:teuthology.orchestra.run.smithi192.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 523 kB/s | 303 kB 00:00 2024-05-09T04:30:59.723 INFO:teuthology.orchestra.run.smithi192.stdout:(121/156): python3-colorama-0.4.4-12.el9.noarch 61 kB/s | 35 kB 00:00 2024-05-09T04:30:59.757 INFO:teuthology.orchestra.run.smithi192.stdout:(122/156): python3-google-auth-2.29.0-1.el9.noa 2.3 MB/s | 206 kB 00:00 2024-05-09T04:30:59.761 INFO:teuthology.orchestra.run.smithi145.stdout:(60/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 204 kB/s | 25 kB 00:00 2024-05-09T04:30:59.812 INFO:teuthology.orchestra.run.smithi145.stdout:(61/156): librdkafka-1.6.1-102.el9.x86_64.rpm 1.8 MB/s | 662 kB 00:00 2024-05-09T04:30:59.887 INFO:teuthology.orchestra.run.smithi145.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 381 kB/s | 47 kB 00:00 2024-05-09T04:30:59.946 INFO:teuthology.orchestra.run.smithi145.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.1 MB/s | 292 kB 00:00 2024-05-09T04:30:59.979 INFO:teuthology.orchestra.run.smithi145.stdout:(64/156): libtool-2.4.6-45.el9.x86_64.rpm 1.4 MB/s | 582 kB 00:00 2024-05-09T04:31:00.007 INFO:teuthology.orchestra.run.smithi192.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 43 kB/s | 11 kB 00:00 2024-05-09T04:31:00.022 INFO:teuthology.orchestra.run.smithi145.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 1.4 MB/s | 188 kB 00:00 2024-05-09T04:31:00.075 INFO:teuthology.orchestra.run.smithi192.stdout:(124/156): python3-grpcio-1.46.7-10.el9.x86_64. 5.9 MB/s | 2.0 MB 00:00 2024-05-09T04:31:00.080 INFO:teuthology.orchestra.run.smithi145.stdout:(66/156): openblas-0.3.21-2.el9.x86_64.rpm 264 kB/s | 35 kB 00:00 2024-05-09T04:31:00.108 INFO:teuthology.orchestra.run.smithi192.stdout:(125/156): python3-grpcio-tools-1.46.7-10.el9.x 379 kB/s | 144 kB 00:00 2024-05-09T04:31:00.133 INFO:teuthology.orchestra.run.smithi192.stdout:(126/156): python3-jaraco-classes-3.2.1-5.el9.n 148 kB/s | 18 kB 00:00 2024-05-09T04:31:00.168 INFO:teuthology.orchestra.run.smithi192.stdout:(127/156): python3-jaraco-collections-3.0.0-8.e 255 kB/s | 23 kB 00:00 2024-05-09T04:31:00.193 INFO:teuthology.orchestra.run.smithi192.stdout:(128/156): python3-jaraco-functools-3.5.0-2.el9 234 kB/s | 19 kB 00:00 2024-05-09T04:31:00.243 INFO:teuthology.orchestra.run.smithi192.stdout:(129/156): python3-jaraco-text-3.2.0-6.el9.noar 181 kB/s | 20 kB 00:00 2024-05-09T04:31:00.409 INFO:teuthology.orchestra.run.smithi192.stdout:(130/156): python3-kubernetes-26.1.0-2.el9.noar 4.2 MB/s | 1.0 MB 00:00 2024-05-09T04:31:00.435 INFO:teuthology.orchestra.run.smithi192.stdout:(131/156): python3-logutils-0.3.5-21.el9.noarch 192 kB/s | 46 kB 00:00 2024-05-09T04:31:00.447 INFO:teuthology.orchestra.run.smithi145.stdout:(67/156): postgresql-13.14-1.el9.x86_64.rpm 4.3 MB/s | 1.6 MB 00:00 2024-05-09T04:31:00.461 INFO:teuthology.orchestra.run.smithi192.stdout:(132/156): python3-more-itertools-8.12.0-2.el9. 369 kB/s | 79 kB 00:00 2024-05-09T04:31:00.485 INFO:teuthology.orchestra.run.smithi192.stdout:(133/156): python3-natsort-7.1.1-5.el9.noarch.r 782 kB/s | 58 kB 00:00 2024-05-09T04:31:00.539 INFO:teuthology.orchestra.run.smithi145.stdout:(68/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 8.6 MB/s | 4.4 MB 00:00 2024-05-09T04:31:00.552 INFO:teuthology.orchestra.run.smithi192.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-05-09T04:31:00.578 INFO:teuthology.orchestra.run.smithi192.stdout:(135/156): python3-portend-3.1.0-2.el9.noarch.r 141 kB/s | 16 kB 00:00 2024-05-09T04:31:00.597 INFO:teuthology.orchestra.run.smithi145.stdout:(69/156): postgresql-contrib-13.14-1.el9.x86_64 6.0 MB/s | 911 kB 00:00 2024-05-09T04:31:00.603 INFO:teuthology.orchestra.run.smithi192.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 769 kB/s | 90 kB 00:00 2024-05-09T04:31:00.628 INFO:teuthology.orchestra.run.smithi192.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 381 kB/s | 28 kB 00:00 2024-05-09T04:31:00.662 INFO:teuthology.orchestra.run.smithi192.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 370 kB/s | 31 kB 00:00 2024-05-09T04:31:00.696 INFO:teuthology.orchestra.run.smithi192.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-05-09T04:31:00.721 INFO:teuthology.orchestra.run.smithi192.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 644 kB/s | 59 kB 00:00 2024-05-09T04:31:00.754 INFO:teuthology.orchestra.run.smithi192.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 399 kB/s | 36 kB 00:00 2024-05-09T04:31:00.779 INFO:teuthology.orchestra.run.smithi192.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 618 kB/s | 51 kB 00:00 2024-05-09T04:31:00.813 INFO:teuthology.orchestra.run.smithi192.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-05-09T04:31:00.847 INFO:teuthology.orchestra.run.smithi192.stdout:(144/156): python3-websocket-client-1.2.3-2.el9 985 kB/s | 90 kB 00:00 2024-05-09T04:31:00.857 INFO:teuthology.orchestra.run.smithi145.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 421 kB/s | 133 kB 00:00 2024-05-09T04:31:00.940 INFO:teuthology.orchestra.run.smithi192.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 2.7 MB/s | 427 kB 00:00 2024-05-09T04:31:00.964 INFO:teuthology.orchestra.run.smithi192.stdout:(146/156): python3-zc-lockfile-2.0-10.el9.noarc 133 kB/s | 20 kB 00:00 2024-05-09T04:31:00.997 INFO:teuthology.orchestra.run.smithi192.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 1.3 MB/s | 192 kB 00:00 2024-05-09T04:31:01.015 INFO:teuthology.orchestra.run.smithi145.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 6.4 MB/s | 1.0 MB 00:00 2024-05-09T04:31:01.148 INFO:teuthology.orchestra.run.smithi145.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 659 kB/s | 87 kB 00:00 2024-05-09T04:31:01.265 INFO:teuthology.orchestra.run.smithi145.stdout:(73/156): postgresql-server-13.14-1.el9.x86_64. 8.7 MB/s | 5.8 MB 00:00 2024-05-09T04:31:01.272 INFO:teuthology.orchestra.run.smithi192.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 5.2 MB/s | 1.6 MB 00:00 2024-05-09T04:31:01.390 INFO:teuthology.orchestra.run.smithi145.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 1.9 MB/s | 244 kB 00:00 2024-05-09T04:31:01.524 INFO:teuthology.orchestra.run.smithi145.stdout:(75/156): python3-jinja2-2.11.3-5.el9.noarch.rp 1.8 MB/s | 248 kB 00:00 2024-05-09T04:31:01.649 INFO:teuthology.orchestra.run.smithi145.stdout:(76/156): python3-jsonpatch-1.21-16.el9.noarch. 206 kB/s | 26 kB 00:00 2024-05-09T04:31:01.783 INFO:teuthology.orchestra.run.smithi145.stdout:(77/156): python3-jsonpointer-2.0-4.el9.noarch. 144 kB/s | 19 kB 00:00 2024-05-09T04:31:01.831 INFO:teuthology.orchestra.run.smithi192.stdout:(149/156): librados2-19.0.0-3568.g165a15e5.el9. 3.9 MB/s | 3.4 MB 00:00 2024-05-09T04:31:01.925 INFO:teuthology.orchestra.run.smithi145.stdout:(78/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.2 MB/s | 172 kB 00:00 2024-05-09T04:31:02.050 INFO:teuthology.orchestra.run.smithi145.stdout:(79/156): python3-markupsafe-1.1.1-12.el9.x86_6 279 kB/s | 35 kB 00:00 2024-05-09T04:31:02.531 INFO:teuthology.orchestra.run.smithi192.stdout:(150/156): NetworkManager-1.47.5-1.el9.x86_64.r 1.8 MB/s | 2.3 MB 00:01 2024-05-09T04:31:02.675 INFO:teuthology.orchestra.run.smithi192.stdout:(151/156): librbd1-19.0.0-3568.g165a15e5.el9.x8 1.9 MB/s | 3.1 MB 00:01 2024-05-09T04:31:02.699 INFO:teuthology.orchestra.run.smithi192.stdout:(152/156): NetworkManager-team-1.47.5-1.el9.x86 251 kB/s | 42 kB 00:00 2024-05-09T04:31:03.032 INFO:teuthology.orchestra.run.smithi192.stdout:(153/156): NetworkManager-libnm-1.47.5-1.el9.x8 1.5 MB/s | 1.8 MB 00:01 2024-05-09T04:31:03.399 INFO:teuthology.orchestra.run.smithi192.stdout:(154/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 6.5 MB/s | 2.4 MB 00:00 2024-05-09T04:31:03.424 INFO:teuthology.orchestra.run.smithi192.stdout:(155/156): NetworkManager-tui-1.47.5-1.el9.x86_ 332 kB/s | 248 kB 00:00 2024-05-09T04:31:05.368 INFO:teuthology.orchestra.run.smithi192.stdout:(156/156): openssl-3.2.1-1.el9.x86_64.rpm 518 kB/s | 1.3 MB 00:02 2024-05-09T04:31:05.376 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-05-09T04:31:05.377 INFO:teuthology.orchestra.run.smithi192.stdout:Total 9.4 MB/s | 226 MB 00:23 2024-05-09T04:31:06.042 INFO:teuthology.orchestra.run.smithi145.stdout:(80/156): openblas-openmp-0.3.21-2.el9.x86_64.r 800 kB/s | 4.7 MB 00:06 2024-05-09T04:31:06.250 INFO:teuthology.orchestra.run.smithi145.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.2 MB/s | 250 kB 00:00 2024-05-09T04:31:06.376 INFO:teuthology.orchestra.run.smithi145.stdout:(82/156): python3-oauthlib-3.1.1-5.el9.noarch.r 1.7 MB/s | 222 kB 00:00 2024-05-09T04:31:06.501 INFO:teuthology.orchestra.run.smithi145.stdout:(83/156): python3-packaging-20.9-5.el9.noarch.r 620 kB/s | 77 kB 00:00 2024-05-09T04:31:06.618 INFO:teuthology.orchestra.run.smithi145.stdout:(84/156): python3-prettytable-0.7.2-27.el9.noar 364 kB/s | 42 kB 00:00 2024-05-09T04:31:06.843 INFO:teuthology.orchestra.run.smithi145.stdout:(85/156): python3-protobuf-3.14.0-13.el9.noarch 1.2 MB/s | 269 kB 00:00 2024-05-09T04:31:06.985 INFO:teuthology.orchestra.run.smithi145.stdout:(86/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.1 MB/s | 159 kB 00:00 2024-05-09T04:31:07.219 INFO:teuthology.orchestra.run.smithi145.stdout:(87/156): python3-pyasn1-modules-0.4.8-6.el9.no 1.2 MB/s | 279 kB 00:00 2024-05-09T04:31:07.344 INFO:teuthology.orchestra.run.smithi145.stdout:(88/156): python3-pytz-2021.1-5.el9.noarch.rpm 410 kB/s | 51 kB 00:00 2024-05-09T04:31:07.461 INFO:teuthology.orchestra.run.smithi145.stdout:(89/156): python3-requests-oauthlib-1.3.0-12.el 462 kB/s | 54 kB 00:00 2024-05-09T04:31:08.053 INFO:teuthology.orchestra.run.smithi145.stdout:(90/156): python3-babel-2.9.1-2.el9.noarch.rpm 884 kB/s | 6.0 MB 00:06 2024-05-09T04:31:08.178 INFO:teuthology.orchestra.run.smithi145.stdout:(91/156): python3-toml-0.10.2-6.el9.noarch.rpm 336 kB/s | 42 kB 00:00 2024-05-09T04:31:08.320 INFO:teuthology.orchestra.run.smithi145.stdout:(92/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 313 kB/s | 44 kB 00:00 2024-05-09T04:31:08.426 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-05-09T04:31:08.667 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-05-09T04:31:08.667 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-05-09T04:31:08.737 INFO:teuthology.orchestra.run.smithi145.stdout:(93/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 781 kB/s | 5.1 MB 00:06 2024-05-09T04:31:08.771 INFO:teuthology.orchestra.run.smithi145.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 491 kB/s | 221 kB 00:00 2024-05-09T04:31:08.854 INFO:teuthology.orchestra.run.smithi145.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 309 kB/s | 36 kB 00:00 2024-05-09T04:31:08.888 INFO:teuthology.orchestra.run.smithi145.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 408 kB/s | 47 kB 00:00 2024-05-09T04:31:08.980 INFO:teuthology.orchestra.run.smithi145.stdout:(97/156): socat-1.7.4.1-5.el9.x86_64.rpm 2.4 MB/s | 305 kB 00:00 2024-05-09T04:31:09.005 INFO:teuthology.orchestra.run.smithi145.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 497 kB/s | 58 kB 00:00 2024-05-09T04:31:09.097 INFO:teuthology.orchestra.run.smithi145.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 807 kB/s | 94 kB 00:00 2024-05-09T04:31:09.403 INFO:teuthology.orchestra.run.smithi145.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 160 kB/s | 64 kB 00:00 2024-05-09T04:31:09.456 INFO:teuthology.orchestra.run.smithi145.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 125 kB/s | 45 kB 00:00 2024-05-09T04:31:09.615 INFO:teuthology.orchestra.run.smithi145.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 106 kB/s | 22 kB 00:00 2024-05-09T04:31:09.840 INFO:teuthology.orchestra.run.smithi145.stdout:(103/156): abseil-cpp-20211102.0-3.el9.x86_64.r 2.4 MB/s | 548 kB 00:00 2024-05-09T04:31:09.890 INFO:teuthology.orchestra.run.smithi145.stdout:(104/156): protobuf-compiler-3.14.0-13.el9.x86_ 1.9 MB/s | 863 kB 00:00 2024-05-09T04:31:09.924 INFO:teuthology.orchestra.run.smithi145.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 3.6 MB/s | 309 kB 00:00 2024-05-09T04:31:09.999 INFO:teuthology.orchestra.run.smithi145.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 180 kB/s | 19 kB 00:00 2024-05-09T04:31:10.066 INFO:teuthology.orchestra.run.smithi145.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 394 kB/s | 26 kB 00:00 2024-05-09T04:31:10.208 INFO:teuthology.orchestra.run.smithi145.stdout:(108/156): libarrow-9.0.0-11.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-05-09T04:31:10.233 INFO:teuthology.orchestra.run.smithi145.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 292 kB/s | 49 kB 00:00 2024-05-09T04:31:10.259 INFO:teuthology.orchestra.run.smithi145.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-05-09T04:31:10.326 INFO:teuthology.orchestra.run.smithi145.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 1.6 MB/s | 151 kB 00:00 2024-05-09T04:31:10.376 INFO:teuthology.orchestra.run.smithi145.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.1 MB/s | 839 kB 00:00 2024-05-09T04:31:10.426 INFO:teuthology.orchestra.run.smithi145.stdout:(113/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 884 kB/s | 43 kB 00:00 2024-05-09T04:31:10.468 INFO:teuthology.orchestra.run.smithi145.stdout:(114/156): python3-asyncssh-2.13.2-5.el9.noarch 3.8 MB/s | 548 kB 00:00 2024-05-09T04:31:10.485 INFO:teuthology.orchestra.run.smithi145.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 556 kB/s | 32 kB 00:00 2024-05-09T04:31:10.519 INFO:teuthology.orchestra.run.smithi145.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 302 kB/s | 15 kB 00:00 2024-05-09T04:31:10.552 INFO:teuthology.orchestra.run.smithi145.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-05-09T04:31:10.586 INFO:teuthology.orchestra.run.smithi145.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 5.3 MB/s | 358 kB 00:00 2024-05-09T04:31:10.620 INFO:teuthology.orchestra.run.smithi145.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.5 MB/s | 303 kB 00:00 2024-05-09T04:31:10.645 INFO:teuthology.orchestra.run.smithi145.stdout:(120/156): python3-colorama-0.4.4-12.el9.noarch 610 kB/s | 35 kB 00:00 2024-05-09T04:31:10.679 INFO:teuthology.orchestra.run.smithi145.stdout:(121/156): python3-google-auth-2.29.0-1.el9.noa 3.5 MB/s | 206 kB 00:00 2024-05-09T04:31:10.729 INFO:teuthology.orchestra.run.smithi145.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 2.9 MB/s | 144 kB 00:00 2024-05-09T04:31:10.796 INFO:teuthology.orchestra.run.smithi145.stdout:(123/156): python3-grpcio-1.46.7-10.el9.x86_64. 14 MB/s | 2.0 MB 00:00 2024-05-09T04:31:10.821 INFO:teuthology.orchestra.run.smithi145.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 117 kB/s | 11 kB 00:00 2024-05-09T04:31:10.846 INFO:teuthology.orchestra.run.smithi145.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 358 kB/s | 18 kB 00:00 2024-05-09T04:31:10.872 INFO:teuthology.orchestra.run.smithi145.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 468 kB/s | 23 kB 00:00 2024-05-09T04:31:10.897 INFO:teuthology.orchestra.run.smithi145.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 394 kB/s | 19 kB 00:00 2024-05-09T04:31:10.923 INFO:teuthology.orchestra.run.smithi145.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 398 kB/s | 20 kB 00:00 2024-05-09T04:31:10.981 INFO:teuthology.orchestra.run.smithi145.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 807 kB/s | 46 kB 00:00 2024-05-09T04:31:11.023 INFO:teuthology.orchestra.run.smithi145.stdout:(130/156): python3-kubernetes-26.1.0-2.el9.noar 8.2 MB/s | 1.0 MB 00:00 2024-05-09T04:31:11.049 INFO:teuthology.orchestra.run.smithi145.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.2 MB/s | 79 kB 00:00 2024-05-09T04:31:11.082 INFO:teuthology.orchestra.run.smithi145.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 991 kB/s | 58 kB 00:00 2024-05-09T04:31:11.116 INFO:teuthology.orchestra.run.smithi145.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-05-09T04:31:11.141 INFO:teuthology.orchestra.run.smithi145.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 284 kB/s | 16 kB 00:00 2024-05-09T04:31:11.175 INFO:teuthology.orchestra.run.smithi145.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-05-09T04:31:11.192 INFO:teuthology.orchestra.run.smithi145.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 568 kB/s | 28 kB 00:00 2024-05-09T04:31:11.217 INFO:teuthology.orchestra.run.smithi145.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 746 kB/s | 31 kB 00:00 2024-05-09T04:31:11.278 INFO:teuthology.orchestra.run.smithi145.stdout:(138/156): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-05-09T04:31:11.301 INFO:teuthology.orchestra.run.smithi145.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 712 kB/s | 59 kB 00:00 2024-05-09T04:31:11.318 INFO:teuthology.orchestra.run.smithi145.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 922 kB/s | 36 kB 00:00 2024-05-09T04:31:11.352 INFO:teuthology.orchestra.run.smithi145.stdout:(141/156): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-05-09T04:31:11.354 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-05-09T04:31:11.355 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-05-09T04:31:11.385 INFO:teuthology.orchestra.run.smithi145.stdout:(142/156): python3-webob-1.8.7-6.el9.noarch.rpm 3.4 MB/s | 230 kB 00:00 2024-05-09T04:31:11.410 INFO:teuthology.orchestra.run.smithi145.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-05-09T04:31:11.444 INFO:teuthology.orchestra.run.smithi145.stdout:(144/156): python3-werkzeug-2.0.3-3.el9.1.noarc 7.3 MB/s | 427 kB 00:00 2024-05-09T04:31:11.469 INFO:teuthology.orchestra.run.smithi145.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 345 kB/s | 20 kB 00:00 2024-05-09T04:31:11.520 INFO:teuthology.orchestra.run.smithi145.stdout:(146/156): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-05-09T04:31:11.595 INFO:teuthology.orchestra.run.smithi145.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-05-09T04:31:11.862 INFO:teuthology.orchestra.run.smithi145.stdout:(148/156): librados2-19.0.0-3568.g165a15e5.el9. 10 MB/s | 3.4 MB 00:00 2024-05-09T04:31:13.229 INFO:teuthology.orchestra.run.smithi145.stdout:(149/156): NetworkManager-1.47.5-1.el9.x86_64.r 1.7 MB/s | 2.3 MB 00:01 2024-05-09T04:31:14.537 INFO:teuthology.orchestra.run.smithi145.stdout:(150/156): NetworkManager-libnm-1.47.5-1.el9.x8 1.4 MB/s | 1.8 MB 00:01 2024-05-09T04:31:14.879 INFO:teuthology.orchestra.run.smithi145.stdout:(151/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 2.2 MB/s | 16 MB 00:07 2024-05-09T04:31:15.138 INFO:teuthology.orchestra.run.smithi145.stdout:(152/156): librbd1-19.0.0-3568.g165a15e5.el9.x8 908 kB/s | 3.1 MB 00:03 2024-05-09T04:31:15.513 INFO:teuthology.orchestra.run.smithi145.stdout:(153/156): NetworkManager-team-1.47.5-1.el9.x86 43 kB/s | 42 kB 00:00 2024-05-09T04:31:16.022 INFO:teuthology.orchestra.run.smithi145.stdout:(154/156): NetworkManager-tui-1.47.5-1.el9.x86_ 218 kB/s | 248 kB 00:01 2024-05-09T04:31:16.622 INFO:teuthology.orchestra.run.smithi145.stdout:(155/156): openssl-3.2.1-1.el9.x86_64.rpm 933 kB/s | 1.3 MB 00:01 2024-05-09T04:31:17.122 INFO:teuthology.orchestra.run.smithi145.stdout:(156/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 1.5 MB/s | 2.4 MB 00:01 2024-05-09T04:31:17.130 INFO:teuthology.orchestra.run.smithi145.stdout:-------------------------------------------------------------------------------- 2024-05-09T04:31:17.130 INFO:teuthology.orchestra.run.smithi145.stdout:Total 6.7 MB/s | 226 MB 00:33 2024-05-09T04:31:17.937 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-05-09T04:31:18.031 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-09T04:31:18.100 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-09T04:31:18.207 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-09T04:31:18.259 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-09T04:31:18.291 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-09T04:31:18.391 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-09T04:31:18.828 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-09T04:31:18.931 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : librados2-2:19.0.0-3568.g165a15e5.el9.x86_64 8/164 2024-05-09T04:31:19.103 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librados2-2:19.0.0-3568.g165a15e5.el9.x86_64 8/164 2024-05-09T04:31:19.132 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephfs2-2:19.0.0-3568.g165a15e5.el9.x86_64 9/164 2024-05-09T04:31:19.258 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libcephfs2-2:19.0.0-3568.g165a15e5.el9.x86_64 9/164 2024-05-09T04:31:19.298 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rados-2:19.0.0-3568.g165a15e5.el9.x86_64 10/164 2024-05-09T04:31:19.589 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-09T04:31:19.653 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-09T04:31:19.703 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephsqlite-2:19.0.0-3568.g165a15e5.el9.x86_64 13/164 2024-05-09T04:31:19.853 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3568.g165a15e5.el9.x86_64 13/164 2024-05-09T04:31:19.888 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libradosstriper1-2:19.0.0-3568.g165a15e5.el9.x86 14/164 2024-05-09T04:31:19.958 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3568.g165a15e5.el9.x86 14/164 2024-05-09T04:31:19.989 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T04:31:20.068 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T04:31:20.264 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction check 2024-05-09T04:31:20.368 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T04:31:20.506 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction check succeeded. 2024-05-09T04:31:20.506 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction test 2024-05-09T04:31:20.742 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T04:31:20.878 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T04:31:20.958 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T04:31:21.169 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T04:31:21.281 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T04:31:21.622 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T04:31:21.816 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-09T04:31:21.846 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-09T04:31:21.901 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-09T04:31:22.017 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-09T04:31:22.068 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-09T04:31:22.140 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-09T04:31:22.189 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-09T04:31:22.227 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-09T04:31:22.345 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-09T04:31:22.443 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-09T04:31:22.480 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-09T04:31:22.537 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-09T04:31:22.645 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-09T04:31:22.693 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-ceph-common-2:19.0.0-3568.g165a15e5.el9. 32/164 2024-05-09T04:31:22.747 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-ceph-argparse-2:19.0.0-3568.g165a15e5.el 33/164 2024-05-09T04:31:22.782 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cephfs-2:19.0.0-3568.g165a15e5.el9.x86_6 34/164 2024-05-09T04:31:22.900 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-09T04:31:22.969 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-09T04:31:23.082 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-09T04:31:23.139 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction test succeeded. 2024-05-09T04:31:23.139 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction 2024-05-09T04:31:23.284 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-09T04:31:23.356 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-09T04:31:24.020 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-09T04:31:24.126 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-09T04:31:24.204 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-09T04:31:24.252 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-09T04:31:24.310 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librados-devel-2:19.0.0-3568.g165a15e5.el9.x86_6 44/164 2024-05-09T04:31:24.384 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-09T04:31:24.596 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-09T04:31:24.673 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-09T04:31:24.791 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-09T04:31:24.868 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-09T04:31:24.954 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-09T04:31:24.991 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-09T04:31:25.029 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-09T04:31:25.111 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-09T04:31:25.156 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-09T04:31:25.218 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-09T04:31:25.276 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-09T04:31:25.474 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-09T04:31:25.546 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-09T04:31:25.583 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-09T04:31:25.640 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-09T04:31:25.770 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-09T04:31:25.808 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-09T04:31:25.854 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-09T04:31:25.898 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-09T04:31:25.930 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-09T04:31:26.140 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-09T04:31:26.328 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-09T04:31:26.379 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-09T04:31:26.418 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-09T04:31:26.467 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-09T04:31:26.532 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-09T04:31:26.582 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-09T04:31:26.623 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-09T04:31:26.676 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-09T04:31:26.728 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-09T04:31:26.805 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-09T04:31:26.852 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-09T04:31:27.006 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-09T04:31:27.067 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-09T04:31:27.109 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-09T04:31:27.179 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-09T04:31:27.326 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-09T04:31:27.389 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-09T04:31:28.357 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-09T04:31:28.413 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-09T04:31:28.446 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-09T04:31:28.571 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-09T04:31:28.633 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-09T04:31:28.754 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-09T04:31:28.806 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-09T04:31:29.039 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-09T04:31:29.069 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-09T04:31:29.175 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-09T04:31:29.586 INFO:teuthology.orchestra.run.smithi145.stdout: Preparing : 1/1 2024-05-09T04:31:29.669 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-09T04:31:29.735 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-09T04:31:29.845 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-09T04:31:29.898 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-09T04:31:29.930 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-09T04:31:29.993 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-09T04:31:30.025 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-09T04:31:30.124 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-09T04:31:30.465 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-09T04:31:30.569 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : librados2-2:19.0.0-3568.g165a15e5.el9.x86_64 8/164 2024-05-09T04:31:30.736 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: librados2-2:19.0.0-3568.g165a15e5.el9.x86_64 8/164 2024-05-09T04:31:30.762 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libcephfs2-2:19.0.0-3568.g165a15e5.el9.x86_64 9/164 2024-05-09T04:31:30.886 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: libcephfs2-2:19.0.0-3568.g165a15e5.el9.x86_64 9/164 2024-05-09T04:31:30.919 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-rados-2:19.0.0-3568.g165a15e5.el9.x86_64 10/164 2024-05-09T04:31:31.199 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-09T04:31:31.266 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-09T04:31:31.291 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libcephsqlite-2:19.0.0-3568.g165a15e5.el9.x86_64 13/164 2024-05-09T04:31:31.436 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3568.g165a15e5.el9.x86_64 13/164 2024-05-09T04:31:31.467 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libradosstriper1-2:19.0.0-3568.g165a15e5.el9.x86 14/164 2024-05-09T04:31:31.553 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3568.g165a15e5.el9.x86 14/164 2024-05-09T04:31:31.585 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T04:31:31.658 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T04:31:31.716 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-09T04:31:31.820 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-09T04:31:31.856 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-09T04:31:31.892 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-09T04:31:31.989 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T04:31:32.258 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-09T04:31:32.299 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : librbd1-2:19.0.0-3568.g165a15e5.el9.x86_64 101/164 2024-05-09T04:31:32.337 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T04:31:32.419 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librbd1-2:19.0.0-3568.g165a15e5.el9.x86_64 101/164 2024-05-09T04:31:32.455 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rbd-2:19.0.0-3568.g165a15e5.el9.x86_64 102/164 2024-05-09T04:31:32.511 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-09T04:31:32.564 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T04:31:32.654 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T04:31:32.863 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T04:31:32.969 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T04:31:33.053 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-09T04:31:33.086 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librgw2-2:19.0.0-3568.g165a15e5.el9.x86_64 105/164 2024-05-09T04:31:33.202 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librgw2-2:19.0.0-3568.g165a15e5.el9.x86_64 105/164 2024-05-09T04:31:33.229 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rgw-2:19.0.0-3568.g165a15e5.el9.x86_64 106/164 2024-05-09T04:31:33.302 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T04:31:33.508 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-09T04:31:33.544 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-09T04:31:33.595 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-09T04:31:33.702 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-09T04:31:33.755 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-09T04:31:33.819 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-09T04:31:33.867 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-09T04:31:33.907 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-09T04:31:34.022 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-09T04:31:34.106 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-09T04:31:34.143 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-09T04:31:34.199 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-09T04:31:34.305 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-09T04:31:34.345 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-ceph-common-2:19.0.0-3568.g165a15e5.el9. 32/164 2024-05-09T04:31:34.397 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-ceph-argparse-2:19.0.0-3568.g165a15e5.el 33/164 2024-05-09T04:31:34.435 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cephfs-2:19.0.0-3568.g165a15e5.el9.x86_6 34/164 2024-05-09T04:31:34.553 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-09T04:31:34.616 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-09T04:31:34.724 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-09T04:31:34.903 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-09T04:31:34.967 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-09T04:31:35.649 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-09T04:31:35.756 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-09T04:31:35.800 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-09T04:31:35.851 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-09T04:31:35.903 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : librados-devel-2:19.0.0-3568.g165a15e5.el9.x86_6 44/164 2024-05-09T04:31:35.951 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-common-2:19.0.0-3568.g165a15e5.el9.x86_64 107/164 2024-05-09T04:31:35.965 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-09T04:31:36.001 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-common-2:19.0.0-3568.g165a15e5.el9.x86_64 107/164 2024-05-09T04:31:36.167 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-09T04:31:36.235 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-09T04:31:36.340 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-09T04:31:36.420 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-09T04:31:36.514 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-09T04:31:36.561 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-09T04:31:36.601 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-09T04:31:36.632 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-common-2:19.0.0-3568.g165a15e5.el9.x86_64 107/164 2024-05-09T04:31:36.664 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-09T04:31:36.689 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-base-2:19.0.0-3568.g165a15e5.el9.x86_64 108/164 2024-05-09T04:31:36.709 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-09T04:31:36.770 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-09T04:31:36.829 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-09T04:31:36.858 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-base-2:19.0.0-3568.g165a15e5.el9.x86_64 108/164 2024-05-09T04:31:36.859 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-09T04:31:36.859 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-09T04:31:36.859 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-09T04:31:36.899 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-selinux-2:19.0.0-3568.g165a15e5.el9.x86_64 109/164 2024-05-09T04:31:37.039 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-09T04:31:37.116 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-09T04:31:37.161 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-09T04:31:37.220 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-09T04:31:37.363 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-09T04:31:37.410 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-09T04:31:37.461 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-09T04:31:37.501 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-09T04:31:37.534 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-09T04:31:37.736 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-09T04:31:37.922 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-09T04:31:37.974 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-09T04:31:38.016 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-09T04:31:38.058 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-09T04:31:38.115 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-09T04:31:38.168 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-09T04:31:38.209 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-09T04:31:38.254 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-09T04:31:38.305 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-09T04:31:38.372 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-09T04:31:38.419 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-09T04:31:38.560 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-09T04:31:38.618 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-09T04:31:38.681 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-09T04:31:38.750 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-09T04:31:38.903 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-09T04:31:38.966 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-09T04:31:39.938 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-09T04:31:39.990 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-09T04:31:40.023 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-09T04:31:40.143 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-09T04:31:40.234 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-09T04:31:40.344 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-09T04:31:40.383 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-09T04:31:40.617 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-09T04:31:40.646 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-09T04:31:40.750 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-09T04:31:41.543 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-09T04:31:41.676 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-09T04:31:43.287 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-09T04:31:43.403 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-09T04:31:43.439 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-09T04:31:43.475 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-09T04:31:43.856 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-09T04:31:43.907 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : librbd1-2:19.0.0-3568.g165a15e5.el9.x86_64 101/164 2024-05-09T04:31:44.029 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: librbd1-2:19.0.0-3568.g165a15e5.el9.x86_64 101/164 2024-05-09T04:31:44.072 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-rbd-2:19.0.0-3568.g165a15e5.el9.x86_64 102/164 2024-05-09T04:31:44.130 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-09T04:31:44.673 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-09T04:31:44.710 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : librgw2-2:19.0.0-3568.g165a15e5.el9.x86_64 105/164 2024-05-09T04:31:44.836 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: librgw2-2:19.0.0-3568.g165a15e5.el9.x86_64 105/164 2024-05-09T04:31:44.870 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-rgw-2:19.0.0-3568.g165a15e5.el9.x86_64 106/164 2024-05-09T04:31:47.652 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-common-2:19.0.0-3568.g165a15e5.el9.x86_64 107/164 2024-05-09T04:31:48.071 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-common-2:19.0.0-3568.g165a15e5.el9.x86_64 107/164 2024-05-09T04:31:48.773 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-common-2:19.0.0-3568.g165a15e5.el9.x86_64 107/164 2024-05-09T04:31:48.840 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-base-2:19.0.0-3568.g165a15e5.el9.x86_64 108/164 2024-05-09T04:31:49.001 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-base-2:19.0.0-3568.g165a15e5.el9.x86_64 108/164 2024-05-09T04:31:49.001 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-09T04:31:49.001 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-09T04:31:49.001 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:31:49.047 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-selinux-2:19.0.0-3568.g165a15e5.el9.x86_64 109/164 2024-05-09T04:31:49.943 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3568.g165a15e5.el9.x86_64 109/164 2024-05-09T04:31:49.944 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /sys 2024-05-09T04:31:49.944 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /proc 2024-05-09T04:31:49.944 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /mnt 2024-05-09T04:31:49.944 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /var/tmp 2024-05-09T04:31:49.944 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /home 2024-05-09T04:31:49.944 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /root 2024-05-09T04:31:49.944 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /tmp 2024-05-09T04:31:49.944 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-09T04:31:50.207 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-volume-2:19.0.0-3568.g165a15e5.el9.noarch 110/164 2024-05-09T04:31:50.259 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-volume-2:19.0.0-3568.g165a15e5.el9.noarch 110/164 2024-05-09T04:31:50.260 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:31:50.260 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-09T04:31:50.260 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-09T04:31:51.944 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-osd-2:19.0.0-3568.g165a15e5.el9.x86_64 111/164 2024-05-09T04:31:52.023 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-osd-2:19.0.0-3568.g165a15e5.el9.x86_64 111/164 2024-05-09T04:31:52.024 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:31:52.024 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-09T04:31:52.024 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T04:31:52.024 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T04:31:52.024 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-09T04:31:52.287 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mds-2:19.0.0-3568.g165a15e5.el9.x86_64 112/164 2024-05-09T04:31:52.351 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mds-2:19.0.0-3568.g165a15e5.el9.x86_64 112/164 2024-05-09T04:31:52.351 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:31:52.351 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-09T04:31:52.351 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T04:31:52.351 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T04:31:52.351 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-09T04:31:52.845 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mon-2:19.0.0-3568.g165a15e5.el9.x86_64 113/164 2024-05-09T04:31:52.932 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mon-2:19.0.0-3568.g165a15e5.el9.x86_64 113/164 2024-05-09T04:31:52.932 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:31:52.932 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-09T04:31:52.932 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T04:31:52.932 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T04:31:52.932 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-09T04:31:53.239 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-09T04:31:53.324 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-09T04:31:53.420 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-09T04:31:53.675 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-09T04:31:53.743 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-09T04:31:53.834 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-09T04:31:53.953 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-09T04:31:54.136 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-09T04:31:54.184 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-09T04:31:54.233 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-09T04:31:54.265 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-09T04:31:54.937 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: cephadm-2:19.0.0-3568.g165a15e5.el9.noarch 125/164 2024-05-09T04:31:54.985 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cephadm-2:19.0.0-3568.g165a15e5.el9.noarch 125/164 2024-05-09T04:31:55.043 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3568.g165a15e5.e 126/164 2024-05-09T04:31:55.151 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3568.g165a15e5. 127/164 2024-05-09T04:31:55.187 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3568.g165a15e5.el9.noa 128/164 2024-05-09T04:31:56.494 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3568.g165a15e5.el9.noa 128/164 2024-05-09T04:31:56.531 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3568.g165a15e5.el9.n 129/164 2024-05-09T04:31:57.627 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3568.g165a15e5.el9.n 129/164 2024-05-09T04:31:57.656 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3568.g165 130/164 2024-05-09T04:31:57.812 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3568.g165 130/164 2024-05-09T04:31:57.986 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3568.g165a15e5.el 131/164 2024-05-09T04:31:58.024 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-2:19.0.0-3568.g165a15e5.el9.x86_64 132/164 2024-05-09T04:31:58.094 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3568.g165a15e5.el9.x86_64 132/164 2024-05-09T04:31:58.095 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:31:58.095 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-09T04:31:58.095 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T04:31:58.095 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T04:31:58.095 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-09T04:31:58.174 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-rook-2:19.0.0-3568.g165a15e5.el9.noarch 133/164 2024-05-09T04:31:58.211 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3568.g165a15e5.el9.noarch 133/164 2024-05-09T04:31:58.337 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-2:19.0.0-3568.g165a15e5.el9.x86_64 134/164 2024-05-09T04:31:58.408 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-fuse-2:19.0.0-3568.g165a15e5.el9.x86_64 135/164 2024-05-09T04:31:59.414 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-09T04:31:59.445 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-radosgw-2:19.0.0-3568.g165a15e5.el9.x86_64 137/164 2024-05-09T04:31:59.530 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3568.g165a15e5.el9.x86_64 137/164 2024-05-09T04:31:59.530 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:31:59.531 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-09T04:31:59.531 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T04:31:59.531 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T04:31:59.531 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-09T04:31:59.579 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3568.g165a1 138/164 2024-05-09T04:31:59.671 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3568.g165a1 138/164 2024-05-09T04:31:59.672 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:31:59.672 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-09T04:31:59.672 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-09T04:31:59.746 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cephfs-mirror-2:19.0.0-3568.g165a15e5.el9.x86_64 139/164 2024-05-09T04:31:59.841 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3568.g165a15e5.el9.x86_64 139/164 2024-05-09T04:31:59.841 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:31:59.842 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-09T04:31:59.842 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T04:31:59.842 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T04:31:59.842 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-09T04:32:00.222 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-mirror-2:19.0.0-3568.g165a15e5.el9.x86_64 140/164 2024-05-09T04:32:00.301 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3568.g165a15e5.el9.x86_64 140/164 2024-05-09T04:32:00.301 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:32:00.301 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-09T04:32:00.301 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T04:32:00.301 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T04:32:00.302 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-09T04:32:02.093 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3568.g165a15e5.el9.x86_64 109/164 2024-05-09T04:32:02.093 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /sys 2024-05-09T04:32:02.093 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /proc 2024-05-09T04:32:02.093 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /mnt 2024-05-09T04:32:02.093 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /var/tmp 2024-05-09T04:32:02.093 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /home 2024-05-09T04:32:02.093 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /root 2024-05-09T04:32:02.094 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /tmp 2024-05-09T04:32:02.094 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:32:02.338 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-volume-2:19.0.0-3568.g165a15e5.el9.noarch 110/164 2024-05-09T04:32:02.388 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-volume-2:19.0.0-3568.g165a15e5.el9.noarch 110/164 2024-05-09T04:32:02.388 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:32:02.388 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-09T04:32:02.389 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:32:04.049 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-osd-2:19.0.0-3568.g165a15e5.el9.x86_64 111/164 2024-05-09T04:32:04.126 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-osd-2:19.0.0-3568.g165a15e5.el9.x86_64 111/164 2024-05-09T04:32:04.126 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:32:04.126 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-09T04:32:04.126 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T04:32:04.127 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T04:32:04.127 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:32:04.383 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mds-2:19.0.0-3568.g165a15e5.el9.x86_64 112/164 2024-05-09T04:32:04.448 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mds-2:19.0.0-3568.g165a15e5.el9.x86_64 112/164 2024-05-09T04:32:04.448 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:32:04.448 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-09T04:32:04.449 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T04:32:04.449 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T04:32:04.449 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:32:04.942 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mon-2:19.0.0-3568.g165a15e5.el9.x86_64 113/164 2024-05-09T04:32:05.009 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mon-2:19.0.0-3568.g165a15e5.el9.x86_64 113/164 2024-05-09T04:32:05.009 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:32:05.010 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-09T04:32:05.010 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T04:32:05.010 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T04:32:05.010 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:32:05.150 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-09T04:32:05.211 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-09T04:32:05.286 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-09T04:32:05.361 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-test-2:19.0.0-3568.g165a15e5.el9.x86_64 141/164 2024-05-09T04:32:05.416 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-fuse-2:19.0.0-3568.g165a15e5.el9.x86_64 142/164 2024-05-09T04:32:05.470 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-nbd-2:19.0.0-3568.g165a15e5.el9.x86_64 143/164 2024-05-09T04:32:05.511 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-09T04:32:05.519 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-09T04:32:05.550 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-09T04:32:05.580 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-09T04:32:05.664 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-09T04:32:05.781 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-09T04:32:05.840 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cephfs-shell-2:19.0.0-3568.g165a15e5.el9.x86_64 146/164 2024-05-09T04:32:05.899 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-09T04:32:05.928 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephfs-devel-2:19.0.0-3568.g165a15e5.el9.x86_ 148/164 2024-05-09T04:32:05.945 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-09T04:32:05.962 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-09T04:32:05.996 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-09T04:32:06.022 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-09T04:32:06.059 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-09T04:32:06.086 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-09T04:32:06.738 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: cephadm-2:19.0.0-3568.g165a15e5.el9.noarch 125/164 2024-05-09T04:32:06.773 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : cephadm-2:19.0.0-3568.g165a15e5.el9.noarch 125/164 2024-05-09T04:32:06.829 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3568.g165a15e5.e 126/164 2024-05-09T04:32:06.891 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T04:32:06.939 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3568.g165a15e5. 127/164 2024-05-09T04:32:06.945 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T04:32:06.966 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3568.g165a15e5.el9.noa 128/164 2024-05-09T04:32:07.000 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T04:32:08.246 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3568.g165a15e5.el9.noa 128/164 2024-05-09T04:32:08.293 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3568.g165a15e5.el9.n 129/164 2024-05-09T04:32:08.297 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cephfs-top-2:19.0.0-3568.g165a15e5.el9.noarch 152/164 2024-05-09T04:32:08.411 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-09T04:32:08.481 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-09T04:32:08.640 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-09T04:32:08.679 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-09T04:32:08.716 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-09T04:32:08.716 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T04:32:08.844 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T04:32:08.892 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-09T04:32:08.892 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T04:32:09.029 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T04:32:09.059 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-09T04:32:09.094 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T04:32:09.094 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T04:32:09.396 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T04:32:09.397 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T04:32:09.400 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3568.g165a15e5.el9.n 129/164 2024-05-09T04:32:09.435 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3568.g165 130/164 2024-05-09T04:32:09.522 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T04:32:09.591 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3568.g165 130/164 2024-05-09T04:32:09.645 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T04:32:09.765 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3568.g165a15e5.el 131/164 2024-05-09T04:32:09.813 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-2:19.0.0-3568.g165a15e5.el9.x86_64 132/164 2024-05-09T04:32:09.882 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3568.g165a15e5.el9.x86_64 132/164 2024-05-09T04:32:09.882 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:32:09.882 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-09T04:32:09.882 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T04:32:09.882 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T04:32:09.883 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:32:09.906 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T04:32:09.906 INFO:teuthology.orchestra.run.smithi192.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-05-09T04:32:09.907 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-09T04:32:09.938 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-rook-2:19.0.0-3568.g165a15e5.el9.noarch 133/164 2024-05-09T04:32:09.976 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3568.g165a15e5.el9.noarch 133/164 2024-05-09T04:32:10.115 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-2:19.0.0-3568.g165a15e5.el9.x86_64 134/164 2024-05-09T04:32:10.185 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-fuse-2:19.0.0-3568.g165a15e5.el9.x86_64 135/164 2024-05-09T04:32:11.196 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-09T04:32:11.224 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-radosgw-2:19.0.0-3568.g165a15e5.el9.x86_64 137/164 2024-05-09T04:32:11.292 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3568.g165a15e5.el9.x86_64 137/164 2024-05-09T04:32:11.293 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:32:11.293 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-09T04:32:11.293 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T04:32:11.293 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T04:32:11.293 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:32:11.350 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3568.g165a1 138/164 2024-05-09T04:32:11.420 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3568.g165a1 138/164 2024-05-09T04:32:11.421 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:32:11.421 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-09T04:32:11.421 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:32:11.509 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : cephfs-mirror-2:19.0.0-3568.g165a15e5.el9.x86_64 139/164 2024-05-09T04:32:11.583 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3568.g165a15e5.el9.x86_64 139/164 2024-05-09T04:32:11.583 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:32:11.583 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-09T04:32:11.583 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T04:32:11.583 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T04:32:11.583 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:32:11.959 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : rbd-mirror-2:19.0.0-3568.g165a15e5.el9.x86_64 140/164 2024-05-09T04:32:12.029 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3568.g165a15e5.el9.x86_64 140/164 2024-05-09T04:32:12.030 INFO:teuthology.orchestra.run.smithi145.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:32:12.030 INFO:teuthology.orchestra.run.smithi145.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-09T04:32:12.030 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T04:32:12.030 INFO:teuthology.orchestra.run.smithi145.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T04:32:12.030 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:32:15.657 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-2:19.0.0-3568.g165a15e5.el9.x86_64 1/164 2024-05-09T04:32:15.657 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-base-2:19.0.0-3568.g165a15e5.el9.x86_64 2/164 2024-05-09T04:32:15.657 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-common-2:19.0.0-3568.g165a15e5.el9.x86_64 3/164 2024-05-09T04:32:15.657 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-fuse-2:19.0.0-3568.g165a15e5.el9.x86_64 4/164 2024-05-09T04:32:15.657 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3568.g165a1 5/164 2024-05-09T04:32:15.657 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mds-2:19.0.0-3568.g165a15e5.el9.x86_64 6/164 2024-05-09T04:32:15.657 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-2:19.0.0-3568.g165a15e5.el9.x86_64 7/164 2024-05-09T04:32:15.657 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mon-2:19.0.0-3568.g165a15e5.el9.x86_64 8/164 2024-05-09T04:32:15.657 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-osd-2:19.0.0-3568.g165a15e5.el9.x86_64 9/164 2024-05-09T04:32:15.657 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-radosgw-2:19.0.0-3568.g165a15e5.el9.x86_64 10/164 2024-05-09T04:32:15.658 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-selinux-2:19.0.0-3568.g165a15e5.el9.x86_64 11/164 2024-05-09T04:32:15.658 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-test-2:19.0.0-3568.g165a15e5.el9.x86_64 12/164 2024-05-09T04:32:15.658 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cephfs-mirror-2:19.0.0-3568.g165a15e5.el9.x86_64 13/164 2024-05-09T04:32:15.658 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cephfs-shell-2:19.0.0-3568.g165a15e5.el9.x86_64 14/164 2024-05-09T04:32:15.658 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephfs-devel-2:19.0.0-3568.g165a15e5.el9.x86_ 15/164 2024-05-09T04:32:15.658 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephfs2-2:19.0.0-3568.g165a15e5.el9.x86_64 16/164 2024-05-09T04:32:15.658 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephsqlite-2:19.0.0-3568.g165a15e5.el9.x86_64 17/164 2024-05-09T04:32:15.658 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados-devel-2:19.0.0-3568.g165a15e5.el9.x86_6 18/164 2024-05-09T04:32:15.658 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libradosstriper1-2:19.0.0-3568.g165a15e5.el9.x86 19/164 2024-05-09T04:32:15.658 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librgw2-2:19.0.0-3568.g165a15e5.el9.x86_64 20/164 2024-05-09T04:32:15.659 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3568.g165a15e5.el 21/164 2024-05-09T04:32:15.659 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-ceph-common-2:19.0.0-3568.g165a15e5.el9. 22/164 2024-05-09T04:32:15.659 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cephfs-2:19.0.0-3568.g165a15e5.el9.x86_6 23/164 2024-05-09T04:32:15.659 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rados-2:19.0.0-3568.g165a15e5.el9.x86_64 24/164 2024-05-09T04:32:15.659 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rbd-2:19.0.0-3568.g165a15e5.el9.x86_64 25/164 2024-05-09T04:32:15.659 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rgw-2:19.0.0-3568.g165a15e5.el9.x86_64 26/164 2024-05-09T04:32:15.659 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-fuse-2:19.0.0-3568.g165a15e5.el9.x86_64 27/164 2024-05-09T04:32:15.672 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-mirror-2:19.0.0-3568.g165a15e5.el9.x86_64 28/164 2024-05-09T04:32:15.673 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-nbd-2:19.0.0-3568.g165a15e5.el9.x86_64 29/164 2024-05-09T04:32:15.673 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3568.g165a15e5. 30/164 2024-05-09T04:32:15.673 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3568.g165a15e5.el9.noa 31/164 2024-05-09T04:32:15.673 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3568.g165a15e5.el9.n 32/164 2024-05-09T04:32:15.673 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3568.g165 33/164 2024-05-09T04:32:15.673 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3568.g165a15e5.el 34/164 2024-05-09T04:32:15.673 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3568.g165a15e5.el9.noarch 35/164 2024-05-09T04:32:15.673 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3568.g165a15e5.e 36/164 2024-05-09T04:32:15.673 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-volume-2:19.0.0-3568.g165a15e5.el9.noarch 37/164 2024-05-09T04:32:15.673 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cephadm-2:19.0.0-3568.g165a15e5.el9.noarch 38/164 2024-05-09T04:32:15.673 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cephfs-top-2:19.0.0-3568.g165a15e5.el9.noarch 39/164 2024-05-09T04:32:15.673 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-09T04:32:15.673 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-09T04:32:15.674 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-09T04:32:15.674 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-09T04:32:15.674 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-09T04:32:15.674 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-09T04:32:15.674 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-09T04:32:15.674 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-09T04:32:15.674 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-09T04:32:15.674 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-09T04:32:15.674 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-09T04:32:15.674 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-09T04:32:15.674 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-09T04:32:15.674 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-09T04:32:15.674 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-09T04:32:15.674 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-09T04:32:15.675 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-09T04:32:15.675 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-09T04:32:15.675 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-09T04:32:15.675 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-09T04:32:15.675 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-09T04:32:15.675 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-09T04:32:15.675 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-09T04:32:15.675 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-09T04:32:15.675 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-09T04:32:15.675 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-09T04:32:15.675 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-09T04:32:15.675 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-09T04:32:15.675 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-09T04:32:15.675 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-09T04:32:15.676 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-09T04:32:15.676 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-09T04:32:15.676 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-09T04:32:15.676 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-09T04:32:15.676 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-09T04:32:15.676 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-09T04:32:15.676 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-09T04:32:15.676 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-09T04:32:15.676 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-09T04:32:15.676 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-09T04:32:15.676 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-09T04:32:15.676 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-09T04:32:15.676 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-09T04:32:15.676 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-09T04:32:15.677 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-09T04:32:15.677 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-09T04:32:15.677 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-09T04:32:15.677 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-09T04:32:15.677 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-09T04:32:15.677 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-09T04:32:15.677 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-09T04:32:15.677 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-09T04:32:15.677 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-09T04:32:15.677 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-09T04:32:15.677 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-09T04:32:15.677 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-09T04:32:15.677 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-09T04:32:15.677 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-09T04:32:15.678 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-09T04:32:15.678 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-09T04:32:15.678 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-09T04:32:15.678 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-09T04:32:15.678 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-09T04:32:15.678 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-09T04:32:15.678 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-09T04:32:15.678 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-09T04:32:15.678 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-09T04:32:15.678 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-09T04:32:15.678 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-09T04:32:15.678 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-09T04:32:15.678 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-09T04:32:15.678 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-09T04:32:15.679 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-09T04:32:15.679 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-09T04:32:15.679 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-09T04:32:15.679 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-09T04:32:15.679 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-09T04:32:15.679 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-09T04:32:15.679 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-09T04:32:15.679 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-09T04:32:15.679 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-09T04:32:15.679 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-09T04:32:15.679 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-09T04:32:15.679 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-09T04:32:15.679 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-09T04:32:15.680 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-09T04:32:15.680 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-09T04:32:15.680 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-09T04:32:15.680 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-09T04:32:15.680 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-09T04:32:15.680 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-09T04:32:15.680 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-09T04:32:15.680 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-09T04:32:15.680 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-09T04:32:15.680 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-09T04:32:15.681 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-09T04:32:15.681 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-09T04:32:15.681 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-09T04:32:15.681 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-09T04:32:15.681 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-09T04:32:15.681 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-09T04:32:15.681 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-09T04:32:15.682 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-09T04:32:15.682 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-09T04:32:15.682 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-09T04:32:15.682 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-09T04:32:15.682 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-09T04:32:15.682 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-09T04:32:15.682 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-09T04:32:15.682 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados2-2:19.0.0-3568.g165a15e5.el9.x86_64 149/164 2024-05-09T04:32:15.682 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-09T04:32:15.682 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librbd1-2:19.0.0-3568.g165a15e5.el9.x86_64 151/164 2024-05-09T04:32:15.683 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-09T04:32:15.683 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-09T04:32:15.683 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-09T04:32:15.683 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-09T04:32:15.683 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-09T04:32:15.683 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-09T04:32:15.683 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-09T04:32:15.683 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-09T04:32:15.683 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-09T04:32:15.683 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-09T04:32:15.683 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-09T04:32:15.683 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-09T04:32:17.035 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-test-2:19.0.0-3568.g165a15e5.el9.x86_64 141/164 2024-05-09T04:32:17.088 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : rbd-fuse-2:19.0.0-3568.g165a15e5.el9.x86_64 142/164 2024-05-09T04:32:17.142 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : rbd-nbd-2:19.0.0-3568.g165a15e5.el9.x86_64 143/164 2024-05-09T04:32:17.176 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-09T04:32:17.214 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-09T04:32:17.551 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : cephfs-shell-2:19.0.0-3568.g165a15e5.el9.x86_64 146/164 2024-05-09T04:32:17.607 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-09T04:32:17.636 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libcephfs-devel-2:19.0.0-3568.g165a15e5.el9.x86_ 148/164 2024-05-09T04:32:17.684 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-09T04:32:17.730 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-09T04:32:17.801 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T04:32:17.801 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-09T04:32:17.801 INFO:teuthology.orchestra.run.smithi192.stdout:Upgraded: 2024-05-09T04:32:17.801 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-09T04:32:17.801 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-09T04:32:17.801 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-09T04:32:17.801 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-09T04:32:17.801 INFO:teuthology.orchestra.run.smithi192.stdout: librados2-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:17.801 INFO:teuthology.orchestra.run.smithi192.stdout: librbd1-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:17.802 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-09T04:32:17.802 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-09T04:32:17.802 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-05-09T04:32:17.802 INFO:teuthology.orchestra.run.smithi192.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-09T04:32:17.802 INFO:teuthology.orchestra.run.smithi192.stdout: automake-1.16.2-8.el9.noarch 2024-05-09T04:32:17.802 INFO:teuthology.orchestra.run.smithi192.stdout: bison-3.7.4-5.el9.x86_64 2024-05-09T04:32:17.802 INFO:teuthology.orchestra.run.smithi192.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-09T04:32:17.802 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:17.802 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-base-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:17.802 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-common-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:17.802 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-fuse-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:17.802 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-grafana-dashboards-2:19.0.0-3568.g165a15e5.el9.noarch 2024-05-09T04:32:17.802 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-immutable-object-cache-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:17.803 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mds-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:17.803 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:17.803 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-cephadm-2:19.0.0-3568.g165a15e5.el9.noarch 2024-05-09T04:32:17.803 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-dashboard-2:19.0.0-3568.g165a15e5.el9.noarch 2024-05-09T04:32:17.803 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3568.g165a15e5.el9.noarch 2024-05-09T04:32:17.803 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-modules-core-2:19.0.0-3568.g165a15e5.el9.noarch 2024-05-09T04:32:17.803 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-rook-2:19.0.0-3568.g165a15e5.el9.noarch 2024-05-09T04:32:17.803 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mon-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:17.803 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-osd-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:17.803 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-prometheus-alerts-2:19.0.0-3568.g165a15e5.el9.noarch 2024-05-09T04:32:17.803 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-radosgw-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:17.803 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-selinux-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:17.803 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-test-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:17.803 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-volume-2:19.0.0-3568.g165a15e5.el9.noarch 2024-05-09T04:32:17.804 INFO:teuthology.orchestra.run.smithi192.stdout: cephadm-2:19.0.0-3568.g165a15e5.el9.noarch 2024-05-09T04:32:17.804 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-mirror-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:17.804 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-shell-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:17.804 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-top-2:19.0.0-3568.g165a15e5.el9.noarch 2024-05-09T04:32:17.804 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-09T04:32:17.804 INFO:teuthology.orchestra.run.smithi192.stdout: flex-2.6.4-9.el9.x86_64 2024-05-09T04:32:17.804 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-09T04:32:17.804 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-09T04:32:17.804 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-09T04:32:17.804 INFO:teuthology.orchestra.run.smithi192.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-09T04:32:17.804 INFO:teuthology.orchestra.run.smithi192.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-09T04:32:17.804 INFO:teuthology.orchestra.run.smithi192.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-09T04:32:17.804 INFO:teuthology.orchestra.run.smithi192.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-09T04:32:17.804 INFO:teuthology.orchestra.run.smithi192.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-09T04:32:17.805 INFO:teuthology.orchestra.run.smithi192.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-09T04:32:17.805 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs-devel-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:17.805 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs2-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:17.805 INFO:teuthology.orchestra.run.smithi192.stdout: libcephsqlite-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:17.805 INFO:teuthology.orchestra.run.smithi192.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-09T04:32:17.805 INFO:teuthology.orchestra.run.smithi192.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-09T04:32:17.805 INFO:teuthology.orchestra.run.smithi192.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-09T04:32:17.805 INFO:teuthology.orchestra.run.smithi192.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-09T04:32:17.805 INFO:teuthology.orchestra.run.smithi192.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-09T04:32:17.805 INFO:teuthology.orchestra.run.smithi192.stdout: librados-devel-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:17.805 INFO:teuthology.orchestra.run.smithi192.stdout: libradosstriper1-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:17.805 INFO:teuthology.orchestra.run.smithi192.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-09T04:32:17.805 INFO:teuthology.orchestra.run.smithi192.stdout: librgw2-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:17.805 INFO:teuthology.orchestra.run.smithi192.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-09T04:32:17.806 INFO:teuthology.orchestra.run.smithi192.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-09T04:32:17.806 INFO:teuthology.orchestra.run.smithi192.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-09T04:32:17.806 INFO:teuthology.orchestra.run.smithi192.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-09T04:32:17.806 INFO:teuthology.orchestra.run.smithi192.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-09T04:32:17.806 INFO:teuthology.orchestra.run.smithi192.stdout: lua-5.4.4-4.el9.x86_64 2024-05-09T04:32:17.806 INFO:teuthology.orchestra.run.smithi192.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-09T04:32:17.806 INFO:teuthology.orchestra.run.smithi192.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-09T04:32:17.806 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-09T04:32:17.806 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-09T04:32:17.806 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-09T04:32:17.806 INFO:teuthology.orchestra.run.smithi192.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-09T04:32:17.806 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-13.14-1.el9.x86_64 2024-05-09T04:32:17.806 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-09T04:32:17.806 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-09T04:32:17.807 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-09T04:32:17.807 INFO:teuthology.orchestra.run.smithi192.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-09T04:32:17.807 INFO:teuthology.orchestra.run.smithi192.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-09T04:32:17.807 INFO:teuthology.orchestra.run.smithi192.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-09T04:32:17.807 INFO:teuthology.orchestra.run.smithi192.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-09T04:32:17.807 INFO:teuthology.orchestra.run.smithi192.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-09T04:32:17.807 INFO:teuthology.orchestra.run.smithi192.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-09T04:32:17.807 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-09T04:32:17.807 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-argparse-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:17.807 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-common-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:17.807 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cephfs-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:17.807 INFO:teuthology.orchestra.run.smithi192.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-09T04:32:17.807 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-09T04:32:17.807 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-09T04:32:17.808 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-09T04:32:17.808 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-09T04:32:17.808 INFO:teuthology.orchestra.run.smithi192.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-09T04:32:17.808 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-09T04:32:17.808 INFO:teuthology.orchestra.run.smithi192.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-09T04:32:17.808 INFO:teuthology.orchestra.run.smithi192.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-09T04:32:17.808 INFO:teuthology.orchestra.run.smithi192.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-09T04:32:17.808 INFO:teuthology.orchestra.run.smithi192.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-09T04:32:17.808 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-09T04:32:17.808 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-09T04:32:17.808 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-09T04:32:17.808 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-09T04:32:17.808 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-09T04:32:17.808 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-09T04:32:17.809 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-09T04:32:17.809 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-09T04:32:17.809 INFO:teuthology.orchestra.run.smithi192.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-09T04:32:17.809 INFO:teuthology.orchestra.run.smithi192.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-09T04:32:17.809 INFO:teuthology.orchestra.run.smithi192.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-09T04:32:17.809 INFO:teuthology.orchestra.run.smithi192.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-09T04:32:17.809 INFO:teuthology.orchestra.run.smithi192.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-09T04:32:17.809 INFO:teuthology.orchestra.run.smithi192.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-09T04:32:17.809 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-09T04:32:17.809 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-09T04:32:17.809 INFO:teuthology.orchestra.run.smithi192.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-09T04:32:17.809 INFO:teuthology.orchestra.run.smithi192.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-09T04:32:17.809 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-09T04:32:17.809 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ply-3.11-14.el9.noarch 2024-05-09T04:32:17.810 INFO:teuthology.orchestra.run.smithi192.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-09T04:32:17.810 INFO:teuthology.orchestra.run.smithi192.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-09T04:32:17.810 INFO:teuthology.orchestra.run.smithi192.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-09T04:32:17.810 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-09T04:32:17.810 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-09T04:32:17.810 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-09T04:32:17.810 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-09T04:32:17.810 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-09T04:32:17.810 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-09T04:32:17.810 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rados-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:17.810 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rbd-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:17.810 INFO:teuthology.orchestra.run.smithi192.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-09T04:32:17.810 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-09T04:32:17.810 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-09T04:32:17.810 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rgw-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:17.811 INFO:teuthology.orchestra.run.smithi192.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-09T04:32:17.811 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-09T04:32:17.811 INFO:teuthology.orchestra.run.smithi192.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-09T04:32:17.811 INFO:teuthology.orchestra.run.smithi192.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-09T04:32:17.811 INFO:teuthology.orchestra.run.smithi192.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-09T04:32:17.811 INFO:teuthology.orchestra.run.smithi192.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-09T04:32:17.811 INFO:teuthology.orchestra.run.smithi192.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-09T04:32:17.811 INFO:teuthology.orchestra.run.smithi192.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-09T04:32:17.811 INFO:teuthology.orchestra.run.smithi192.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-09T04:32:17.811 INFO:teuthology.orchestra.run.smithi192.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-09T04:32:17.811 INFO:teuthology.orchestra.run.smithi192.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-09T04:32:17.811 INFO:teuthology.orchestra.run.smithi192.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-09T04:32:17.811 INFO:teuthology.orchestra.run.smithi192.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-09T04:32:17.811 INFO:teuthology.orchestra.run.smithi192.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-09T04:32:17.812 INFO:teuthology.orchestra.run.smithi192.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-09T04:32:17.812 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-fuse-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:17.812 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-mirror-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:17.812 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-nbd-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:17.812 INFO:teuthology.orchestra.run.smithi192.stdout: re2-1:20211101-3.el9.x86_64 2024-05-09T04:32:17.812 INFO:teuthology.orchestra.run.smithi192.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-09T04:32:17.812 INFO:teuthology.orchestra.run.smithi192.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-09T04:32:17.812 INFO:teuthology.orchestra.run.smithi192.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-09T04:32:17.812 INFO:teuthology.orchestra.run.smithi192.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-09T04:32:17.812 INFO:teuthology.orchestra.run.smithi192.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-09T04:32:17.813 INFO:teuthology.orchestra.run.smithi192.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-09T04:32:17.813 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-09T04:32:17.813 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-05-09T04:32:18.394 DEBUG:teuthology.parallel:result is None 2024-05-09T04:32:18.652 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T04:32:18.709 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T04:32:18.765 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T04:32:20.107 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : cephfs-top-2:19.0.0-3568.g165a15e5.el9.noarch 152/164 2024-05-09T04:32:20.208 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-09T04:32:20.281 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-09T04:32:20.457 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-09T04:32:20.493 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-09T04:32:20.531 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-09T04:32:20.531 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T04:32:20.658 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T04:32:20.714 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-09T04:32:20.714 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T04:32:20.851 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T04:32:20.889 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-09T04:32:20.923 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T04:32:20.924 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T04:32:21.244 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T04:32:21.244 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T04:32:21.382 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T04:32:21.511 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T04:32:21.763 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T04:32:21.763 INFO:teuthology.orchestra.run.smithi145.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-05-09T04:32:21.763 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:32:27.503 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-2:19.0.0-3568.g165a15e5.el9.x86_64 1/164 2024-05-09T04:32:27.504 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-base-2:19.0.0-3568.g165a15e5.el9.x86_64 2/164 2024-05-09T04:32:27.504 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-common-2:19.0.0-3568.g165a15e5.el9.x86_64 3/164 2024-05-09T04:32:27.504 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-fuse-2:19.0.0-3568.g165a15e5.el9.x86_64 4/164 2024-05-09T04:32:27.504 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3568.g165a1 5/164 2024-05-09T04:32:27.504 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mds-2:19.0.0-3568.g165a15e5.el9.x86_64 6/164 2024-05-09T04:32:27.504 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-2:19.0.0-3568.g165a15e5.el9.x86_64 7/164 2024-05-09T04:32:27.504 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mon-2:19.0.0-3568.g165a15e5.el9.x86_64 8/164 2024-05-09T04:32:27.504 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-osd-2:19.0.0-3568.g165a15e5.el9.x86_64 9/164 2024-05-09T04:32:27.504 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-radosgw-2:19.0.0-3568.g165a15e5.el9.x86_64 10/164 2024-05-09T04:32:27.504 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-selinux-2:19.0.0-3568.g165a15e5.el9.x86_64 11/164 2024-05-09T04:32:27.505 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-test-2:19.0.0-3568.g165a15e5.el9.x86_64 12/164 2024-05-09T04:32:27.505 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : cephfs-mirror-2:19.0.0-3568.g165a15e5.el9.x86_64 13/164 2024-05-09T04:32:27.505 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : cephfs-shell-2:19.0.0-3568.g165a15e5.el9.x86_64 14/164 2024-05-09T04:32:27.505 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libcephfs-devel-2:19.0.0-3568.g165a15e5.el9.x86_ 15/164 2024-05-09T04:32:27.505 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libcephfs2-2:19.0.0-3568.g165a15e5.el9.x86_64 16/164 2024-05-09T04:32:27.505 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libcephsqlite-2:19.0.0-3568.g165a15e5.el9.x86_64 17/164 2024-05-09T04:32:27.505 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librados-devel-2:19.0.0-3568.g165a15e5.el9.x86_6 18/164 2024-05-09T04:32:27.505 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libradosstriper1-2:19.0.0-3568.g165a15e5.el9.x86 19/164 2024-05-09T04:32:27.505 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librgw2-2:19.0.0-3568.g165a15e5.el9.x86_64 20/164 2024-05-09T04:32:27.505 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3568.g165a15e5.el 21/164 2024-05-09T04:32:27.505 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-ceph-common-2:19.0.0-3568.g165a15e5.el9. 22/164 2024-05-09T04:32:27.505 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cephfs-2:19.0.0-3568.g165a15e5.el9.x86_6 23/164 2024-05-09T04:32:27.505 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-rados-2:19.0.0-3568.g165a15e5.el9.x86_64 24/164 2024-05-09T04:32:27.505 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-rbd-2:19.0.0-3568.g165a15e5.el9.x86_64 25/164 2024-05-09T04:32:27.505 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-rgw-2:19.0.0-3568.g165a15e5.el9.x86_64 26/164 2024-05-09T04:32:27.506 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : rbd-fuse-2:19.0.0-3568.g165a15e5.el9.x86_64 27/164 2024-05-09T04:32:27.517 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : rbd-mirror-2:19.0.0-3568.g165a15e5.el9.x86_64 28/164 2024-05-09T04:32:27.518 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : rbd-nbd-2:19.0.0-3568.g165a15e5.el9.x86_64 29/164 2024-05-09T04:32:27.518 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3568.g165a15e5. 30/164 2024-05-09T04:32:27.518 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3568.g165a15e5.el9.noa 31/164 2024-05-09T04:32:27.518 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3568.g165a15e5.el9.n 32/164 2024-05-09T04:32:27.518 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3568.g165 33/164 2024-05-09T04:32:27.518 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3568.g165a15e5.el 34/164 2024-05-09T04:32:27.518 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3568.g165a15e5.el9.noarch 35/164 2024-05-09T04:32:27.518 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3568.g165a15e5.e 36/164 2024-05-09T04:32:27.518 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-volume-2:19.0.0-3568.g165a15e5.el9.noarch 37/164 2024-05-09T04:32:27.518 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : cephadm-2:19.0.0-3568.g165a15e5.el9.noarch 38/164 2024-05-09T04:32:27.518 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : cephfs-top-2:19.0.0-3568.g165a15e5.el9.noarch 39/164 2024-05-09T04:32:27.518 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-09T04:32:27.518 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-09T04:32:27.519 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-09T04:32:27.519 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-09T04:32:27.519 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-09T04:32:27.519 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-09T04:32:27.519 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-09T04:32:27.519 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-09T04:32:27.519 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-09T04:32:27.519 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-09T04:32:27.519 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-09T04:32:27.519 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-09T04:32:27.519 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-09T04:32:27.519 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-09T04:32:27.519 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-09T04:32:27.519 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-09T04:32:27.520 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-09T04:32:27.520 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-09T04:32:27.520 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-09T04:32:27.520 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-09T04:32:27.520 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-09T04:32:27.520 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-09T04:32:27.520 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-09T04:32:27.520 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-09T04:32:27.520 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-09T04:32:27.520 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-09T04:32:27.520 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-09T04:32:27.520 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-09T04:32:27.520 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-09T04:32:27.521 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-09T04:32:27.521 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-09T04:32:27.521 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-09T04:32:27.521 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-09T04:32:27.521 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-09T04:32:27.521 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-09T04:32:27.521 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-09T04:32:27.521 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-09T04:32:27.521 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-09T04:32:27.521 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-09T04:32:27.521 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-09T04:32:27.521 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-09T04:32:27.521 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-09T04:32:27.521 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-09T04:32:27.522 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-09T04:32:27.522 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-09T04:32:27.522 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-09T04:32:27.522 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-09T04:32:27.522 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-09T04:32:27.522 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-09T04:32:27.522 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-09T04:32:27.522 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-09T04:32:27.522 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-09T04:32:27.522 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-09T04:32:27.522 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-09T04:32:27.522 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-09T04:32:27.526 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-09T04:32:27.526 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-09T04:32:27.526 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-09T04:32:27.526 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-09T04:32:27.526 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-09T04:32:27.526 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-09T04:32:27.526 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-09T04:32:27.526 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-09T04:32:27.526 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-09T04:32:27.526 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-09T04:32:27.526 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-09T04:32:27.526 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-09T04:32:27.526 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-09T04:32:27.527 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-09T04:32:27.527 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-09T04:32:27.527 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-09T04:32:27.527 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-09T04:32:27.527 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-09T04:32:27.527 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-09T04:32:27.527 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-09T04:32:27.527 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-09T04:32:27.527 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-09T04:32:27.527 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-09T04:32:27.527 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-09T04:32:27.527 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-09T04:32:27.527 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-09T04:32:27.527 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-09T04:32:27.528 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-09T04:32:27.528 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-09T04:32:27.528 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-09T04:32:27.528 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-09T04:32:27.528 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-09T04:32:27.528 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-09T04:32:27.528 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-09T04:32:27.528 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-09T04:32:27.528 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-09T04:32:27.528 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-09T04:32:27.528 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-09T04:32:27.528 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-09T04:32:27.528 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-09T04:32:27.528 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-09T04:32:27.529 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-09T04:32:27.529 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-09T04:32:27.529 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-09T04:32:27.529 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-09T04:32:27.529 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-09T04:32:27.529 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-09T04:32:27.529 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-09T04:32:27.529 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-09T04:32:27.529 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-09T04:32:27.529 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-09T04:32:27.529 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-09T04:32:27.529 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-09T04:32:27.529 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-09T04:32:27.529 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librados2-2:19.0.0-3568.g165a15e5.el9.x86_64 149/164 2024-05-09T04:32:27.530 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-09T04:32:27.530 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librbd1-2:19.0.0-3568.g165a15e5.el9.x86_64 151/164 2024-05-09T04:32:27.530 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-09T04:32:27.530 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-09T04:32:27.530 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-09T04:32:27.530 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-09T04:32:27.530 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-09T04:32:27.530 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-09T04:32:27.530 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-09T04:32:27.530 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-09T04:32:27.530 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-09T04:32:27.530 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-09T04:32:27.530 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-09T04:32:27.530 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-09T04:32:29.733 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T04:32:29.733 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:32:29.733 INFO:teuthology.orchestra.run.smithi145.stdout:Upgraded: 2024-05-09T04:32:29.733 INFO:teuthology.orchestra.run.smithi145.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-09T04:32:29.733 INFO:teuthology.orchestra.run.smithi145.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-09T04:32:29.733 INFO:teuthology.orchestra.run.smithi145.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-09T04:32:29.733 INFO:teuthology.orchestra.run.smithi145.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-09T04:32:29.733 INFO:teuthology.orchestra.run.smithi145.stdout: librados2-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:29.733 INFO:teuthology.orchestra.run.smithi145.stdout: librbd1-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:29.734 INFO:teuthology.orchestra.run.smithi145.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-09T04:32:29.734 INFO:teuthology.orchestra.run.smithi145.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-09T04:32:29.734 INFO:teuthology.orchestra.run.smithi145.stdout:Installed: 2024-05-09T04:32:29.734 INFO:teuthology.orchestra.run.smithi145.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-09T04:32:29.734 INFO:teuthology.orchestra.run.smithi145.stdout: automake-1.16.2-8.el9.noarch 2024-05-09T04:32:29.734 INFO:teuthology.orchestra.run.smithi145.stdout: bison-3.7.4-5.el9.x86_64 2024-05-09T04:32:29.734 INFO:teuthology.orchestra.run.smithi145.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-09T04:32:29.734 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:29.734 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-base-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:29.734 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-common-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:29.734 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-fuse-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:29.734 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-grafana-dashboards-2:19.0.0-3568.g165a15e5.el9.noarch 2024-05-09T04:32:29.735 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-immutable-object-cache-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:29.735 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mds-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:29.735 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:29.735 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-cephadm-2:19.0.0-3568.g165a15e5.el9.noarch 2024-05-09T04:32:29.735 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-dashboard-2:19.0.0-3568.g165a15e5.el9.noarch 2024-05-09T04:32:29.735 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3568.g165a15e5.el9.noarch 2024-05-09T04:32:29.735 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-modules-core-2:19.0.0-3568.g165a15e5.el9.noarch 2024-05-09T04:32:29.735 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-rook-2:19.0.0-3568.g165a15e5.el9.noarch 2024-05-09T04:32:29.735 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mon-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:29.735 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-osd-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:29.735 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-prometheus-alerts-2:19.0.0-3568.g165a15e5.el9.noarch 2024-05-09T04:32:29.735 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-radosgw-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:29.736 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-selinux-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:29.736 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-test-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:29.736 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-volume-2:19.0.0-3568.g165a15e5.el9.noarch 2024-05-09T04:32:29.736 INFO:teuthology.orchestra.run.smithi145.stdout: cephadm-2:19.0.0-3568.g165a15e5.el9.noarch 2024-05-09T04:32:29.736 INFO:teuthology.orchestra.run.smithi145.stdout: cephfs-mirror-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:29.736 INFO:teuthology.orchestra.run.smithi145.stdout: cephfs-shell-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:29.736 INFO:teuthology.orchestra.run.smithi145.stdout: cephfs-top-2:19.0.0-3568.g165a15e5.el9.noarch 2024-05-09T04:32:29.736 INFO:teuthology.orchestra.run.smithi145.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-09T04:32:29.736 INFO:teuthology.orchestra.run.smithi145.stdout: flex-2.6.4-9.el9.x86_64 2024-05-09T04:32:29.736 INFO:teuthology.orchestra.run.smithi145.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-09T04:32:29.736 INFO:teuthology.orchestra.run.smithi145.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-09T04:32:29.736 INFO:teuthology.orchestra.run.smithi145.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-09T04:32:29.736 INFO:teuthology.orchestra.run.smithi145.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-09T04:32:29.737 INFO:teuthology.orchestra.run.smithi145.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-09T04:32:29.737 INFO:teuthology.orchestra.run.smithi145.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-09T04:32:29.737 INFO:teuthology.orchestra.run.smithi145.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-09T04:32:29.737 INFO:teuthology.orchestra.run.smithi145.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-09T04:32:29.737 INFO:teuthology.orchestra.run.smithi145.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-09T04:32:29.737 INFO:teuthology.orchestra.run.smithi145.stdout: libcephfs-devel-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:29.737 INFO:teuthology.orchestra.run.smithi145.stdout: libcephfs2-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:29.737 INFO:teuthology.orchestra.run.smithi145.stdout: libcephsqlite-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:29.737 INFO:teuthology.orchestra.run.smithi145.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-09T04:32:29.737 INFO:teuthology.orchestra.run.smithi145.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-09T04:32:29.737 INFO:teuthology.orchestra.run.smithi145.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-09T04:32:29.737 INFO:teuthology.orchestra.run.smithi145.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-09T04:32:29.737 INFO:teuthology.orchestra.run.smithi145.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-09T04:32:29.737 INFO:teuthology.orchestra.run.smithi145.stdout: librados-devel-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:29.738 INFO:teuthology.orchestra.run.smithi145.stdout: libradosstriper1-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:29.738 INFO:teuthology.orchestra.run.smithi145.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-09T04:32:29.738 INFO:teuthology.orchestra.run.smithi145.stdout: librgw2-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:29.738 INFO:teuthology.orchestra.run.smithi145.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-09T04:32:29.738 INFO:teuthology.orchestra.run.smithi145.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-09T04:32:29.738 INFO:teuthology.orchestra.run.smithi145.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-09T04:32:29.738 INFO:teuthology.orchestra.run.smithi145.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-09T04:32:29.738 INFO:teuthology.orchestra.run.smithi145.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-09T04:32:29.738 INFO:teuthology.orchestra.run.smithi145.stdout: lua-5.4.4-4.el9.x86_64 2024-05-09T04:32:29.738 INFO:teuthology.orchestra.run.smithi145.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-09T04:32:29.738 INFO:teuthology.orchestra.run.smithi145.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-09T04:32:29.738 INFO:teuthology.orchestra.run.smithi145.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-09T04:32:29.740 INFO:teuthology.orchestra.run.smithi145.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-09T04:32:29.740 INFO:teuthology.orchestra.run.smithi145.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-09T04:32:29.740 INFO:teuthology.orchestra.run.smithi145.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-09T04:32:29.740 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-13.14-1.el9.x86_64 2024-05-09T04:32:29.740 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-09T04:32:29.740 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-09T04:32:29.740 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-09T04:32:29.740 INFO:teuthology.orchestra.run.smithi145.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-09T04:32:29.740 INFO:teuthology.orchestra.run.smithi145.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-09T04:32:29.740 INFO:teuthology.orchestra.run.smithi145.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-09T04:32:29.741 INFO:teuthology.orchestra.run.smithi145.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-09T04:32:29.741 INFO:teuthology.orchestra.run.smithi145.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-09T04:32:29.741 INFO:teuthology.orchestra.run.smithi145.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-09T04:32:29.741 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-09T04:32:29.741 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ceph-argparse-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:29.741 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ceph-common-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:29.741 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cephfs-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:29.741 INFO:teuthology.orchestra.run.smithi145.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-09T04:32:29.741 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-09T04:32:29.741 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-09T04:32:29.741 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-09T04:32:29.741 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-09T04:32:29.741 INFO:teuthology.orchestra.run.smithi145.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-09T04:32:29.741 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-09T04:32:29.741 INFO:teuthology.orchestra.run.smithi145.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-09T04:32:29.742 INFO:teuthology.orchestra.run.smithi145.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-09T04:32:29.742 INFO:teuthology.orchestra.run.smithi145.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-09T04:32:29.742 INFO:teuthology.orchestra.run.smithi145.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-09T04:32:29.742 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-09T04:32:29.742 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-09T04:32:29.742 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-09T04:32:29.742 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-09T04:32:29.742 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-09T04:32:29.742 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-09T04:32:29.742 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-09T04:32:29.742 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-09T04:32:29.742 INFO:teuthology.orchestra.run.smithi145.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-09T04:32:29.742 INFO:teuthology.orchestra.run.smithi145.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-09T04:32:29.743 INFO:teuthology.orchestra.run.smithi145.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-09T04:32:29.743 INFO:teuthology.orchestra.run.smithi145.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-09T04:32:29.743 INFO:teuthology.orchestra.run.smithi145.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-09T04:32:29.743 INFO:teuthology.orchestra.run.smithi145.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-09T04:32:29.743 INFO:teuthology.orchestra.run.smithi145.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-09T04:32:29.743 INFO:teuthology.orchestra.run.smithi145.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-09T04:32:29.743 INFO:teuthology.orchestra.run.smithi145.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-09T04:32:29.743 INFO:teuthology.orchestra.run.smithi145.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-09T04:32:29.743 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-09T04:32:29.743 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ply-3.11-14.el9.noarch 2024-05-09T04:32:29.743 INFO:teuthology.orchestra.run.smithi145.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-09T04:32:29.743 INFO:teuthology.orchestra.run.smithi145.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-09T04:32:29.743 INFO:teuthology.orchestra.run.smithi145.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-09T04:32:29.743 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-09T04:32:29.744 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-09T04:32:29.744 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-09T04:32:29.744 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-09T04:32:29.744 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-09T04:32:29.744 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-09T04:32:29.744 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rados-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:29.744 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rbd-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:29.744 INFO:teuthology.orchestra.run.smithi145.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-09T04:32:29.744 INFO:teuthology.orchestra.run.smithi145.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-09T04:32:29.744 INFO:teuthology.orchestra.run.smithi145.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-09T04:32:29.744 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rgw-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:29.744 INFO:teuthology.orchestra.run.smithi145.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-09T04:32:29.744 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-09T04:32:29.744 INFO:teuthology.orchestra.run.smithi145.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-09T04:32:29.744 INFO:teuthology.orchestra.run.smithi145.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-09T04:32:29.745 INFO:teuthology.orchestra.run.smithi145.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-09T04:32:29.745 INFO:teuthology.orchestra.run.smithi145.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-09T04:32:29.745 INFO:teuthology.orchestra.run.smithi145.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-09T04:32:29.745 INFO:teuthology.orchestra.run.smithi145.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-09T04:32:29.745 INFO:teuthology.orchestra.run.smithi145.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-09T04:32:29.745 INFO:teuthology.orchestra.run.smithi145.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-09T04:32:29.745 INFO:teuthology.orchestra.run.smithi145.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-09T04:32:29.745 INFO:teuthology.orchestra.run.smithi145.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-09T04:32:29.745 INFO:teuthology.orchestra.run.smithi145.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-09T04:32:29.745 INFO:teuthology.orchestra.run.smithi145.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-09T04:32:29.745 INFO:teuthology.orchestra.run.smithi145.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-09T04:32:29.745 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-fuse-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:29.745 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-mirror-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:29.745 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-nbd-2:19.0.0-3568.g165a15e5.el9.x86_64 2024-05-09T04:32:29.745 INFO:teuthology.orchestra.run.smithi145.stdout: re2-1:20211101-3.el9.x86_64 2024-05-09T04:32:29.746 INFO:teuthology.orchestra.run.smithi145.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-09T04:32:29.746 INFO:teuthology.orchestra.run.smithi145.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-09T04:32:29.746 INFO:teuthology.orchestra.run.smithi145.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-09T04:32:29.746 INFO:teuthology.orchestra.run.smithi145.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-09T04:32:29.746 INFO:teuthology.orchestra.run.smithi145.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-09T04:32:29.746 INFO:teuthology.orchestra.run.smithi145.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-09T04:32:29.746 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:32:29.746 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2024-05-09T04:32:30.345 DEBUG:teuthology.parallel:result is None 2024-05-09T04:32:30.346 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=165a15e51d08073b5317c443415b0d8a7141b1ab 2024-05-09T04:32:30.508 DEBUG:teuthology.orchestra.run.smithi145:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-09T04:32:30.562 INFO:teuthology.orchestra.run.smithi145.stdout:19.0.0-3568.g165a15e5.el9 2024-05-09T04:32:30.563 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3568.g165a15e5.el9 2024-05-09T04:32:30.563 INFO:teuthology.task.install:The correct ceph version 19.0.0-3568.g165a15e5 is installed. 2024-05-09T04:32:30.565 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=165a15e51d08073b5317c443415b0d8a7141b1ab 2024-05-09T04:32:30.919 DEBUG:teuthology.orchestra.run.smithi192:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-09T04:32:30.992 INFO:teuthology.orchestra.run.smithi192.stdout:19.0.0-3568.g165a15e5.el9 2024-05-09T04:32:30.992 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3568.g165a15e5.el9 2024-05-09T04:32:30.993 INFO:teuthology.task.install:The correct ceph version 19.0.0-3568.g165a15e5 is installed. 2024-05-09T04:32:30.995 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-09T04:32:30.995 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-05-09T04:32:30.995 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-09T04:32:31.072 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-09T04:32:31.072 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-09T04:32:31.166 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-09T04:32:31.166 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-05-09T04:32:31.166 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/usr/bin/daemon-helper 2024-05-09T04:32:31.240 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-09T04:32:31.349 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-09T04:32:31.350 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/daemon-helper 2024-05-09T04:32:31.439 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-09T04:32:31.518 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-09T04:32:31.518 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-05-09T04:32:31.519 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-09T04:32:31.586 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-09T04:32:31.692 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-09T04:32:31.692 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-09T04:32:31.782 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-09T04:32:31.907 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-09T04:32:31.907 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-05-09T04:32:31.907 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/usr/bin/stdin-killer 2024-05-09T04:32:31.978 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-09T04:32:32.079 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-09T04:32:32.079 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/stdin-killer 2024-05-09T04:32:32.163 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-09T04:32:32.247 INFO:teuthology.run_tasks:Running task ceph... 2024-05-09T04:32:32.342 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-05-09T04:32:32.342 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod 777 /var/log/ceph 2024-05-09T04:32:32.347 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 777 /var/log/ceph 2024-05-09T04:32:32.438 INFO:tasks.ceph:Disabling ceph logrotate... 2024-05-09T04:32:32.438 DEBUG:teuthology.orchestra.run.smithi145:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-09T04:32:32.442 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-09T04:32:32.529 INFO:tasks.ceph:Creating extra log directories... 2024-05-09T04:32:32.529 DEBUG:teuthology.orchestra.run.smithi145:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-09T04:32:32.534 DEBUG:teuthology.orchestra.run.smithi192:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-09T04:32:32.626 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-05-09T04:32:32.626 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-05-09T04:32:32.627 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-05-09T04:32:32.627 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-05-09T04:32:32.627 DEBUG:teuthology.orchestra.run.smithi145:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-09T04:32:32.670 DEBUG:teuthology.orchestra.run.smithi145:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-09T04:32:32.853 DEBUG:teuthology.orchestra.run.smithi145:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-05-09T04:32:32.914 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-09T04:32:32.914 DEBUG:teuthology.orchestra.run.smithi192:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-09T04:32:32.972 DEBUG:teuthology.orchestra.run.smithi192:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-09T04:32:33.207 DEBUG:teuthology.orchestra.run.smithi192:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-05-09T04:32:33.301 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-05-09T04:32:33.301 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-05-09T04:32:33.301 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/vshankar-2024-05-09_04:13:07-fs-wip-63259-3-testing-default-smithi/7699554', 'branch': 'wip-63259-3', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/backtrace}', 'email': 'vshankar@redhat.com', 'first_in_suite': False, 'job_id': '7699554', 'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'vshankar-2024-05-09_04:13:07-fs-wip-63259-3-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-63259-3'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': '165a15e51d08073b5317c443415b0d8a7141b1ab'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '165a15e51d08073b5317c443415b0d8a7141b1ab'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-63259-3', 'sha1': '165a15e51d08073b5317c443415b0d8a7141b1ab'}}, 'owner': 'scheduled_vshankar@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 3813, 'sha1': '165a15e51d08073b5317c443415b0d8a7141b1ab', 'sleep_before_teardown': 0, 'subset': '31/1024', 'suite': 'fs', 'suite_branch': 'wip-63259-3', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_165a15e51d08073b5317c443415b0d8a7141b1ab/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '165a15e51d08073b5317c443415b0d8a7141b1ab', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi145.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLbQwlhmHtv3FPtQ4lWSaMnaJDWcYWkvMgaLuyzLEAV9V+L8YHnFXnK/VG8fxHcr0xO4Ji0IGJpz/N3gwrufNIU=', 'smithi192.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKFyOmXPPT42NFJaLMrEgmPLkAxUtASgsUI6YWYHHTb3wl+Jzf5CH15lRCG5CrSf9NjCEFwjvdapjhI71FidzDU='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '165a15e51d08073b5317c443415b0d8a7141b1ab'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': '165a15e51d08073b5317c443415b0d8a7141b1ab', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_backtrace']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '1ae7ad82388e92a475afff437d49054826c019a1', 'timestamp': '2024-05-09_04:13:07', 'tube': 'smithi', 'user': 'vshankar', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-05-09T04:32:33.302 DEBUG:teuthology.orchestra.run.smithi145:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-09T04:32:33.307 DEBUG:teuthology.orchestra.run.smithi192:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-09T04:32:33.364 DEBUG:teuthology.orchestra.run.smithi145:> sudo install -d -m0777 -- /var/run/ceph 2024-05-09T04:32:33.389 DEBUG:teuthology.orchestra.run.smithi192:> sudo install -d -m0777 -- /var/run/ceph 2024-05-09T04:32:33.476 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-05-09T04:32:33.476 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/scratch_devs of=/dev/stdout 2024-05-09T04:32:33.516 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-09T04:32:33.516 DEBUG:teuthology.orchestra.run.smithi145:> stat /dev/vg_nvme/lv_1 2024-05-09T04:32:33.596 INFO:teuthology.orchestra.run.smithi145.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-09T04:32:33.596 INFO:teuthology.orchestra.run.smithi145.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T04:32:33.596 INFO:teuthology.orchestra.run.smithi145.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-05-09T04:32:33.596 INFO:teuthology.orchestra.run.smithi145.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T04:32:33.596 INFO:teuthology.orchestra.run.smithi145.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T04:32:33.596 INFO:teuthology.orchestra.run.smithi145.stdout:Access: 2024-05-09 04:32:22.214379528 +0000 2024-05-09T04:32:33.596 INFO:teuthology.orchestra.run.smithi145.stdout:Modify: 2024-05-09 04:29:43.673419837 +0000 2024-05-09T04:32:33.596 INFO:teuthology.orchestra.run.smithi145.stdout:Change: 2024-05-09 04:29:43.673419837 +0000 2024-05-09T04:32:33.596 INFO:teuthology.orchestra.run.smithi145.stdout: Birth: 2024-05-09 04:29:43.673419837 +0000 2024-05-09T04:32:33.596 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-09T04:32:33.698 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records in 2024-05-09T04:32:33.698 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records out 2024-05-09T04:32:33.698 INFO:teuthology.orchestra.run.smithi145.stderr:512 bytes copied, 0.000971857 s, 527 kB/s 2024-05-09T04:32:33.702 DEBUG:teuthology.orchestra.run.smithi145:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-09T04:32:33.743 DEBUG:teuthology.orchestra.run.smithi145:> stat /dev/vg_nvme/lv_2 2024-05-09T04:32:33.825 INFO:teuthology.orchestra.run.smithi145.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-09T04:32:33.825 INFO:teuthology.orchestra.run.smithi145.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T04:32:33.826 INFO:teuthology.orchestra.run.smithi145.stdout:Device: 6h/6d Inode: 652 Links: 1 2024-05-09T04:32:33.826 INFO:teuthology.orchestra.run.smithi145.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T04:32:33.826 INFO:teuthology.orchestra.run.smithi145.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T04:32:33.826 INFO:teuthology.orchestra.run.smithi145.stdout:Access: 2024-05-09 04:32:22.215379506 +0000 2024-05-09T04:32:33.826 INFO:teuthology.orchestra.run.smithi145.stdout:Modify: 2024-05-09 04:29:44.273407007 +0000 2024-05-09T04:32:33.826 INFO:teuthology.orchestra.run.smithi145.stdout:Change: 2024-05-09 04:29:44.273407007 +0000 2024-05-09T04:32:33.826 INFO:teuthology.orchestra.run.smithi145.stdout: Birth: 2024-05-09 04:29:44.273407007 +0000 2024-05-09T04:32:33.826 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-09T04:32:33.925 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records in 2024-05-09T04:32:33.925 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records out 2024-05-09T04:32:33.925 INFO:teuthology.orchestra.run.smithi145.stderr:512 bytes copied, 0.000316156 s, 1.6 MB/s 2024-05-09T04:32:33.928 DEBUG:teuthology.orchestra.run.smithi145:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-09T04:32:33.970 DEBUG:teuthology.orchestra.run.smithi145:> stat /dev/vg_nvme/lv_3 2024-05-09T04:32:34.056 INFO:teuthology.orchestra.run.smithi145.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-09T04:32:34.056 INFO:teuthology.orchestra.run.smithi145.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T04:32:34.056 INFO:teuthology.orchestra.run.smithi145.stdout:Device: 6h/6d Inode: 670 Links: 1 2024-05-09T04:32:34.056 INFO:teuthology.orchestra.run.smithi145.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T04:32:34.056 INFO:teuthology.orchestra.run.smithi145.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T04:32:34.056 INFO:teuthology.orchestra.run.smithi145.stdout:Access: 2024-05-09 04:32:22.215379506 +0000 2024-05-09T04:32:34.056 INFO:teuthology.orchestra.run.smithi145.stdout:Modify: 2024-05-09 04:29:44.867394305 +0000 2024-05-09T04:32:34.057 INFO:teuthology.orchestra.run.smithi145.stdout:Change: 2024-05-09 04:29:44.867394305 +0000 2024-05-09T04:32:34.057 INFO:teuthology.orchestra.run.smithi145.stdout: Birth: 2024-05-09 04:29:44.867394305 +0000 2024-05-09T04:32:34.057 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-09T04:32:34.162 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records in 2024-05-09T04:32:34.162 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records out 2024-05-09T04:32:34.162 INFO:teuthology.orchestra.run.smithi145.stderr:512 bytes copied, 0.000304751 s, 1.7 MB/s 2024-05-09T04:32:34.165 DEBUG:teuthology.orchestra.run.smithi145:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-09T04:32:34.205 DEBUG:teuthology.orchestra.run.smithi145:> stat /dev/vg_nvme/lv_4 2024-05-09T04:32:34.283 INFO:teuthology.orchestra.run.smithi145.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-09T04:32:34.283 INFO:teuthology.orchestra.run.smithi145.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T04:32:34.283 INFO:teuthology.orchestra.run.smithi145.stdout:Device: 6h/6d Inode: 690 Links: 1 2024-05-09T04:32:34.283 INFO:teuthology.orchestra.run.smithi145.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T04:32:34.283 INFO:teuthology.orchestra.run.smithi145.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T04:32:34.283 INFO:teuthology.orchestra.run.smithi145.stdout:Access: 2024-05-09 04:32:22.216379485 +0000 2024-05-09T04:32:34.283 INFO:teuthology.orchestra.run.smithi145.stdout:Modify: 2024-05-09 04:29:45.426382352 +0000 2024-05-09T04:32:34.283 INFO:teuthology.orchestra.run.smithi145.stdout:Change: 2024-05-09 04:29:45.426382352 +0000 2024-05-09T04:32:34.283 INFO:teuthology.orchestra.run.smithi145.stdout: Birth: 2024-05-09 04:29:45.426382352 +0000 2024-05-09T04:32:34.283 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-09T04:32:34.391 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records in 2024-05-09T04:32:34.391 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records out 2024-05-09T04:32:34.391 INFO:teuthology.orchestra.run.smithi145.stderr:512 bytes copied, 0.000365098 s, 1.4 MB/s 2024-05-09T04:32:34.394 DEBUG:teuthology.orchestra.run.smithi145:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-09T04:32:34.446 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-09T04:32:34.446 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-09T04:32:34.446 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/scratch_devs of=/dev/stdout 2024-05-09T04:32:34.502 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-09T04:32:34.503 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_1 2024-05-09T04:32:34.555 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-09T04:32:34.555 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T04:32:34.555 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 6h/6d Inode: 633 Links: 1 2024-05-09T04:32:34.555 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T04:32:34.555 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T04:32:34.555 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-05-09 04:32:10.341232828 +0000 2024-05-09T04:32:34.555 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-05-09 04:29:35.455211920 +0000 2024-05-09T04:32:34.556 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-05-09 04:29:35.455211920 +0000 2024-05-09T04:32:34.556 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2024-05-09 04:29:35.455211920 +0000 2024-05-09T04:32:34.556 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-09T04:32:34.636 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-05-09T04:32:34.637 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-05-09T04:32:34.637 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000375146 s, 1.4 MB/s 2024-05-09T04:32:34.640 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-09T04:32:34.694 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_2 2024-05-09T04:32:34.745 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-09T04:32:34.745 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T04:32:34.745 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 6h/6d Inode: 653 Links: 1 2024-05-09T04:32:34.745 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T04:32:34.745 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T04:32:34.745 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-05-09 04:32:10.342232809 +0000 2024-05-09T04:32:34.745 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-05-09 04:29:36.078199990 +0000 2024-05-09T04:32:34.745 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-05-09 04:29:36.078199990 +0000 2024-05-09T04:32:34.745 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2024-05-09 04:29:36.078199990 +0000 2024-05-09T04:32:34.746 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-09T04:32:34.829 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-05-09T04:32:34.829 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-05-09T04:32:34.829 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000501083 s, 1.0 MB/s 2024-05-09T04:32:34.833 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-09T04:32:34.889 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_3 2024-05-09T04:32:34.938 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-09T04:32:34.938 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T04:32:34.938 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 6h/6d Inode: 670 Links: 1 2024-05-09T04:32:34.939 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T04:32:34.939 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T04:32:34.939 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-05-09 04:32:10.342232809 +0000 2024-05-09T04:32:34.939 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-05-09 04:29:36.650189035 +0000 2024-05-09T04:32:34.939 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-05-09 04:29:36.650189035 +0000 2024-05-09T04:32:34.939 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2024-05-09 04:29:36.650189035 +0000 2024-05-09T04:32:34.939 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-09T04:32:35.004 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-05-09T04:32:35.005 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-05-09T04:32:35.005 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000363876 s, 1.4 MB/s 2024-05-09T04:32:35.008 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-09T04:32:35.060 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_4 2024-05-09T04:32:35.111 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-09T04:32:35.111 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T04:32:35.111 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 6h/6d Inode: 689 Links: 1 2024-05-09T04:32:35.111 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T04:32:35.111 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T04:32:35.111 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-05-09 04:32:10.343232790 +0000 2024-05-09T04:32:35.111 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-05-09 04:29:37.263177296 +0000 2024-05-09T04:32:35.111 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-05-09 04:29:37.263177296 +0000 2024-05-09T04:32:35.111 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2024-05-09 04:29:37.263177296 +0000 2024-05-09T04:32:35.111 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-09T04:32:35.189 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-05-09T04:32:35.190 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-05-09T04:32:35.190 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000316318 s, 1.6 MB/s 2024-05-09T04:32:35.193 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-09T04:32:35.247 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-09T04:32:35.248 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi145.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi192.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-09T04:32:35.248 INFO:tasks.ceph:Generating config... 2024-05-09T04:32:35.250 INFO:tasks.ceph:[client] client mount timeout = 600 2024-05-09T04:32:35.250 INFO:tasks.ceph:[client] debug client = 20 2024-05-09T04:32:35.250 INFO:tasks.ceph:[client] debug ms = 1 2024-05-09T04:32:35.250 INFO:tasks.ceph:[client] pid file = 2024-05-09T04:32:35.251 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-05-09T04:32:35.251 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-05-09T04:32:35.251 INFO:tasks.ceph:[mds] debug mds = 20 2024-05-09T04:32:35.251 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-05-09T04:32:35.251 INFO:tasks.ceph:[mds] debug ms = 1 2024-05-09T04:32:35.251 INFO:tasks.ceph:[mds] mds debug frag = True 2024-05-09T04:32:35.251 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-05-09T04:32:35.252 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-05-09T04:32:35.252 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-05-09T04:32:35.252 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-05-09T04:32:35.252 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-05-09T04:32:35.252 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-05-09T04:32:35.252 INFO:tasks.ceph:[mgr] debug client = 20 2024-05-09T04:32:35.253 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-05-09T04:32:35.253 INFO:tasks.ceph:[mgr] debug ms = 1 2024-05-09T04:32:35.253 INFO:tasks.ceph:[mon] debug mon = 20 2024-05-09T04:32:35.253 INFO:tasks.ceph:[mon] debug ms = 1 2024-05-09T04:32:35.253 INFO:tasks.ceph:[mon] debug paxos = 20 2024-05-09T04:32:35.253 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-05-09T04:32:35.253 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-05-09T04:32:35.254 INFO:tasks.ceph:[osd] bdev async discard = True 2024-05-09T04:32:35.254 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-05-09T04:32:35.254 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-05-09T04:32:35.254 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-05-09T04:32:35.254 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-05-09T04:32:35.254 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-05-09T04:32:35.255 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-05-09T04:32:35.255 INFO:tasks.ceph:[osd] debug ms = 1 2024-05-09T04:32:35.255 INFO:tasks.ceph:[osd] debug osd = 20 2024-05-09T04:32:35.255 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-05-09T04:32:35.255 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-05-09T04:32:35.255 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-05-09T04:32:35.256 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-05-09T04:32:35.256 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-05-09T04:32:35.256 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-05-09T04:32:35.256 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-05-09T04:32:35.256 INFO:tasks.ceph:Setting up mon.a... 2024-05-09T04:32:35.256 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-05-09T04:32:35.405 INFO:teuthology.orchestra.run.smithi145.stdout:creating /etc/ceph/ceph.keyring 2024-05-09T04:32:35.413 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-05-09T04:32:35.535 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-09T04:32:35.608 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.145'), ('mon.b', '172.21.15.192'), ('mon.c', '[v2:172.21.15.192:3301,v1:172.21.15.192:6790]')] 2024-05-09T04:32:35.608 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.145,172.21.15.192,[v2:172.21.15.192:3301,v1:172.21.15.192:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-05-09T04:32:35.609 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-05-09T04:32:35.609 DEBUG:teuthology.orchestra.run.smithi145:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-05-09T04:32:35.661 DEBUG:teuthology.orchestra.run.smithi145:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.145 --add b 172.21.15.192 --addv c '[v2:172.21.15.192:3301,v1:172.21.15.192:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-05-09T04:32:35.812 INFO:teuthology.orchestra.run.smithi145.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-05-09T04:32:35.813 INFO:teuthology.orchestra.run.smithi145.stdout:monmaptool: generated fsid 957ee6c1-073b-44cf-9b93-9491b3fdfc1d 2024-05-09T04:32:35.813 INFO:teuthology.orchestra.run.smithi145.stdout:setting min_mon_release = quincy 2024-05-09T04:32:35.813 INFO:teuthology.orchestra.run.smithi145.stdout:epoch 0 2024-05-09T04:32:35.813 INFO:teuthology.orchestra.run.smithi145.stdout:fsid 957ee6c1-073b-44cf-9b93-9491b3fdfc1d 2024-05-09T04:32:35.813 INFO:teuthology.orchestra.run.smithi145.stdout:last_changed 2024-05-09T04:32:35.814258+0000 2024-05-09T04:32:35.813 INFO:teuthology.orchestra.run.smithi145.stdout:created 2024-05-09T04:32:35.814258+0000 2024-05-09T04:32:35.813 INFO:teuthology.orchestra.run.smithi145.stdout:min_mon_release 17 (quincy) 2024-05-09T04:32:35.813 INFO:teuthology.orchestra.run.smithi145.stdout:election_strategy: 1 2024-05-09T04:32:35.813 INFO:teuthology.orchestra.run.smithi145.stdout:0: [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] mon.a 2024-05-09T04:32:35.813 INFO:teuthology.orchestra.run.smithi145.stdout:1: [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] mon.b 2024-05-09T04:32:35.813 INFO:teuthology.orchestra.run.smithi145.stdout:2: [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] mon.c 2024-05-09T04:32:35.813 INFO:teuthology.orchestra.run.smithi145.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-05-09T04:32:35.816 DEBUG:teuthology.orchestra.run.smithi145:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-05-09T04:32:35.866 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 957ee6c1-073b-44cf-9b93-9491b3fdfc1d... 2024-05-09T04:32:35.867 DEBUG:teuthology.orchestra.run.smithi145:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-09T04:32:35.871 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-09T04:32:36.002 INFO:teuthology.orchestra.run.smithi145.stdout:[global] 2024-05-09T04:32:36.003 INFO:teuthology.orchestra.run.smithi145.stdout: chdir = "" 2024-05-09T04:32:36.003 INFO:teuthology.orchestra.run.smithi145.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-09T04:32:36.003 INFO:teuthology.orchestra.run.smithi145.stdout: auth supported = cephx 2024-05-09T04:32:36.003 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:32:36.003 INFO:teuthology.orchestra.run.smithi145.stdout: filestore xattr use omap = true 2024-05-09T04:32:36.003 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:32:36.003 INFO:teuthology.orchestra.run.smithi145.stdout: mon clock drift allowed = 1.000 2024-05-09T04:32:36.003 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:32:36.003 INFO:teuthology.orchestra.run.smithi145.stdout: osd crush chooseleaf type = 0 2024-05-09T04:32:36.003 INFO:teuthology.orchestra.run.smithi145.stdout: auth debug = true 2024-05-09T04:32:36.003 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:32:36.003 INFO:teuthology.orchestra.run.smithi145.stdout: ms die on old message = true 2024-05-09T04:32:36.003 INFO:teuthology.orchestra.run.smithi145.stdout: ms die on bug = true 2024-05-09T04:32:36.004 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:32:36.004 INFO:teuthology.orchestra.run.smithi145.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-09T04:32:36.004 INFO:teuthology.orchestra.run.smithi145.stdout: mon pg warn max object skew = 0 2024-05-09T04:32:36.004 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:32:36.004 INFO:teuthology.orchestra.run.smithi145.stdout: # disable pg_autoscaler by default for new pools 2024-05-09T04:32:36.004 INFO:teuthology.orchestra.run.smithi145.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-09T04:32:36.004 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:32:36.004 INFO:teuthology.orchestra.run.smithi145.stdout: osd pool default size = 2 2024-05-09T04:32:36.004 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:32:36.004 INFO:teuthology.orchestra.run.smithi145.stdout: mon osd allow primary affinity = true 2024-05-09T04:32:36.004 INFO:teuthology.orchestra.run.smithi145.stdout: mon osd allow pg remap = true 2024-05-09T04:32:36.004 INFO:teuthology.orchestra.run.smithi145.stdout: mon warn on legacy crush tunables = false 2024-05-09T04:32:36.004 INFO:teuthology.orchestra.run.smithi145.stdout: mon warn on crush straw calc version zero = false 2024-05-09T04:32:36.004 INFO:teuthology.orchestra.run.smithi145.stdout: mon warn on no sortbitwise = false 2024-05-09T04:32:36.004 INFO:teuthology.orchestra.run.smithi145.stdout: mon warn on osd down out interval zero = false 2024-05-09T04:32:36.004 INFO:teuthology.orchestra.run.smithi145.stdout: mon warn on too few osds = false 2024-05-09T04:32:36.005 INFO:teuthology.orchestra.run.smithi145.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-09T04:32:36.005 INFO:teuthology.orchestra.run.smithi145.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-09T04:32:36.005 INFO:teuthology.orchestra.run.smithi145.stdout: mon_allow_pool_size_one = true 2024-05-09T04:32:36.005 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:32:36.005 INFO:teuthology.orchestra.run.smithi145.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-09T04:32:36.005 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:32:36.005 INFO:teuthology.orchestra.run.smithi145.stdout: osd default data pool replay window = 5 2024-05-09T04:32:36.005 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:32:36.005 INFO:teuthology.orchestra.run.smithi145.stdout: mon allow pool delete = true 2024-05-09T04:32:36.005 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:32:36.005 INFO:teuthology.orchestra.run.smithi145.stdout: mon cluster log file level = debug 2024-05-09T04:32:36.005 INFO:teuthology.orchestra.run.smithi145.stdout: debug asserts on shutdown = true 2024-05-09T04:32:36.005 INFO:teuthology.orchestra.run.smithi145.stdout: mon health detail to clog = false 2024-05-09T04:32:36.005 INFO:teuthology.orchestra.run.smithi145.stdout: mon host = "172.21.15.145,172.21.15.192,[v2:172.21.15.192:3301,v1:172.21.15.192:6790]" 2024-05-09T04:32:36.005 INFO:teuthology.orchestra.run.smithi145.stdout: fsid = 957ee6c1-073b-44cf-9b93-9491b3fdfc1d 2024-05-09T04:32:36.006 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:32:36.006 INFO:teuthology.orchestra.run.smithi145.stdout:[osd] 2024-05-09T04:32:36.006 INFO:teuthology.orchestra.run.smithi145.stdout: osd journal size = 100 2024-05-09T04:32:36.006 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:32:36.006 INFO:teuthology.orchestra.run.smithi145.stdout: osd scrub load threshold = 5.0 2024-05-09T04:32:36.006 INFO:teuthology.orchestra.run.smithi145.stdout: osd scrub max interval = 600 2024-05-09T04:32:36.006 INFO:teuthology.orchestra.run.smithi145.stdout: osd mclock profile = high_recovery_ops 2024-05-09T04:32:36.006 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:32:36.006 INFO:teuthology.orchestra.run.smithi145.stdout: osd recover clone overlap = true 2024-05-09T04:32:36.006 INFO:teuthology.orchestra.run.smithi145.stdout: osd recovery max chunk = 1048576 2024-05-09T04:32:36.006 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:32:36.006 INFO:teuthology.orchestra.run.smithi145.stdout: osd debug shutdown = true 2024-05-09T04:32:36.006 INFO:teuthology.orchestra.run.smithi145.stdout: osd debug op order = true 2024-05-09T04:32:36.006 INFO:teuthology.orchestra.run.smithi145.stdout: osd debug verify stray on activate = true 2024-05-09T04:32:36.006 INFO:teuthology.orchestra.run.smithi145.stdout: osd debug trim objects = true 2024-05-09T04:32:36.007 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:32:36.007 INFO:teuthology.orchestra.run.smithi145.stdout: osd open classes on start = true 2024-05-09T04:32:36.007 INFO:teuthology.orchestra.run.smithi145.stdout: osd debug pg log writeout = true 2024-05-09T04:32:36.007 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:32:36.007 INFO:teuthology.orchestra.run.smithi145.stdout: osd deep scrub update digest min age = 30 2024-05-09T04:32:36.007 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:32:36.007 INFO:teuthology.orchestra.run.smithi145.stdout: osd map max advance = 10 2024-05-09T04:32:36.007 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:32:36.007 INFO:teuthology.orchestra.run.smithi145.stdout: journal zero on create = true 2024-05-09T04:32:36.007 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:32:36.007 INFO:teuthology.orchestra.run.smithi145.stdout: filestore ondisk finisher threads = 3 2024-05-09T04:32:36.007 INFO:teuthology.orchestra.run.smithi145.stdout: filestore apply finisher threads = 3 2024-05-09T04:32:36.007 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:32:36.007 INFO:teuthology.orchestra.run.smithi145.stdout: bdev debug aio = true 2024-05-09T04:32:36.007 INFO:teuthology.orchestra.run.smithi145.stdout: osd debug misdirected ops = true 2024-05-09T04:32:36.008 INFO:teuthology.orchestra.run.smithi145.stdout: bdev async discard = True 2024-05-09T04:32:36.008 INFO:teuthology.orchestra.run.smithi145.stdout: bdev enable discard = True 2024-05-09T04:32:36.008 INFO:teuthology.orchestra.run.smithi145.stdout: bluestore allocator = bitmap 2024-05-09T04:32:36.008 INFO:teuthology.orchestra.run.smithi145.stdout: bluestore block size = 96636764160 2024-05-09T04:32:36.008 INFO:teuthology.orchestra.run.smithi145.stdout: bluestore fsck on mount = True 2024-05-09T04:32:36.008 INFO:teuthology.orchestra.run.smithi145.stdout: debug bluefs = 1/20 2024-05-09T04:32:36.008 INFO:teuthology.orchestra.run.smithi145.stdout: debug bluestore = 1/20 2024-05-09T04:32:36.008 INFO:teuthology.orchestra.run.smithi145.stdout: debug ms = 1 2024-05-09T04:32:36.008 INFO:teuthology.orchestra.run.smithi145.stdout: debug osd = 20 2024-05-09T04:32:36.008 INFO:teuthology.orchestra.run.smithi145.stdout: debug rocksdb = 4/10 2024-05-09T04:32:36.008 INFO:teuthology.orchestra.run.smithi145.stdout: mon osd backfillfull_ratio = 0.85 2024-05-09T04:32:36.008 INFO:teuthology.orchestra.run.smithi145.stdout: mon osd full ratio = 0.9 2024-05-09T04:32:36.008 INFO:teuthology.orchestra.run.smithi145.stdout: mon osd nearfull ratio = 0.8 2024-05-09T04:32:36.008 INFO:teuthology.orchestra.run.smithi145.stdout: osd failsafe full ratio = 0.95 2024-05-09T04:32:36.008 INFO:teuthology.orchestra.run.smithi145.stdout: osd objectstore = bluestore 2024-05-09T04:32:36.009 INFO:teuthology.orchestra.run.smithi145.stdout: osd op complaint time = 180 2024-05-09T04:32:36.009 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:32:36.009 INFO:teuthology.orchestra.run.smithi145.stdout:[mgr] 2024-05-09T04:32:36.009 INFO:teuthology.orchestra.run.smithi145.stdout: debug ms = 1 2024-05-09T04:32:36.009 INFO:teuthology.orchestra.run.smithi145.stdout: debug mgr = 20 2024-05-09T04:32:36.009 INFO:teuthology.orchestra.run.smithi145.stdout: debug mon = 20 2024-05-09T04:32:36.009 INFO:teuthology.orchestra.run.smithi145.stdout: debug auth = 20 2024-05-09T04:32:36.009 INFO:teuthology.orchestra.run.smithi145.stdout: mon reweight min pgs per osd = 4 2024-05-09T04:32:36.009 INFO:teuthology.orchestra.run.smithi145.stdout: mon reweight min bytes per osd = 10 2024-05-09T04:32:36.009 INFO:teuthology.orchestra.run.smithi145.stdout: mgr/telemetry/nag = false 2024-05-09T04:32:36.009 INFO:teuthology.orchestra.run.smithi145.stdout: debug client = 20 2024-05-09T04:32:36.009 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:32:36.009 INFO:teuthology.orchestra.run.smithi145.stdout:[mon] 2024-05-09T04:32:36.009 INFO:teuthology.orchestra.run.smithi145.stdout: debug ms = 1 2024-05-09T04:32:36.009 INFO:teuthology.orchestra.run.smithi145.stdout: debug mon = 20 2024-05-09T04:32:36.010 INFO:teuthology.orchestra.run.smithi145.stdout: debug paxos = 20 2024-05-09T04:32:36.010 INFO:teuthology.orchestra.run.smithi145.stdout: debug auth = 20 2024-05-09T04:32:36.010 INFO:teuthology.orchestra.run.smithi145.stdout: mon data avail warn = 5 2024-05-09T04:32:36.010 INFO:teuthology.orchestra.run.smithi145.stdout: mon mgr mkfs grace = 240 2024-05-09T04:32:36.010 INFO:teuthology.orchestra.run.smithi145.stdout: mon reweight min pgs per osd = 4 2024-05-09T04:32:36.010 INFO:teuthology.orchestra.run.smithi145.stdout: mon osd reporter subtree level = osd 2024-05-09T04:32:36.010 INFO:teuthology.orchestra.run.smithi145.stdout: mon osd prime pg temp = true 2024-05-09T04:32:36.010 INFO:teuthology.orchestra.run.smithi145.stdout: mon reweight min bytes per osd = 10 2024-05-09T04:32:36.010 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:32:36.010 INFO:teuthology.orchestra.run.smithi145.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-09T04:32:36.010 INFO:teuthology.orchestra.run.smithi145.stdout: auth mon ticket ttl = 660 # 11m 2024-05-09T04:32:36.010 INFO:teuthology.orchestra.run.smithi145.stdout: auth service ticket ttl = 240 # 4m 2024-05-09T04:32:36.010 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:32:36.010 INFO:teuthology.orchestra.run.smithi145.stdout: # don't complain about insecure global_id in the test suite 2024-05-09T04:32:36.010 INFO:teuthology.orchestra.run.smithi145.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-09T04:32:36.011 INFO:teuthology.orchestra.run.smithi145.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-09T04:32:36.011 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:32:36.011 INFO:teuthology.orchestra.run.smithi145.stdout: # 1m isn't quite enough 2024-05-09T04:32:36.011 INFO:teuthology.orchestra.run.smithi145.stdout: mon_down_mkfs_grace = 2m 2024-05-09T04:32:36.011 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:32:36.011 INFO:teuthology.orchestra.run.smithi145.stdout: mon_warn_on_filestore_osds = false 2024-05-09T04:32:36.011 INFO:teuthology.orchestra.run.smithi145.stdout: mon down mkfs grace = 300 2024-05-09T04:32:36.011 INFO:teuthology.orchestra.run.smithi145.stdout: mon op complaint time = 120 2024-05-09T04:32:36.011 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:32:36.011 INFO:teuthology.orchestra.run.smithi145.stdout:[client] 2024-05-09T04:32:36.011 INFO:teuthology.orchestra.run.smithi145.stdout: rgw cache enabled = true 2024-05-09T04:32:36.011 INFO:teuthology.orchestra.run.smithi145.stdout: rgw enable ops log = true 2024-05-09T04:32:36.011 INFO:teuthology.orchestra.run.smithi145.stdout: rgw enable usage log = true 2024-05-09T04:32:36.011 INFO:teuthology.orchestra.run.smithi145.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-09T04:32:36.011 INFO:teuthology.orchestra.run.smithi145.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-09T04:32:36.011 INFO:teuthology.orchestra.run.smithi145.stdout: client mount timeout = 600 2024-05-09T04:32:36.012 INFO:teuthology.orchestra.run.smithi145.stdout: debug client = 20 2024-05-09T04:32:36.012 INFO:teuthology.orchestra.run.smithi145.stdout: debug ms = 1 2024-05-09T04:32:36.012 INFO:teuthology.orchestra.run.smithi145.stdout: pid file = "" 2024-05-09T04:32:36.012 INFO:teuthology.orchestra.run.smithi145.stdout: rados mon op timeout = 900 2024-05-09T04:32:36.012 INFO:teuthology.orchestra.run.smithi145.stdout: rados osd op timeout = 900 2024-05-09T04:32:36.012 INFO:teuthology.orchestra.run.smithi145.stdout:[mon.a] 2024-05-09T04:32:36.012 INFO:teuthology.orchestra.run.smithi145.stdout:[mon.b] 2024-05-09T04:32:36.012 INFO:teuthology.orchestra.run.smithi145.stdout:[mon.c] 2024-05-09T04:32:36.012 INFO:teuthology.orchestra.run.smithi145.stdout:[mds.a] 2024-05-09T04:32:36.012 INFO:teuthology.orchestra.run.smithi145.stdout:[mds.c] 2024-05-09T04:32:36.012 INFO:teuthology.orchestra.run.smithi145.stdout:[mds.b] 2024-05-09T04:32:36.012 INFO:teuthology.orchestra.run.smithi145.stdout:[mds.d] 2024-05-09T04:32:36.012 INFO:teuthology.orchestra.run.smithi145.stdout:[mds] 2024-05-09T04:32:36.012 INFO:teuthology.orchestra.run.smithi145.stdout: debug mds = 20 2024-05-09T04:32:36.012 INFO:teuthology.orchestra.run.smithi145.stdout: debug mds balancer = 20 2024-05-09T04:32:36.012 INFO:teuthology.orchestra.run.smithi145.stdout: debug ms = 1 2024-05-09T04:32:36.013 INFO:teuthology.orchestra.run.smithi145.stdout: mds debug frag = True 2024-05-09T04:32:36.013 INFO:teuthology.orchestra.run.smithi145.stdout: mds debug scatterstat = True 2024-05-09T04:32:36.013 INFO:teuthology.orchestra.run.smithi145.stdout: mds op complaint time = 180 2024-05-09T04:32:36.013 INFO:teuthology.orchestra.run.smithi145.stdout: mds verify scatter = True 2024-05-09T04:32:36.013 INFO:teuthology.orchestra.run.smithi145.stdout: osd op complaint time = 180 2024-05-09T04:32:36.013 INFO:teuthology.orchestra.run.smithi145.stdout: rados mon op timeout = 900 2024-05-09T04:32:36.013 INFO:teuthology.orchestra.run.smithi145.stdout: rados osd op timeout = 900 2024-05-09T04:32:36.013 INFO:teuthology.orchestra.run.smithi192.stdout:[global] 2024-05-09T04:32:36.014 INFO:teuthology.orchestra.run.smithi192.stdout: chdir = "" 2024-05-09T04:32:36.014 INFO:teuthology.orchestra.run.smithi192.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-09T04:32:36.014 INFO:teuthology.orchestra.run.smithi192.stdout: auth supported = cephx 2024-05-09T04:32:36.014 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-09T04:32:36.014 INFO:teuthology.orchestra.run.smithi192.stdout: filestore xattr use omap = true 2024-05-09T04:32:36.014 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-09T04:32:36.014 INFO:teuthology.orchestra.run.smithi192.stdout: mon clock drift allowed = 1.000 2024-05-09T04:32:36.014 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-09T04:32:36.014 INFO:teuthology.orchestra.run.smithi192.stdout: osd crush chooseleaf type = 0 2024-05-09T04:32:36.014 INFO:teuthology.orchestra.run.smithi192.stdout: auth debug = true 2024-05-09T04:32:36.014 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-09T04:32:36.014 INFO:teuthology.orchestra.run.smithi192.stdout: ms die on old message = true 2024-05-09T04:32:36.014 INFO:teuthology.orchestra.run.smithi192.stdout: ms die on bug = true 2024-05-09T04:32:36.014 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-09T04:32:36.014 INFO:teuthology.orchestra.run.smithi192.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-09T04:32:36.015 INFO:teuthology.orchestra.run.smithi192.stdout: mon pg warn max object skew = 0 2024-05-09T04:32:36.015 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-09T04:32:36.015 INFO:teuthology.orchestra.run.smithi192.stdout: # disable pg_autoscaler by default for new pools 2024-05-09T04:32:36.015 INFO:teuthology.orchestra.run.smithi192.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-09T04:32:36.015 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-09T04:32:36.015 INFO:teuthology.orchestra.run.smithi192.stdout: osd pool default size = 2 2024-05-09T04:32:36.015 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-09T04:32:36.015 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd allow primary affinity = true 2024-05-09T04:32:36.015 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd allow pg remap = true 2024-05-09T04:32:36.015 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on legacy crush tunables = false 2024-05-09T04:32:36.015 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on crush straw calc version zero = false 2024-05-09T04:32:36.015 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on no sortbitwise = false 2024-05-09T04:32:36.015 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on osd down out interval zero = false 2024-05-09T04:32:36.015 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on too few osds = false 2024-05-09T04:32:36.015 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-09T04:32:36.016 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-09T04:32:36.016 INFO:teuthology.orchestra.run.smithi192.stdout: mon_allow_pool_size_one = true 2024-05-09T04:32:36.016 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-09T04:32:36.016 INFO:teuthology.orchestra.run.smithi192.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-09T04:32:36.016 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-09T04:32:36.016 INFO:teuthology.orchestra.run.smithi192.stdout: osd default data pool replay window = 5 2024-05-09T04:32:36.016 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-09T04:32:36.016 INFO:teuthology.orchestra.run.smithi192.stdout: mon allow pool delete = true 2024-05-09T04:32:36.016 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-09T04:32:36.016 INFO:teuthology.orchestra.run.smithi192.stdout: mon cluster log file level = debug 2024-05-09T04:32:36.016 INFO:teuthology.orchestra.run.smithi192.stdout: debug asserts on shutdown = true 2024-05-09T04:32:36.016 INFO:teuthology.orchestra.run.smithi192.stdout: mon health detail to clog = false 2024-05-09T04:32:36.016 INFO:teuthology.orchestra.run.smithi192.stdout: mon host = "172.21.15.145,172.21.15.192,[v2:172.21.15.192:3301,v1:172.21.15.192:6790]" 2024-05-09T04:32:36.016 INFO:teuthology.orchestra.run.smithi192.stdout: fsid = 957ee6c1-073b-44cf-9b93-9491b3fdfc1d 2024-05-09T04:32:36.016 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-09T04:32:36.016 INFO:teuthology.orchestra.run.smithi192.stdout:[osd] 2024-05-09T04:32:36.017 INFO:teuthology.orchestra.run.smithi192.stdout: osd journal size = 100 2024-05-09T04:32:36.017 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-09T04:32:36.017 INFO:teuthology.orchestra.run.smithi192.stdout: osd scrub load threshold = 5.0 2024-05-09T04:32:36.017 INFO:teuthology.orchestra.run.smithi192.stdout: osd scrub max interval = 600 2024-05-09T04:32:36.017 INFO:teuthology.orchestra.run.smithi192.stdout: osd mclock profile = high_recovery_ops 2024-05-09T04:32:36.017 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-09T04:32:36.017 INFO:teuthology.orchestra.run.smithi192.stdout: osd recover clone overlap = true 2024-05-09T04:32:36.017 INFO:teuthology.orchestra.run.smithi192.stdout: osd recovery max chunk = 1048576 2024-05-09T04:32:36.017 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-09T04:32:36.017 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug shutdown = true 2024-05-09T04:32:36.017 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug op order = true 2024-05-09T04:32:36.017 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug verify stray on activate = true 2024-05-09T04:32:36.017 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug trim objects = true 2024-05-09T04:32:36.017 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-09T04:32:36.017 INFO:teuthology.orchestra.run.smithi192.stdout: osd open classes on start = true 2024-05-09T04:32:36.017 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug pg log writeout = true 2024-05-09T04:32:36.018 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-09T04:32:36.018 INFO:teuthology.orchestra.run.smithi192.stdout: osd deep scrub update digest min age = 30 2024-05-09T04:32:36.018 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-09T04:32:36.018 INFO:teuthology.orchestra.run.smithi192.stdout: osd map max advance = 10 2024-05-09T04:32:36.018 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-09T04:32:36.018 INFO:teuthology.orchestra.run.smithi192.stdout: journal zero on create = true 2024-05-09T04:32:36.018 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-09T04:32:36.018 INFO:teuthology.orchestra.run.smithi192.stdout: filestore ondisk finisher threads = 3 2024-05-09T04:32:36.018 INFO:teuthology.orchestra.run.smithi192.stdout: filestore apply finisher threads = 3 2024-05-09T04:32:36.018 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-09T04:32:36.018 INFO:teuthology.orchestra.run.smithi192.stdout: bdev debug aio = true 2024-05-09T04:32:36.018 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug misdirected ops = true 2024-05-09T04:32:36.018 INFO:teuthology.orchestra.run.smithi192.stdout: bdev async discard = True 2024-05-09T04:32:36.018 INFO:teuthology.orchestra.run.smithi192.stdout: bdev enable discard = True 2024-05-09T04:32:36.018 INFO:teuthology.orchestra.run.smithi192.stdout: bluestore allocator = bitmap 2024-05-09T04:32:36.019 INFO:teuthology.orchestra.run.smithi192.stdout: bluestore block size = 96636764160 2024-05-09T04:32:36.019 INFO:teuthology.orchestra.run.smithi192.stdout: bluestore fsck on mount = True 2024-05-09T04:32:36.019 INFO:teuthology.orchestra.run.smithi192.stdout: debug bluefs = 1/20 2024-05-09T04:32:36.019 INFO:teuthology.orchestra.run.smithi192.stdout: debug bluestore = 1/20 2024-05-09T04:32:36.019 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2024-05-09T04:32:36.019 INFO:teuthology.orchestra.run.smithi192.stdout: debug osd = 20 2024-05-09T04:32:36.019 INFO:teuthology.orchestra.run.smithi192.stdout: debug rocksdb = 4/10 2024-05-09T04:32:36.019 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd backfillfull_ratio = 0.85 2024-05-09T04:32:36.019 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd full ratio = 0.9 2024-05-09T04:32:36.019 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd nearfull ratio = 0.8 2024-05-09T04:32:36.019 INFO:teuthology.orchestra.run.smithi192.stdout: osd failsafe full ratio = 0.95 2024-05-09T04:32:36.019 INFO:teuthology.orchestra.run.smithi192.stdout: osd objectstore = bluestore 2024-05-09T04:32:36.019 INFO:teuthology.orchestra.run.smithi192.stdout: osd op complaint time = 180 2024-05-09T04:32:36.019 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-09T04:32:36.019 INFO:teuthology.orchestra.run.smithi192.stdout:[mgr] 2024-05-09T04:32:36.019 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2024-05-09T04:32:36.020 INFO:teuthology.orchestra.run.smithi192.stdout: debug mgr = 20 2024-05-09T04:32:36.020 INFO:teuthology.orchestra.run.smithi192.stdout: debug mon = 20 2024-05-09T04:32:36.020 INFO:teuthology.orchestra.run.smithi192.stdout: debug auth = 20 2024-05-09T04:32:36.020 INFO:teuthology.orchestra.run.smithi192.stdout: mon reweight min pgs per osd = 4 2024-05-09T04:32:36.020 INFO:teuthology.orchestra.run.smithi192.stdout: mon reweight min bytes per osd = 10 2024-05-09T04:32:36.020 INFO:teuthology.orchestra.run.smithi192.stdout: mgr/telemetry/nag = false 2024-05-09T04:32:36.020 INFO:teuthology.orchestra.run.smithi192.stdout: debug client = 20 2024-05-09T04:32:36.020 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-09T04:32:36.020 INFO:teuthology.orchestra.run.smithi192.stdout:[mon] 2024-05-09T04:32:36.020 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2024-05-09T04:32:36.020 INFO:teuthology.orchestra.run.smithi192.stdout: debug mon = 20 2024-05-09T04:32:36.020 INFO:teuthology.orchestra.run.smithi192.stdout: debug paxos = 20 2024-05-09T04:32:36.020 INFO:teuthology.orchestra.run.smithi192.stdout: debug auth = 20 2024-05-09T04:32:36.020 INFO:teuthology.orchestra.run.smithi192.stdout: mon data avail warn = 5 2024-05-09T04:32:36.020 INFO:teuthology.orchestra.run.smithi192.stdout: mon mgr mkfs grace = 240 2024-05-09T04:32:36.020 INFO:teuthology.orchestra.run.smithi192.stdout: mon reweight min pgs per osd = 4 2024-05-09T04:32:36.021 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd reporter subtree level = osd 2024-05-09T04:32:36.021 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd prime pg temp = true 2024-05-09T04:32:36.021 INFO:teuthology.orchestra.run.smithi192.stdout: mon reweight min bytes per osd = 10 2024-05-09T04:32:36.021 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-09T04:32:36.021 INFO:teuthology.orchestra.run.smithi192.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-09T04:32:36.021 INFO:teuthology.orchestra.run.smithi192.stdout: auth mon ticket ttl = 660 # 11m 2024-05-09T04:32:36.021 INFO:teuthology.orchestra.run.smithi192.stdout: auth service ticket ttl = 240 # 4m 2024-05-09T04:32:36.021 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-09T04:32:36.021 INFO:teuthology.orchestra.run.smithi192.stdout: # don't complain about insecure global_id in the test suite 2024-05-09T04:32:36.021 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-09T04:32:36.021 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-09T04:32:36.021 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-09T04:32:36.021 INFO:teuthology.orchestra.run.smithi192.stdout: # 1m isn't quite enough 2024-05-09T04:32:36.021 INFO:teuthology.orchestra.run.smithi192.stdout: mon_down_mkfs_grace = 2m 2024-05-09T04:32:36.021 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-09T04:32:36.021 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_filestore_osds = false 2024-05-09T04:32:36.022 INFO:teuthology.orchestra.run.smithi192.stdout: mon down mkfs grace = 300 2024-05-09T04:32:36.022 INFO:teuthology.orchestra.run.smithi192.stdout: mon op complaint time = 120 2024-05-09T04:32:36.022 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-05-09T04:32:36.022 INFO:teuthology.orchestra.run.smithi192.stdout:[client] 2024-05-09T04:32:36.022 INFO:teuthology.orchestra.run.smithi192.stdout: rgw cache enabled = true 2024-05-09T04:32:36.022 INFO:teuthology.orchestra.run.smithi192.stdout: rgw enable ops log = true 2024-05-09T04:32:36.022 INFO:teuthology.orchestra.run.smithi192.stdout: rgw enable usage log = true 2024-05-09T04:32:36.022 INFO:teuthology.orchestra.run.smithi192.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-09T04:32:36.022 INFO:teuthology.orchestra.run.smithi192.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-09T04:32:36.022 INFO:teuthology.orchestra.run.smithi192.stdout: client mount timeout = 600 2024-05-09T04:32:36.022 INFO:teuthology.orchestra.run.smithi192.stdout: debug client = 20 2024-05-09T04:32:36.022 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2024-05-09T04:32:36.022 INFO:teuthology.orchestra.run.smithi192.stdout: pid file = "" 2024-05-09T04:32:36.022 INFO:teuthology.orchestra.run.smithi192.stdout: rados mon op timeout = 900 2024-05-09T04:32:36.022 INFO:teuthology.orchestra.run.smithi192.stdout: rados osd op timeout = 900 2024-05-09T04:32:36.022 INFO:teuthology.orchestra.run.smithi192.stdout:[mon.a] 2024-05-09T04:32:36.023 INFO:teuthology.orchestra.run.smithi192.stdout:[mon.b] 2024-05-09T04:32:36.023 INFO:teuthology.orchestra.run.smithi192.stdout:[mon.c] 2024-05-09T04:32:36.023 INFO:teuthology.orchestra.run.smithi192.stdout:[mds.a] 2024-05-09T04:32:36.023 INFO:teuthology.orchestra.run.smithi192.stdout:[mds.c] 2024-05-09T04:32:36.023 INFO:teuthology.orchestra.run.smithi192.stdout:[mds.b] 2024-05-09T04:32:36.023 INFO:teuthology.orchestra.run.smithi192.stdout:[mds.d] 2024-05-09T04:32:36.023 INFO:teuthology.orchestra.run.smithi192.stdout:[mds] 2024-05-09T04:32:36.023 INFO:teuthology.orchestra.run.smithi192.stdout: debug mds = 20 2024-05-09T04:32:36.023 INFO:teuthology.orchestra.run.smithi192.stdout: debug mds balancer = 20 2024-05-09T04:32:36.023 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2024-05-09T04:32:36.023 INFO:teuthology.orchestra.run.smithi192.stdout: mds debug frag = True 2024-05-09T04:32:36.023 INFO:teuthology.orchestra.run.smithi192.stdout: mds debug scatterstat = True 2024-05-09T04:32:36.023 INFO:teuthology.orchestra.run.smithi192.stdout: mds op complaint time = 180 2024-05-09T04:32:36.023 INFO:teuthology.orchestra.run.smithi192.stdout: mds verify scatter = True 2024-05-09T04:32:36.023 INFO:teuthology.orchestra.run.smithi192.stdout: osd op complaint time = 180 2024-05-09T04:32:36.023 INFO:teuthology.orchestra.run.smithi192.stdout: rados mon op timeout = 900 2024-05-09T04:32:36.024 INFO:teuthology.orchestra.run.smithi192.stdout: rados osd op timeout = 900 2024-05-09T04:32:36.049 INFO:tasks.ceph:Creating admin key on mon.a... 2024-05-09T04:32:36.050 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-05-09T04:32:36.177 INFO:tasks.ceph:Copying monmap to all nodes... 2024-05-09T04:32:36.177 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-05-09T04:32:36.178 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-05-09T04:32:36.224 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-05-09T04:32:36.224 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-05-09T04:32:36.320 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi145.front.sepia.ceph.com 2024-05-09T04:32:36.320 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-05-09T04:32:36.320 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-09T04:32:36.320 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-09T04:32:36.447 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-05-09T04:32:36.447 DEBUG:teuthology.orchestra.run.smithi145:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-09T04:32:36.548 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi192.front.sepia.ceph.com 2024-05-09T04:32:36.548 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-09T04:32:36.548 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-09T04:32:36.548 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-09T04:32:36.685 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-09T04:32:36.685 DEBUG:teuthology.orchestra.run.smithi192:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-09T04:32:36.782 INFO:tasks.ceph:Setting up mon nodes... 2024-05-09T04:32:36.782 INFO:tasks.ceph:Setting up mgr nodes... 2024-05-09T04:32:36.782 DEBUG:teuthology.orchestra.run.smithi145:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-05-09T04:32:36.932 INFO:teuthology.orchestra.run.smithi145.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-05-09T04:32:36.942 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-05-09T04:32:37.118 INFO:teuthology.orchestra.run.smithi192.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-05-09T04:32:37.129 INFO:tasks.ceph:Setting up mds nodes... 2024-05-09T04:32:37.129 DEBUG:teuthology.orchestra.run.smithi145:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-05-09T04:32:37.291 INFO:teuthology.orchestra.run.smithi145.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-05-09T04:32:37.298 DEBUG:teuthology.orchestra.run.smithi145:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-05-09T04:32:37.379 DEBUG:teuthology.orchestra.run.smithi145:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-05-09T04:32:37.531 INFO:teuthology.orchestra.run.smithi145.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-05-09T04:32:37.540 DEBUG:teuthology.orchestra.run.smithi145:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-05-09T04:32:37.624 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-05-09T04:32:37.781 INFO:teuthology.orchestra.run.smithi192.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-05-09T04:32:37.789 DEBUG:teuthology.orchestra.run.smithi192:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-05-09T04:32:37.863 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-05-09T04:32:38.020 INFO:teuthology.orchestra.run.smithi192.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-05-09T04:32:38.028 DEBUG:teuthology.orchestra.run.smithi192:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-05-09T04:32:38.111 INFO:tasks.ceph_client:Setting up client nodes... 2024-05-09T04:32:38.111 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-09T04:32:38.230 INFO:teuthology.orchestra.run.smithi145.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-05-09T04:32:38.268 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-05-09T04:32:38.382 INFO:teuthology.orchestra.run.smithi145.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-05-09T04:32:38.425 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-05-09T04:32:38.546 INFO:teuthology.orchestra.run.smithi192.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-05-09T04:32:38.592 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-05-09T04:32:38.709 INFO:teuthology.orchestra.run.smithi192.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-05-09T04:32:38.748 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-05-09T04:32:38.748 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi145.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi192.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-09T04:32:38.748 DEBUG:teuthology.orchestra.run.smithi145:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-05-09T04:32:38.839 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-09T04:32:38.839 INFO:tasks.ceph:role: osd.0 2024-05-09T04:32:38.839 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi145.front.sepia.ceph.com 2024-05-09T04:32:38.839 DEBUG:teuthology.orchestra.run.smithi145:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-09T04:32:38.924 INFO:teuthology.orchestra.run.smithi145.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T04:32:38.924 INFO:teuthology.orchestra.run.smithi145.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T04:32:38.924 INFO:teuthology.orchestra.run.smithi145.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T04:32:38.924 INFO:teuthology.orchestra.run.smithi145.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-09T04:32:38.924 INFO:teuthology.orchestra.run.smithi145.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T04:32:38.924 INFO:teuthology.orchestra.run.smithi145.stdout: = sunit=0 swidth=0 blks 2024-05-09T04:32:38.924 INFO:teuthology.orchestra.run.smithi145.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T04:32:38.925 INFO:teuthology.orchestra.run.smithi145.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-09T04:32:38.925 INFO:teuthology.orchestra.run.smithi145.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T04:32:38.925 INFO:teuthology.orchestra.run.smithi145.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T04:32:39.334 INFO:teuthology.orchestra.run.smithi145.stdout:Discarding blocks...Done. 2024-05-09T04:32:39.340 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi145.front.sepia.ceph.com -o noatime 2024-05-09T04:32:39.340 DEBUG:teuthology.orchestra.run.smithi145:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-05-09T04:32:39.483 DEBUG:teuthology.orchestra.run.smithi145:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-05-09T04:32:39.573 DEBUG:teuthology.orchestra.run.smithi145:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-05-09T04:32:39.652 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-09T04:32:39.652 INFO:tasks.ceph:role: osd.1 2024-05-09T04:32:39.652 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi145.front.sepia.ceph.com 2024-05-09T04:32:39.653 DEBUG:teuthology.orchestra.run.smithi145:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-09T04:32:39.734 INFO:teuthology.orchestra.run.smithi145.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T04:32:39.734 INFO:teuthology.orchestra.run.smithi145.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T04:32:39.734 INFO:teuthology.orchestra.run.smithi145.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T04:32:39.734 INFO:teuthology.orchestra.run.smithi145.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-09T04:32:39.734 INFO:teuthology.orchestra.run.smithi145.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T04:32:39.735 INFO:teuthology.orchestra.run.smithi145.stdout: = sunit=0 swidth=0 blks 2024-05-09T04:32:39.735 INFO:teuthology.orchestra.run.smithi145.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T04:32:39.735 INFO:teuthology.orchestra.run.smithi145.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-09T04:32:39.735 INFO:teuthology.orchestra.run.smithi145.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T04:32:39.735 INFO:teuthology.orchestra.run.smithi145.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T04:32:40.088 INFO:teuthology.orchestra.run.smithi145.stdout:Discarding blocks...Done. 2024-05-09T04:32:40.093 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi145.front.sepia.ceph.com -o noatime 2024-05-09T04:32:40.093 DEBUG:teuthology.orchestra.run.smithi145:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-05-09T04:32:40.187 DEBUG:teuthology.orchestra.run.smithi145:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-05-09T04:32:40.255 DEBUG:teuthology.orchestra.run.smithi145:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-05-09T04:32:40.318 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-09T04:32:40.319 INFO:tasks.ceph:role: osd.2 2024-05-09T04:32:40.319 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi145.front.sepia.ceph.com 2024-05-09T04:32:40.319 DEBUG:teuthology.orchestra.run.smithi145:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-09T04:32:40.386 INFO:teuthology.orchestra.run.smithi145.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T04:32:40.386 INFO:teuthology.orchestra.run.smithi145.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T04:32:40.386 INFO:teuthology.orchestra.run.smithi145.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T04:32:40.386 INFO:teuthology.orchestra.run.smithi145.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-09T04:32:40.386 INFO:teuthology.orchestra.run.smithi145.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T04:32:40.387 INFO:teuthology.orchestra.run.smithi145.stdout: = sunit=0 swidth=0 blks 2024-05-09T04:32:40.387 INFO:teuthology.orchestra.run.smithi145.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T04:32:40.387 INFO:teuthology.orchestra.run.smithi145.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-09T04:32:40.387 INFO:teuthology.orchestra.run.smithi145.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T04:32:40.387 INFO:teuthology.orchestra.run.smithi145.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T04:32:40.767 INFO:teuthology.orchestra.run.smithi145.stdout:Discarding blocks...Done. 2024-05-09T04:32:40.773 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi145.front.sepia.ceph.com -o noatime 2024-05-09T04:32:40.773 DEBUG:teuthology.orchestra.run.smithi145:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-05-09T04:32:40.875 DEBUG:teuthology.orchestra.run.smithi145:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-05-09T04:32:40.942 DEBUG:teuthology.orchestra.run.smithi145:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-05-09T04:32:41.001 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-09T04:32:41.001 INFO:tasks.ceph:role: osd.3 2024-05-09T04:32:41.001 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi145.front.sepia.ceph.com 2024-05-09T04:32:41.001 DEBUG:teuthology.orchestra.run.smithi145:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-09T04:32:41.080 INFO:teuthology.orchestra.run.smithi145.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T04:32:41.080 INFO:teuthology.orchestra.run.smithi145.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T04:32:41.080 INFO:teuthology.orchestra.run.smithi145.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T04:32:41.080 INFO:teuthology.orchestra.run.smithi145.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-09T04:32:41.081 INFO:teuthology.orchestra.run.smithi145.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T04:32:41.081 INFO:teuthology.orchestra.run.smithi145.stdout: = sunit=0 swidth=0 blks 2024-05-09T04:32:41.081 INFO:teuthology.orchestra.run.smithi145.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T04:32:41.081 INFO:teuthology.orchestra.run.smithi145.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-09T04:32:41.081 INFO:teuthology.orchestra.run.smithi145.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T04:32:41.081 INFO:teuthology.orchestra.run.smithi145.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T04:32:41.485 INFO:teuthology.orchestra.run.smithi145.stdout:Discarding blocks...Done. 2024-05-09T04:32:41.492 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi145.front.sepia.ceph.com -o noatime 2024-05-09T04:32:41.492 DEBUG:teuthology.orchestra.run.smithi145:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-05-09T04:32:41.631 DEBUG:teuthology.orchestra.run.smithi145:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-05-09T04:32:41.712 DEBUG:teuthology.orchestra.run.smithi145:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T04:32:41.844 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:32:41.845+0000 7f4eeedb6640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-05-09T04:32:41.845 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:32:41.846+0000 7f4eeedb6640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-05-09T04:32:41.845 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:32:41.846+0000 7f4eeedb6640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-09T04:32:41.846 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:32:41.846+0000 7f4eeedb6640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-09T04:32:41.846 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:32:41.846+0000 7f4eeedb6640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-09T04:32:41.854 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:32:41.855+0000 7f4eeedb6640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-05-09T04:32:45.072 DEBUG:teuthology.orchestra.run.smithi145:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-09T04:32:45.158 DEBUG:teuthology.orchestra.run.smithi145:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T04:32:45.285 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:32:45.286+0000 7f10e6e19640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-05-09T04:32:45.286 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:32:45.286+0000 7f10e6e19640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-05-09T04:32:45.286 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:32:45.286+0000 7f10e6e19640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-09T04:32:45.286 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:32:45.286+0000 7f10e6e19640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-09T04:32:45.286 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:32:45.286+0000 7f10e6e19640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-09T04:32:45.288 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:32:45.289+0000 7f10e6e19640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-05-09T04:32:48.500 DEBUG:teuthology.orchestra.run.smithi145:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-09T04:32:48.586 DEBUG:teuthology.orchestra.run.smithi145:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T04:32:48.705 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:32:48.706+0000 7f5eaea0d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-05-09T04:32:48.706 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:32:48.706+0000 7f5eaea0d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-05-09T04:32:48.706 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:32:48.706+0000 7f5eaea0d640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-09T04:32:48.706 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:32:48.706+0000 7f5eaea0d640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-09T04:32:48.706 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:32:48.706+0000 7f5eaea0d640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-09T04:32:48.708 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:32:48.709+0000 7f5eaea0d640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-05-09T04:32:51.669 DEBUG:teuthology.orchestra.run.smithi145:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-09T04:32:51.754 DEBUG:teuthology.orchestra.run.smithi145:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T04:32:51.879 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:32:51.879+0000 7f9f6881a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-05-09T04:32:51.879 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:32:51.879+0000 7f9f6881a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-05-09T04:32:51.879 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:32:51.879+0000 7f9f6881a640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-09T04:32:51.879 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:32:51.880+0000 7f9f6881a640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-09T04:32:51.880 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:32:51.880+0000 7f9f6881a640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-09T04:32:51.882 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:32:51.882+0000 7f9f6881a640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-05-09T04:32:54.847 DEBUG:teuthology.orchestra.run.smithi145:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-09T04:32:54.933 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-05-09T04:32:55.023 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-09T04:32:55.023 INFO:tasks.ceph:role: osd.4 2024-05-09T04:32:55.023 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi192.front.sepia.ceph.com 2024-05-09T04:32:55.023 DEBUG:teuthology.orchestra.run.smithi192:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-09T04:32:55.113 INFO:teuthology.orchestra.run.smithi192.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T04:32:55.113 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T04:32:55.113 INFO:teuthology.orchestra.run.smithi192.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T04:32:55.113 INFO:teuthology.orchestra.run.smithi192.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-09T04:32:55.113 INFO:teuthology.orchestra.run.smithi192.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T04:32:55.113 INFO:teuthology.orchestra.run.smithi192.stdout: = sunit=0 swidth=0 blks 2024-05-09T04:32:55.113 INFO:teuthology.orchestra.run.smithi192.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T04:32:55.113 INFO:teuthology.orchestra.run.smithi192.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-09T04:32:55.113 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T04:32:55.113 INFO:teuthology.orchestra.run.smithi192.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T04:32:55.517 INFO:teuthology.orchestra.run.smithi192.stdout:Discarding blocks...Done. 2024-05-09T04:32:55.523 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi192.front.sepia.ceph.com -o noatime 2024-05-09T04:32:55.523 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-05-09T04:32:55.680 DEBUG:teuthology.orchestra.run.smithi192:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-05-09T04:32:55.773 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-05-09T04:32:55.856 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-09T04:32:55.856 INFO:tasks.ceph:role: osd.5 2024-05-09T04:32:55.856 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi192.front.sepia.ceph.com 2024-05-09T04:32:55.856 DEBUG:teuthology.orchestra.run.smithi192:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-09T04:32:55.941 INFO:teuthology.orchestra.run.smithi192.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T04:32:55.942 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T04:32:55.942 INFO:teuthology.orchestra.run.smithi192.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T04:32:55.942 INFO:teuthology.orchestra.run.smithi192.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-09T04:32:55.942 INFO:teuthology.orchestra.run.smithi192.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T04:32:55.942 INFO:teuthology.orchestra.run.smithi192.stdout: = sunit=0 swidth=0 blks 2024-05-09T04:32:55.942 INFO:teuthology.orchestra.run.smithi192.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T04:32:55.942 INFO:teuthology.orchestra.run.smithi192.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-09T04:32:55.942 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T04:32:55.942 INFO:teuthology.orchestra.run.smithi192.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T04:32:56.299 INFO:teuthology.orchestra.run.smithi192.stdout:Discarding blocks...Done. 2024-05-09T04:32:56.305 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi192.front.sepia.ceph.com -o noatime 2024-05-09T04:32:56.306 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-05-09T04:32:56.421 DEBUG:teuthology.orchestra.run.smithi192:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-05-09T04:32:56.498 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-05-09T04:32:56.576 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-09T04:32:56.576 INFO:tasks.ceph:role: osd.6 2024-05-09T04:32:56.576 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi192.front.sepia.ceph.com 2024-05-09T04:32:56.576 DEBUG:teuthology.orchestra.run.smithi192:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-09T04:32:56.663 INFO:teuthology.orchestra.run.smithi192.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T04:32:56.663 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T04:32:56.663 INFO:teuthology.orchestra.run.smithi192.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T04:32:56.663 INFO:teuthology.orchestra.run.smithi192.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-09T04:32:56.663 INFO:teuthology.orchestra.run.smithi192.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T04:32:56.663 INFO:teuthology.orchestra.run.smithi192.stdout: = sunit=0 swidth=0 blks 2024-05-09T04:32:56.663 INFO:teuthology.orchestra.run.smithi192.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T04:32:56.663 INFO:teuthology.orchestra.run.smithi192.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-09T04:32:56.663 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T04:32:56.663 INFO:teuthology.orchestra.run.smithi192.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T04:32:57.051 INFO:teuthology.orchestra.run.smithi192.stdout:Discarding blocks...Done. 2024-05-09T04:32:57.059 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi192.front.sepia.ceph.com -o noatime 2024-05-09T04:32:57.059 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-05-09T04:32:57.204 DEBUG:teuthology.orchestra.run.smithi192:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-05-09T04:32:57.297 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-05-09T04:32:57.376 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-09T04:32:57.376 INFO:tasks.ceph:role: osd.7 2024-05-09T04:32:57.376 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi192.front.sepia.ceph.com 2024-05-09T04:32:57.377 DEBUG:teuthology.orchestra.run.smithi192:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-09T04:32:57.462 INFO:teuthology.orchestra.run.smithi192.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T04:32:57.463 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T04:32:57.463 INFO:teuthology.orchestra.run.smithi192.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T04:32:57.463 INFO:teuthology.orchestra.run.smithi192.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-09T04:32:57.463 INFO:teuthology.orchestra.run.smithi192.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T04:32:57.463 INFO:teuthology.orchestra.run.smithi192.stdout: = sunit=0 swidth=0 blks 2024-05-09T04:32:57.463 INFO:teuthology.orchestra.run.smithi192.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T04:32:57.463 INFO:teuthology.orchestra.run.smithi192.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-09T04:32:57.463 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T04:32:57.463 INFO:teuthology.orchestra.run.smithi192.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T04:32:57.873 INFO:teuthology.orchestra.run.smithi192.stdout:Discarding blocks...Done. 2024-05-09T04:32:57.880 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi192.front.sepia.ceph.com -o noatime 2024-05-09T04:32:57.880 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-05-09T04:32:58.033 DEBUG:teuthology.orchestra.run.smithi192:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-05-09T04:32:58.115 DEBUG:teuthology.orchestra.run.smithi192:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T04:32:58.262 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:32:58.261+0000 7fe86e41a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-05-09T04:32:58.263 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:32:58.261+0000 7fe86e41a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-05-09T04:32:58.263 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:32:58.261+0000 7fe86e41a640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-09T04:32:58.263 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:32:58.261+0000 7fe86e41a640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-09T04:32:58.263 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:32:58.261+0000 7fe86e41a640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-09T04:32:58.264 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:32:58.265+0000 7fe86e41a640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-05-09T04:33:01.230 DEBUG:teuthology.orchestra.run.smithi192:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-09T04:33:01.309 DEBUG:teuthology.orchestra.run.smithi192:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T04:33:01.438 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:01.438+0000 7f078961f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-05-09T04:33:01.438 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:01.439+0000 7f078961f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-05-09T04:33:01.439 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:01.439+0000 7f078961f640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-09T04:33:01.439 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:01.439+0000 7f078961f640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-09T04:33:01.439 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:01.439+0000 7f078961f640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-09T04:33:01.441 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:01.442+0000 7f078961f640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-05-09T04:33:03.304 DEBUG:teuthology.orchestra.run.smithi145:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T04:33:03.311 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T04:33:04.424 DEBUG:teuthology.orchestra.run.smithi192:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-09T04:33:04.516 DEBUG:teuthology.orchestra.run.smithi192:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T04:33:04.655 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:04.655+0000 7f5136815640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-05-09T04:33:04.655 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:04.655+0000 7f5136815640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-05-09T04:33:04.655 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:04.655+0000 7f5136815640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-09T04:33:04.655 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:04.655+0000 7f5136815640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-09T04:33:04.655 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:04.655+0000 7f5136815640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-09T04:33:04.657 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:04.658+0000 7f5136815640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-05-09T04:33:07.640 DEBUG:teuthology.orchestra.run.smithi192:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-05-09T04:33:07.728 DEBUG:teuthology.orchestra.run.smithi192:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T04:33:07.856 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:07.856+0000 7fd5b720d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-05-09T04:33:07.856 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:07.857+0000 7fd5b720d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-05-09T04:33:07.856 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:07.857+0000 7fd5b720d640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-09T04:33:07.856 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:07.857+0000 7fd5b720d640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-09T04:33:07.856 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:07.857+0000 7fd5b720d640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-09T04:33:07.858 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:07.859+0000 7fd5b720d640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-05-09T04:33:11.085 DEBUG:teuthology.orchestra.run.smithi192:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-05-09T04:33:11.170 INFO:tasks.ceph:Reading keys from all nodes... 2024-05-09T04:33:11.171 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-05-09T04:33:11.171 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-05-09T04:33:11.258 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-05-09T04:33:11.259 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-05-09T04:33:11.343 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-05-09T04:33:11.344 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-05-09T04:33:11.420 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-05-09T04:33:11.420 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-05-09T04:33:11.540 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-05-09T04:33:11.540 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-05-09T04:33:11.623 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-05-09T04:33:11.623 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-05-09T04:33:11.706 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-05-09T04:33:11.706 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-05-09T04:33:11.790 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-09T04:33:11.790 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-05-09T04:33:11.880 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-09T04:33:11.880 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-05-09T04:33:11.967 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-09T04:33:11.967 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-05-09T04:33:12.051 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-09T04:33:12.051 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-05-09T04:33:12.134 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-09T04:33:12.134 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-05-09T04:33:12.260 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-09T04:33:12.261 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-05-09T04:33:12.344 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-09T04:33:12.344 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-05-09T04:33:12.429 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-05-09T04:33:12.429 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-05-09T04:33:12.484 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-05-09T04:33:12.484 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-05-09T04:33:12.537 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-09T04:33:12.537 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-05-09T04:33:12.596 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-09T04:33:12.596 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-05-09T04:33:12.648 INFO:tasks.ceph:Adding keys to all mons... 2024-05-09T04:33:12.649 DEBUG:teuthology.orchestra.run.smithi145:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-09T04:33:12.657 DEBUG:teuthology.orchestra.run.smithi192:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-09T04:33:12.733 INFO:teuthology.orchestra.run.smithi145.stdout:[mgr.y] 2024-05-09T04:33:12.734 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQDkUTxmSBenNxAAns0hdQXzIWDm3C86Jrc8gw== 2024-05-09T04:33:12.735 INFO:teuthology.orchestra.run.smithi145.stdout:[mds.a] 2024-05-09T04:33:12.735 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQDlUTxmfnR3ERAAUsvH1ebffiEsH8dMX+Xe1w== 2024-05-09T04:33:12.735 INFO:teuthology.orchestra.run.smithi145.stdout:[mds.c] 2024-05-09T04:33:12.735 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQDlUTxm1//EHxAADR9WdmuEIMrbMzQDHi9ltQ== 2024-05-09T04:33:12.735 INFO:teuthology.orchestra.run.smithi145.stdout:[osd.0] 2024-05-09T04:33:12.735 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQDpUTxms65yMhAARwU3xBmDigriIR7IGY00iQ== 2024-05-09T04:33:12.735 INFO:teuthology.orchestra.run.smithi145.stdout:[osd.1] 2024-05-09T04:33:12.735 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQDtUTxmfJohERAAEVklN8QfeSfc8C8vZTlE2A== 2024-05-09T04:33:12.735 INFO:teuthology.orchestra.run.smithi145.stdout:[osd.2] 2024-05-09T04:33:12.735 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQDwUTxmFvcpKhAARtXZ1qSYuJp6gZCTAj0AiA== 2024-05-09T04:33:12.735 INFO:teuthology.orchestra.run.smithi145.stdout:[osd.3] 2024-05-09T04:33:12.735 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQDzUTxmqcp8NBAAWjK3tnEyljnJEu/wAx77TQ== 2024-05-09T04:33:12.735 INFO:teuthology.orchestra.run.smithi145.stdout:[mgr.x] 2024-05-09T04:33:12.735 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQDlUTxmYdonBxAA9AoBwsKJCkAOsuTcY2zXHQ== 2024-05-09T04:33:12.736 INFO:teuthology.orchestra.run.smithi145.stdout:[mds.b] 2024-05-09T04:33:12.736 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQDlUTxm02GmLhAAI7JaX2VaRh08cXbDLpBHDg== 2024-05-09T04:33:12.736 INFO:teuthology.orchestra.run.smithi145.stdout:[mds.d] 2024-05-09T04:33:12.736 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQDmUTxm5qxUARAA6NW1Rj3N4APnz2hYdHOpKw== 2024-05-09T04:33:12.736 INFO:teuthology.orchestra.run.smithi145.stdout:[osd.4] 2024-05-09T04:33:12.736 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQD6UTxmtvWdDxAAe9r1F/szPZid+E9/Vuio7Q== 2024-05-09T04:33:12.736 INFO:teuthology.orchestra.run.smithi145.stdout:[osd.5] 2024-05-09T04:33:12.736 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQD9UTxm0LE1GhAAIm98EcczdlqMnivEAjIsHQ== 2024-05-09T04:33:12.736 INFO:teuthology.orchestra.run.smithi145.stdout:[osd.6] 2024-05-09T04:33:12.736 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQAAUjxmSToZJxAAYIKW6x8/ARj4Nnz1tKgGQw== 2024-05-09T04:33:12.736 INFO:teuthology.orchestra.run.smithi145.stdout:[osd.7] 2024-05-09T04:33:12.736 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQADUjxmkEIcMxAAn5KshgT0Ugl/LXKcIssEcQ== 2024-05-09T04:33:12.736 INFO:teuthology.orchestra.run.smithi145.stdout:[client.0] 2024-05-09T04:33:12.736 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQDmUTxmfB7QDRAAs0TGpMOXJSerMXVrobw+hA== 2024-05-09T04:33:12.737 INFO:teuthology.orchestra.run.smithi145.stdout:[client.2] 2024-05-09T04:33:12.737 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQDmUTxmAn/iFhAAj7Wb0NYC8Z7KEMrsgrOkjA== 2024-05-09T04:33:12.737 INFO:teuthology.orchestra.run.smithi145.stdout:[client.1] 2024-05-09T04:33:12.737 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQDmUTxmZoCtIBAAwylggR+NZPk4qXCIbv9i9g== 2024-05-09T04:33:12.737 INFO:teuthology.orchestra.run.smithi145.stdout:[client.3] 2024-05-09T04:33:12.737 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQDmUTxmcUBaKhAA9XU8rgRmVEKjczR91U76hA== 2024-05-09T04:33:12.737 INFO:teuthology.orchestra.run.smithi192.stdout:[mgr.y] 2024-05-09T04:33:12.738 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQDkUTxmSBenNxAAns0hdQXzIWDm3C86Jrc8gw== 2024-05-09T04:33:12.738 INFO:teuthology.orchestra.run.smithi192.stdout:[mds.a] 2024-05-09T04:33:12.738 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQDlUTxmfnR3ERAAUsvH1ebffiEsH8dMX+Xe1w== 2024-05-09T04:33:12.738 INFO:teuthology.orchestra.run.smithi192.stdout:[mds.c] 2024-05-09T04:33:12.738 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQDlUTxm1//EHxAADR9WdmuEIMrbMzQDHi9ltQ== 2024-05-09T04:33:12.738 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.0] 2024-05-09T04:33:12.738 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQDpUTxms65yMhAARwU3xBmDigriIR7IGY00iQ== 2024-05-09T04:33:12.738 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.1] 2024-05-09T04:33:12.738 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQDtUTxmfJohERAAEVklN8QfeSfc8C8vZTlE2A== 2024-05-09T04:33:12.738 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.2] 2024-05-09T04:33:12.738 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQDwUTxmFvcpKhAARtXZ1qSYuJp6gZCTAj0AiA== 2024-05-09T04:33:12.738 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.3] 2024-05-09T04:33:12.739 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQDzUTxmqcp8NBAAWjK3tnEyljnJEu/wAx77TQ== 2024-05-09T04:33:12.739 INFO:teuthology.orchestra.run.smithi192.stdout:[mgr.x] 2024-05-09T04:33:12.739 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQDlUTxmYdonBxAA9AoBwsKJCkAOsuTcY2zXHQ== 2024-05-09T04:33:12.739 INFO:teuthology.orchestra.run.smithi192.stdout:[mds.b] 2024-05-09T04:33:12.739 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQDlUTxm02GmLhAAI7JaX2VaRh08cXbDLpBHDg== 2024-05-09T04:33:12.739 INFO:teuthology.orchestra.run.smithi192.stdout:[mds.d] 2024-05-09T04:33:12.739 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQDmUTxm5qxUARAA6NW1Rj3N4APnz2hYdHOpKw== 2024-05-09T04:33:12.739 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.4] 2024-05-09T04:33:12.739 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQD6UTxmtvWdDxAAe9r1F/szPZid+E9/Vuio7Q== 2024-05-09T04:33:12.739 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.5] 2024-05-09T04:33:12.739 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQD9UTxm0LE1GhAAIm98EcczdlqMnivEAjIsHQ== 2024-05-09T04:33:12.739 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.6] 2024-05-09T04:33:12.739 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQAAUjxmSToZJxAAYIKW6x8/ARj4Nnz1tKgGQw== 2024-05-09T04:33:12.739 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.7] 2024-05-09T04:33:12.739 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQADUjxmkEIcMxAAn5KshgT0Ugl/LXKcIssEcQ== 2024-05-09T04:33:12.739 INFO:teuthology.orchestra.run.smithi192.stdout:[client.0] 2024-05-09T04:33:12.740 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQDmUTxmfB7QDRAAs0TGpMOXJSerMXVrobw+hA== 2024-05-09T04:33:12.740 INFO:teuthology.orchestra.run.smithi192.stdout:[client.2] 2024-05-09T04:33:12.740 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQDmUTxmAn/iFhAAj7Wb0NYC8Z7KEMrsgrOkjA== 2024-05-09T04:33:12.740 INFO:teuthology.orchestra.run.smithi192.stdout:[client.1] 2024-05-09T04:33:12.740 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQDmUTxmZoCtIBAAwylggR+NZPk4qXCIbv9i9g== 2024-05-09T04:33:12.740 INFO:teuthology.orchestra.run.smithi192.stdout:[client.3] 2024-05-09T04:33:12.740 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQDmUTxmcUBaKhAA9XU8rgRmVEKjczR91U76hA== 2024-05-09T04:33:12.741 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-09T04:33:12.790 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-09T04:33:12.929 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T04:33:12.937 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T04:33:13.076 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T04:33:13.083 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T04:33:13.215 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T04:33:13.222 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T04:33:13.359 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T04:33:13.367 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T04:33:13.498 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T04:33:13.503 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T04:33:13.641 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T04:33:13.648 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T04:33:13.777 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-09T04:33:13.784 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-09T04:33:13.919 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T04:33:13.924 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T04:33:14.061 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T04:33:14.069 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T04:33:14.198 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T04:33:14.204 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T04:33:14.329 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T04:33:14.334 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T04:33:14.460 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T04:33:14.466 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T04:33:14.596 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T04:33:14.601 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T04:33:14.734 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-09T04:33:14.742 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-09T04:33:14.880 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-09T04:33:14.897 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-09T04:33:15.043 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-09T04:33:15.050 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-09T04:33:15.186 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-09T04:33:15.194 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-09T04:33:15.332 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-05-09T04:33:15.333 DEBUG:teuthology.orchestra.run.smithi145:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-05-09T04:33:15.410 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-09T04:33:15.551 DEBUG:teuthology.orchestra.run.smithi145:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-05-09T04:33:15.630 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-05-09T04:33:15.720 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-09T04:33:15.868 DEBUG:teuthology.orchestra.run.smithi192:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-05-09T04:33:15.947 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-05-09T04:33:16.029 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-09T04:33:16.176 DEBUG:teuthology.orchestra.run.smithi192:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-05-09T04:33:16.258 DEBUG:teuthology.orchestra.run.smithi145:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-09T04:33:16.266 DEBUG:teuthology.orchestra.run.smithi192:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-09T04:33:16.318 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-05-09T04:33:16.319 INFO:tasks.ceph.mon.a:Restarting daemon 2024-05-09T04:33:16.319 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-05-09T04:33:16.326 INFO:tasks.ceph.mon.a:Started 2024-05-09T04:33:16.326 INFO:tasks.ceph.mon.b:Restarting daemon 2024-05-09T04:33:16.326 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-05-09T04:33:16.331 INFO:tasks.ceph.mon.b:Started 2024-05-09T04:33:16.331 INFO:tasks.ceph.mon.c:Restarting daemon 2024-05-09T04:33:16.331 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-05-09T04:33:16.338 INFO:tasks.ceph.mon.c:Started 2024-05-09T04:33:16.338 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-05-09T04:33:16.338 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-05-09T04:33:16.338 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-05-09T04:33:16.343 INFO:tasks.ceph.mgr.y:Started 2024-05-09T04:33:16.343 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-05-09T04:33:16.343 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-05-09T04:33:16.348 INFO:tasks.ceph.mgr.x:Started 2024-05-09T04:33:16.349 INFO:tasks.ceph:Setting crush tunables to default 2024-05-09T04:33:16.349 DEBUG:teuthology.orchestra.run.smithi145:> sudo ceph --cluster ceph osd crush tunables default 2024-05-09T04:33:16.518 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:16.518+0000 7f0865600640 1 Processor -- start 2024-05-09T04:33:16.518 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:16.518+0000 7f0865600640 1 -- start start 2024-05-09T04:33:16.519 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:16.519+0000 7f0865600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f086015f4e0 0x7f086016a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:16.519 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:16.519+0000 7f0865600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08600cfda0 con 0x7f08600d3cd0 2024-05-09T04:33:16.519 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:16.519+0000 7f0865600640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08600d0080 con 0x7f086015f100 2024-05-09T04:33:16.519 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:16.519+0000 7f0865600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f08600b8e70 con 0x7f086015f4e0 2024-05-09T04:33:16.520 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:16.520+0000 7f0864c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f086015f4e0 0x7f086016a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:16.520 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:16.520+0000 7f085fe00640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f086015f100 0x7f08600d35c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.145:38866/0 (socket says 172.21.15.145:38866) 2024-05-09T04:33:16.520 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:16.520+0000 7f085f400640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f08600d3cd0 0x7f08600d1950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:51368/0 (socket says 172.21.15.145:51368) 2024-05-09T04:33:16.520 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:16.520+0000 7f085f400640 1 -- 172.21.15.145:0/3831199777 learned_addr learned my addr 172.21.15.145:0/3831199777 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:33:16.520 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:16.520+0000 7f0864c00640 1 -- 172.21.15.145:0/3831199777 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f086015f4e0 msgr2=0x7f086016a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-09T04:33:16.520 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:16.520+0000 7f0864c00640 1 -- 172.21.15.145:0/3831199777 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f086015f4e0 msgr2=0x7f086016a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T04:33:16.520 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:16.520+0000 7f0864c00640 1 --2- 172.21.15.145:0/3831199777 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f086015f4e0 0x7f086016a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T04:33:16.520 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:16.520+0000 7f0864c00640 1 --2- 172.21.15.145:0/3831199777 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f086015f4e0 0x7f086016a120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T04:33:16.721 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:16.722+0000 7f0864c00640 1 --2- 172.21.15.145:0/3831199777 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f086015f4e0 0x7f086016a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:16.722 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:16.723+0000 7f0864c00640 1 -- 172.21.15.145:0/3831199777 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f086015f4e0 msgr2=0x7f086016a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-09T04:33:16.722 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:16.723+0000 7f0864c00640 1 -- 172.21.15.145:0/3831199777 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f086015f4e0 msgr2=0x7f086016a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T04:33:16.722 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:16.723+0000 7f0864c00640 1 --2- 172.21.15.145:0/3831199777 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f086015f4e0 0x7f086016a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T04:33:16.722 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:16.723+0000 7f0864c00640 1 --2- 172.21.15.145:0/3831199777 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f086015f4e0 0x7f086016a120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T04:33:17.126 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:17.124+0000 7f0864c00640 1 --2- 172.21.15.145:0/3831199777 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f086015f4e0 0x7f086016a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:17.127 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:17.125+0000 7f0864c00640 1 -- 172.21.15.145:0/3831199777 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f086015f4e0 msgr2=0x7f086016a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-09T04:33:17.127 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:17.125+0000 7f0864c00640 1 -- 172.21.15.145:0/3831199777 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f086015f4e0 msgr2=0x7f086016a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T04:33:17.127 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:17.125+0000 7f0864c00640 1 --2- 172.21.15.145:0/3831199777 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f086015f4e0 0x7f086016a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T04:33:17.127 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:17.125+0000 7f0864c00640 1 --2- 172.21.15.145:0/3831199777 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f086015f4e0 0x7f086016a120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-09T04:33:17.927 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:17.928+0000 7f0864c00640 1 --2- 172.21.15.145:0/3831199777 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f086015f4e0 0x7f086016a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:17.927 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:17.928+0000 7f0864c00640 1 -- 172.21.15.145:0/3831199777 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f086015f4e0 msgr2=0x7f086016a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-09T04:33:17.927 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:17.928+0000 7f0864c00640 1 -- 172.21.15.145:0/3831199777 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f086015f4e0 msgr2=0x7f086016a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T04:33:17.927 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:17.928+0000 7f0864c00640 1 --2- 172.21.15.145:0/3831199777 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f086015f4e0 0x7f086016a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T04:33:17.927 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:17.928+0000 7f0864c00640 1 --2- 172.21.15.145:0/3831199777 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f086015f4e0 0x7f086016a120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-09T04:33:19.520 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:19.520+0000 7f085d600640 1 -- 172.21.15.145:0/3831199777 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f086015f4e0 msgr2=0x7f086016a120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T04:33:19.520 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:19.520+0000 7f085d600640 1 --2- 172.21.15.145:0/3831199777 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f086015f4e0 0x7f086016a120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:19.520 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:19.520+0000 7f085d600640 1 -- 172.21.15.145:0/3831199777 >> v1:172.21.15.192:6789/0 conn(0x7f086015f100 legacy=0x7f08600d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:19.520 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:19.520+0000 7f085d600640 1 -- 172.21.15.145:0/3831199777 >> v1:172.21.15.145:6789/0 conn(0x7f08600d3cd0 legacy=0x7f08600d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:19.520 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:19.521+0000 7f085d600640 1 --2- 172.21.15.145:0/3831199777 >> v2:172.21.15.145:3300/0 conn(0x7f0840003710 0x7f0840005b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:19.520 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:19.521+0000 7f085d600640 1 --2- 172.21.15.145:0/3831199777 >> v2:172.21.15.192:3300/0 conn(0x7f08400061c0 0x7f0840008680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:19.520 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:19.521+0000 7f085d600640 1 -- 172.21.15.145:0/3831199777 --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f08600b8e70 con 0x7f0840003710 2024-05-09T04:33:19.520 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:19.521+0000 7f085d600640 1 -- 172.21.15.145:0/3831199777 --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f0840008d30 con 0x7f08400061c0 2024-05-09T04:33:19.521 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:19.521+0000 7f085fe00640 1 --2- 172.21.15.145:0/3831199777 >> v2:172.21.15.145:3300/0 conn(0x7f0840003710 0x7f0840005b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:19.521 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:19.521+0000 7f085f400640 1 --2- 172.21.15.145:0/3831199777 >> v2:172.21.15.192:3300/0 conn(0x7f08400061c0 0x7f0840008680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:19.522 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:19.522+0000 7f085fe00640 1 -- 172.21.15.145:0/3831199777 >> v2:172.21.15.145:3300/0 conn(0x7f0840003710 msgr2=0x7f0840005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-09T04:33:19.522 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:19.522+0000 7f085fe00640 1 -- 172.21.15.145:0/3831199777 >> v2:172.21.15.145:3300/0 conn(0x7f0840003710 msgr2=0x7f0840005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T04:33:19.522 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:19.522+0000 7f085fe00640 1 --2- 172.21.15.145:0/3831199777 >> v2:172.21.15.145:3300/0 conn(0x7f0840003710 0x7f0840005b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T04:33:19.522 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:19.522+0000 7f085fe00640 1 --2- 172.21.15.145:0/3831199777 >> v2:172.21.15.145:3300/0 conn(0x7f0840003710 0x7f0840005b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T04:33:19.522 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:19.522+0000 7f085f400640 1 -- 172.21.15.145:0/3831199777 >> v2:172.21.15.192:3300/0 conn(0x7f08400061c0 msgr2=0x7f0840008680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-09T04:33:19.522 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:19.522+0000 7f085f400640 1 -- 172.21.15.145:0/3831199777 >> v2:172.21.15.192:3300/0 conn(0x7f08400061c0 msgr2=0x7f0840008680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T04:33:19.522 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:19.522+0000 7f085f400640 1 --2- 172.21.15.145:0/3831199777 >> v2:172.21.15.192:3300/0 conn(0x7f08400061c0 0x7f0840008680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T04:33:19.522 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:19.522+0000 7f085f400640 1 --2- 172.21.15.145:0/3831199777 >> v2:172.21.15.192:3300/0 conn(0x7f08400061c0 0x7f0840008680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T04:33:19.723 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:19.723+0000 7f085fe00640 1 --2- 172.21.15.145:0/3831199777 >> v2:172.21.15.145:3300/0 conn(0x7f0840003710 0x7f0840005b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:19.723 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:19.723+0000 7f085f400640 1 --2- 172.21.15.145:0/3831199777 >> v2:172.21.15.192:3300/0 conn(0x7f08400061c0 0x7f0840008680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:19.723 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:19.724+0000 7f085fe00640 1 -- 172.21.15.145:0/3831199777 >> v2:172.21.15.145:3300/0 conn(0x7f0840003710 msgr2=0x7f0840005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-09T04:33:19.723 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:19.724+0000 7f085fe00640 1 -- 172.21.15.145:0/3831199777 >> v2:172.21.15.145:3300/0 conn(0x7f0840003710 msgr2=0x7f0840005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T04:33:19.723 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:19.724+0000 7f085fe00640 1 --2- 172.21.15.145:0/3831199777 >> v2:172.21.15.145:3300/0 conn(0x7f0840003710 0x7f0840005b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T04:33:19.723 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:19.724+0000 7f085f400640 1 -- 172.21.15.145:0/3831199777 >> v2:172.21.15.192:3300/0 conn(0x7f08400061c0 msgr2=0x7f0840008680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-09T04:33:19.723 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:19.724+0000 7f085fe00640 1 --2- 172.21.15.145:0/3831199777 >> v2:172.21.15.145:3300/0 conn(0x7f0840003710 0x7f0840005b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T04:33:19.723 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:19.724+0000 7f085f400640 1 -- 172.21.15.145:0/3831199777 >> v2:172.21.15.192:3300/0 conn(0x7f08400061c0 msgr2=0x7f0840008680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T04:33:19.723 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:19.724+0000 7f085f400640 1 --2- 172.21.15.145:0/3831199777 >> v2:172.21.15.192:3300/0 conn(0x7f08400061c0 0x7f0840008680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T04:33:19.723 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:19.724+0000 7f085f400640 1 --2- 172.21.15.145:0/3831199777 >> v2:172.21.15.192:3300/0 conn(0x7f08400061c0 0x7f0840008680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T04:33:20.125 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:20.125+0000 7f085fe00640 1 --2- 172.21.15.145:0/3831199777 >> v2:172.21.15.145:3300/0 conn(0x7f0840003710 0x7f0840005b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:20.125 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:20.125+0000 7f085f400640 1 --2- 172.21.15.145:0/3831199777 >> v2:172.21.15.192:3300/0 conn(0x7f08400061c0 0x7f0840008680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:20.125 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:20.126+0000 7f085fe00640 1 -- 172.21.15.145:0/3831199777 >> v2:172.21.15.145:3300/0 conn(0x7f0840003710 msgr2=0x7f0840005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-09T04:33:20.125 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:20.126+0000 7f085fe00640 1 -- 172.21.15.145:0/3831199777 >> v2:172.21.15.145:3300/0 conn(0x7f0840003710 msgr2=0x7f0840005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T04:33:20.125 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:20.126+0000 7f085fe00640 1 --2- 172.21.15.145:0/3831199777 >> v2:172.21.15.145:3300/0 conn(0x7f0840003710 0x7f0840005b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T04:33:20.125 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:20.126+0000 7f085f400640 1 -- 172.21.15.145:0/3831199777 >> v2:172.21.15.192:3300/0 conn(0x7f08400061c0 msgr2=0x7f0840008680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-09T04:33:20.125 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:20.126+0000 7f085f400640 1 -- 172.21.15.145:0/3831199777 >> v2:172.21.15.192:3300/0 conn(0x7f08400061c0 msgr2=0x7f0840008680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T04:33:20.125 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:20.126+0000 7f085fe00640 1 --2- 172.21.15.145:0/3831199777 >> v2:172.21.15.145:3300/0 conn(0x7f0840003710 0x7f0840005b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-09T04:33:20.126 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:20.126+0000 7f085f400640 1 --2- 172.21.15.145:0/3831199777 >> v2:172.21.15.192:3300/0 conn(0x7f08400061c0 0x7f0840008680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T04:33:20.126 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:20.126+0000 7f085f400640 1 --2- 172.21.15.145:0/3831199777 >> v2:172.21.15.192:3300/0 conn(0x7f08400061c0 0x7f0840008680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-09T04:33:20.926 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:20.927+0000 7f085fe00640 1 --2- 172.21.15.145:0/3831199777 >> v2:172.21.15.145:3300/0 conn(0x7f0840003710 0x7f0840005b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:20.927 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:20.928+0000 7f085f400640 1 --2- 172.21.15.145:0/3831199777 >> v2:172.21.15.192:3300/0 conn(0x7f08400061c0 0x7f0840008680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:20.927 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:20.928+0000 7f085fe00640 1 -- 172.21.15.145:0/3831199777 >> v2:172.21.15.145:3300/0 conn(0x7f0840003710 msgr2=0x7f0840005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-09T04:33:20.927 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:20.928+0000 7f085fe00640 1 -- 172.21.15.145:0/3831199777 >> v2:172.21.15.145:3300/0 conn(0x7f0840003710 msgr2=0x7f0840005b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T04:33:20.928 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:20.928+0000 7f085fe00640 1 --2- 172.21.15.145:0/3831199777 >> v2:172.21.15.145:3300/0 conn(0x7f0840003710 0x7f0840005b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T04:33:20.928 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:20.928+0000 7f085f400640 1 -- 172.21.15.145:0/3831199777 >> v2:172.21.15.192:3300/0 conn(0x7f08400061c0 msgr2=0x7f0840008680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-09T04:33:20.928 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:20.928+0000 7f085f400640 1 -- 172.21.15.145:0/3831199777 >> v2:172.21.15.192:3300/0 conn(0x7f08400061c0 msgr2=0x7f0840008680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T04:33:20.928 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:20.928+0000 7f085f400640 1 --2- 172.21.15.145:0/3831199777 >> v2:172.21.15.192:3300/0 conn(0x7f08400061c0 0x7f0840008680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T04:33:20.928 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:20.928+0000 7f085fe00640 1 --2- 172.21.15.145:0/3831199777 >> v2:172.21.15.145:3300/0 conn(0x7f0840003710 0x7f0840005b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-09T04:33:20.928 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:20.928+0000 7f085f400640 1 --2- 172.21.15.145:0/3831199777 >> v2:172.21.15.192:3300/0 conn(0x7f08400061c0 0x7f0840008680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-09T04:33:22.520 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:22.521+0000 7f085d600640 1 -- 172.21.15.145:0/3831199777 >> v2:172.21.15.192:3300/0 conn(0x7f08400061c0 msgr2=0x7f0840008680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T04:33:22.520 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:22.521+0000 7f085d600640 1 --2- 172.21.15.145:0/3831199777 >> v2:172.21.15.192:3300/0 conn(0x7f08400061c0 0x7f0840008680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:22.520 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:22.521+0000 7f085d600640 1 -- 172.21.15.145:0/3831199777 >> v2:172.21.15.145:3300/0 conn(0x7f0840003710 msgr2=0x7f0840005b70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T04:33:22.521 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:22.521+0000 7f085d600640 1 --2- 172.21.15.145:0/3831199777 >> v2:172.21.15.145:3300/0 conn(0x7f0840003710 0x7f0840005b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:22.521 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:22.521+0000 7f085d600640 1 --2- 172.21.15.145:0/3831199777 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f086015f4e0 0x7f084000b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:22.521 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:22.521+0000 7f0864c00640 1 -- 172.21.15.145:0/3831199777 reap_dead start 2024-05-09T04:33:22.521 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:22.521+0000 7f085d600640 1 --2- 172.21.15.145:0/3831199777 >> v2:172.21.15.192:3300/0 conn(0x7f08400061c0 0x7f0840008680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:22.521 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:22.521+0000 7f085d600640 1 -- 172.21.15.145:0/3831199777 --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08600d0360 con 0x7f086015f100 2024-05-09T04:33:22.521 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:22.521+0000 7f085d600640 1 -- 172.21.15.145:0/3831199777 --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f08600b8e70 con 0x7f08400061c0 2024-05-09T04:33:22.521 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:22.521+0000 7f085d600640 1 -- 172.21.15.145:0/3831199777 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0840008d30 con 0x7f086015f4e0 2024-05-09T04:33:22.522 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:22.522+0000 7f0864c00640 1 --2- 172.21.15.145:0/3831199777 >> v2:172.21.15.192:3300/0 conn(0x7f08400061c0 0x7f0840008680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:22.522 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:22.522+0000 7f085fe00640 1 --2- 172.21.15.145:0/3831199777 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f086015f4e0 0x7f084000b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:22.522 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:22.523+0000 7f085fe00640 1 -- 172.21.15.145:0/3831199777 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f086015f4e0 msgr2=0x7f084000b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-09T04:33:22.522 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:22.523+0000 7f085fe00640 1 -- 172.21.15.145:0/3831199777 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f086015f4e0 msgr2=0x7f084000b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T04:33:22.522 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:22.523+0000 7f085fe00640 1 --2- 172.21.15.145:0/3831199777 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f086015f4e0 0x7f084000b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T04:33:22.522 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:22.523+0000 7f0864c00640 1 -- 172.21.15.145:0/3831199777 >> v2:172.21.15.192:3300/0 conn(0x7f08400061c0 msgr2=0x7f0840008680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-09T04:33:22.522 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:22.523+0000 7f0864c00640 1 -- 172.21.15.145:0/3831199777 >> v2:172.21.15.192:3300/0 conn(0x7f08400061c0 msgr2=0x7f0840008680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T04:33:22.522 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:22.523+0000 7f0864c00640 1 --2- 172.21.15.145:0/3831199777 >> v2:172.21.15.192:3300/0 conn(0x7f08400061c0 0x7f0840008680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T04:33:22.522 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:22.523+0000 7f0864c00640 1 --2- 172.21.15.145:0/3831199777 >> v2:172.21.15.192:3300/0 conn(0x7f08400061c0 0x7f0840008680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T04:33:22.522 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:22.523+0000 7f085fe00640 1 --2- 172.21.15.145:0/3831199777 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f086015f4e0 0x7f084000b130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T04:33:22.724 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:22.724+0000 7f0864c00640 1 --2- 172.21.15.145:0/3831199777 >> v2:172.21.15.192:3300/0 conn(0x7f08400061c0 0x7f0840008680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:22.724 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:22.725+0000 7f085fe00640 1 --2- 172.21.15.145:0/3831199777 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f086015f4e0 0x7f084000b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:22.724 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:22.725+0000 7f0864c00640 1 -- 172.21.15.145:0/3831199777 >> v2:172.21.15.192:3300/0 conn(0x7f08400061c0 msgr2=0x7f0840008680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-09T04:33:22.724 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:22.725+0000 7f0864c00640 1 -- 172.21.15.145:0/3831199777 >> v2:172.21.15.192:3300/0 conn(0x7f08400061c0 msgr2=0x7f0840008680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T04:33:22.724 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:22.725+0000 7f0864c00640 1 --2- 172.21.15.145:0/3831199777 >> v2:172.21.15.192:3300/0 conn(0x7f08400061c0 0x7f0840008680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T04:33:22.724 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:22.725+0000 7f085fe00640 1 -- 172.21.15.145:0/3831199777 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f086015f4e0 msgr2=0x7f084000b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-09T04:33:22.724 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:22.725+0000 7f0864c00640 1 --2- 172.21.15.145:0/3831199777 >> v2:172.21.15.192:3300/0 conn(0x7f08400061c0 0x7f0840008680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T04:33:22.724 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:22.725+0000 7f085fe00640 1 -- 172.21.15.145:0/3831199777 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f086015f4e0 msgr2=0x7f084000b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T04:33:22.724 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:22.725+0000 7f085fe00640 1 --2- 172.21.15.145:0/3831199777 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f086015f4e0 0x7f084000b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T04:33:22.724 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:22.725+0000 7f085fe00640 1 --2- 172.21.15.145:0/3831199777 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f086015f4e0 0x7f084000b130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T04:33:23.126 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:23.126+0000 7f0864c00640 1 --2- 172.21.15.145:0/3831199777 >> v2:172.21.15.192:3300/0 conn(0x7f08400061c0 0x7f0840008680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:23.126 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:23.127+0000 7f085fe00640 1 --2- 172.21.15.145:0/3831199777 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f086015f4e0 0x7f084000b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:23.126 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:23.127+0000 7f0864c00640 1 -- 172.21.15.145:0/3831199777 >> v2:172.21.15.192:3300/0 conn(0x7f08400061c0 msgr2=0x7f0840008680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-09T04:33:23.126 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:23.127+0000 7f0864c00640 1 -- 172.21.15.145:0/3831199777 >> v2:172.21.15.192:3300/0 conn(0x7f08400061c0 msgr2=0x7f0840008680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T04:33:23.126 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:23.127+0000 7f0864c00640 1 --2- 172.21.15.145:0/3831199777 >> v2:172.21.15.192:3300/0 conn(0x7f08400061c0 0x7f0840008680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T04:33:23.126 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:23.127+0000 7f0864c00640 1 --2- 172.21.15.145:0/3831199777 >> v2:172.21.15.192:3300/0 conn(0x7f08400061c0 0x7f0840008680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-09T04:33:23.126 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:23.127+0000 7f085fe00640 1 -- 172.21.15.145:0/3831199777 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f086015f4e0 msgr2=0x7f084000b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-09T04:33:23.127 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:23.127+0000 7f085fe00640 1 -- 172.21.15.145:0/3831199777 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f086015f4e0 msgr2=0x7f084000b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T04:33:23.127 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:23.127+0000 7f085fe00640 1 --2- 172.21.15.145:0/3831199777 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f086015f4e0 0x7f084000b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T04:33:23.127 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:23.127+0000 7f085fe00640 1 --2- 172.21.15.145:0/3831199777 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f086015f4e0 0x7f084000b130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-09T04:33:23.929 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:23.929+0000 7f0864c00640 1 --2- 172.21.15.145:0/3831199777 >> v2:172.21.15.192:3300/0 conn(0x7f08400061c0 0x7f0840008680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:23.929 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:23.929+0000 7f085fe00640 1 --2- 172.21.15.145:0/3831199777 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f086015f4e0 0x7f084000b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:23.929 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:23.929+0000 7f0864c00640 1 -- 172.21.15.145:0/3831199777 >> v2:172.21.15.192:3300/0 conn(0x7f08400061c0 msgr2=0x7f0840008680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-09T04:33:23.929 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:23.929+0000 7f0864c00640 1 -- 172.21.15.145:0/3831199777 >> v2:172.21.15.192:3300/0 conn(0x7f08400061c0 msgr2=0x7f0840008680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T04:33:23.929 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:23.929+0000 7f0864c00640 1 --2- 172.21.15.145:0/3831199777 >> v2:172.21.15.192:3300/0 conn(0x7f08400061c0 0x7f0840008680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T04:33:23.930 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:23.929+0000 7f0864c00640 1 --2- 172.21.15.145:0/3831199777 >> v2:172.21.15.192:3300/0 conn(0x7f08400061c0 0x7f0840008680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-09T04:33:23.930 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:23.929+0000 7f085fe00640 1 -- 172.21.15.145:0/3831199777 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f086015f4e0 msgr2=0x7f084000b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-09T04:33:23.930 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:23.929+0000 7f085fe00640 1 -- 172.21.15.145:0/3831199777 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f086015f4e0 msgr2=0x7f084000b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T04:33:23.930 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:23.929+0000 7f085fe00640 1 --2- 172.21.15.145:0/3831199777 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f086015f4e0 0x7f084000b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T04:33:23.930 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:23.929+0000 7f085fe00640 1 --2- 172.21.15.145:0/3831199777 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f086015f4e0 0x7f084000b130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-09T04:33:25.521 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:25.521+0000 7f085d600640 1 -- 172.21.15.145:0/3831199777 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f086015f4e0 msgr2=0x7f084000b130 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T04:33:25.521 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:25.521+0000 7f085d600640 1 --2- 172.21.15.145:0/3831199777 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f086015f4e0 0x7f084000b130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:25.521 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:25.521+0000 7f085d600640 1 -- 172.21.15.145:0/3831199777 >> v2:172.21.15.192:3300/0 conn(0x7f08400061c0 msgr2=0x7f0840008680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T04:33:25.521 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:25.521+0000 7f085d600640 1 --2- 172.21.15.145:0/3831199777 >> v2:172.21.15.192:3300/0 conn(0x7f08400061c0 0x7f0840008680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:25.522 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:25.521+0000 7f085d600640 1 -- 172.21.15.145:0/3831199777 >> v1:172.21.15.192:6789/0 conn(0x7f086015f100 legacy=0x7f084000d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:25.522 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:25.521+0000 7f085d600640 1 --2- 172.21.15.145:0/3831199777 >> v2:172.21.15.145:3300/0 conn(0x7f084000eef0 0x7f084000f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:25.522 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:25.522+0000 7f085d600640 1 -- 172.21.15.145:0/3831199777 --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0840003c00 con 0x7f084000f870 2024-05-09T04:33:25.522 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:25.522+0000 7f085d600640 1 -- 172.21.15.145:0/3831199777 --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f08600b8e70 con 0x7f084000eef0 2024-05-09T04:33:25.523 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:25.523+0000 7f085fe00640 1 --2- 172.21.15.145:0/3831199777 >> v2:172.21.15.145:3300/0 conn(0x7f084000eef0 0x7f084000f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:25.524 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:25.523+0000 7f085fe00640 1 -- 172.21.15.145:0/3831199777 >> v2:172.21.15.145:3300/0 conn(0x7f084000eef0 msgr2=0x7f084000f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-09T04:33:25.524 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:25.523+0000 7f085fe00640 1 -- 172.21.15.145:0/3831199777 >> v2:172.21.15.145:3300/0 conn(0x7f084000eef0 msgr2=0x7f084000f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T04:33:25.524 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:25.523+0000 7f085fe00640 1 --2- 172.21.15.145:0/3831199777 >> v2:172.21.15.145:3300/0 conn(0x7f084000eef0 0x7f084000f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T04:33:25.525 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:25.523+0000 7f085fe00640 1 --2- 172.21.15.145:0/3831199777 >> v2:172.21.15.145:3300/0 conn(0x7f084000eef0 0x7f084000f330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T04:33:25.723 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:25.724+0000 7f085fe00640 1 --2- 172.21.15.145:0/3831199777 >> v2:172.21.15.145:3300/0 conn(0x7f084000eef0 0x7f084000f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:25.730 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:25.725+0000 7f085fe00640 1 -- 172.21.15.145:0/3831199777 >> v2:172.21.15.145:3300/0 conn(0x7f084000eef0 msgr2=0x7f084000f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-09T04:33:25.730 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:25.725+0000 7f085fe00640 1 -- 172.21.15.145:0/3831199777 >> v2:172.21.15.145:3300/0 conn(0x7f084000eef0 msgr2=0x7f084000f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T04:33:25.730 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:25.725+0000 7f085fe00640 1 --2- 172.21.15.145:0/3831199777 >> v2:172.21.15.145:3300/0 conn(0x7f084000eef0 0x7f084000f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T04:33:25.730 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:25.725+0000 7f085fe00640 1 --2- 172.21.15.145:0/3831199777 >> v2:172.21.15.145:3300/0 conn(0x7f084000eef0 0x7f084000f330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T04:33:26.125 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.126+0000 7f085fe00640 1 --2- 172.21.15.145:0/3831199777 >> v2:172.21.15.145:3300/0 conn(0x7f084000eef0 0x7f084000f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:26.126 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.127+0000 7f085fe00640 1 -- 172.21.15.145:0/3831199777 >> v2:172.21.15.145:3300/0 conn(0x7f084000eef0 msgr2=0x7f084000f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-09T04:33:26.126 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.127+0000 7f085fe00640 1 -- 172.21.15.145:0/3831199777 >> v2:172.21.15.145:3300/0 conn(0x7f084000eef0 msgr2=0x7f084000f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T04:33:26.126 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.127+0000 7f085fe00640 1 --2- 172.21.15.145:0/3831199777 >> v2:172.21.15.145:3300/0 conn(0x7f084000eef0 0x7f084000f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T04:33:26.126 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.127+0000 7f085fe00640 1 --2- 172.21.15.145:0/3831199777 >> v2:172.21.15.145:3300/0 conn(0x7f084000eef0 0x7f084000f330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-09T04:33:26.579 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.580+0000 7f085ea00640 1 -- 172.21.15.145:0/3831199777 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1022310067 0 0) 0x7f0840003c00 con 0x7f084000f870 2024-05-09T04:33:26.579 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.580+0000 7f085ea00640 1 -- 172.21.15.145:0/3831199777 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0844003520 con 0x7f084000f870 2024-05-09T04:33:26.581 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.582+0000 7f085ea00640 1 -- 172.21.15.145:0/3831199777 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 544023100 0 0) 0x7f0844003520 con 0x7f084000f870 2024-05-09T04:33:26.581 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.582+0000 7f085ea00640 1 -- 172.21.15.145:0/3831199777 >> v2:172.21.15.145:3300/0 conn(0x7f084000eef0 msgr2=0x7f084000f330 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T04:33:26.581 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.582+0000 7f085ea00640 1 --2- 172.21.15.145:0/3831199777 >> v2:172.21.15.145:3300/0 conn(0x7f084000eef0 0x7f084000f330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:26.581 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.582+0000 7f085ea00640 1 -- 172.21.15.145:0/3831199777 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0840014610 con 0x7f084000f870 2024-05-09T04:33:26.582 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.583+0000 7f085ea00640 1 -- 172.21.15.145:0/3831199777 <== mon.0 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3474413261 0 0) 0x7f084c001460 con 0x7f084000f870 2024-05-09T04:33:26.582 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.583+0000 7f085ea00640 1 -- 172.21.15.145:0/3831199777 >> v1:172.21.15.145:6789/0 conn(0x7f084000f870 legacy=0x7f0840011ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:26.582 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.583+0000 7f085ea00640 1 --2- 172.21.15.145:0/3831199777 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f08440039b0 0x7f0844003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:26.582 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.584+0000 7f0864c00640 1 -- 172.21.15.145:0/3831199777 reap_dead start 2024-05-09T04:33:26.582 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.584+0000 7f085ea00640 1 --2- 172.21.15.145:0/3831199777 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f086015f4e0 0x7f0844004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:26.583 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.584+0000 7f085ea00640 1 --2- 172.21.15.145:0/3831199777 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0844004d90 0x7f0844005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:26.583 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.584+0000 7f085ea00640 1 -- 172.21.15.145:0/3831199777 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f08600b8e70 con 0x7f08440039b0 2024-05-09T04:33:26.583 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.584+0000 7f085ea00640 1 -- 172.21.15.145:0/3831199777 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f084400bc40 con 0x7f0844004d90 2024-05-09T04:33:26.583 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.584+0000 7f085ea00640 1 -- 172.21.15.145:0/3831199777 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f084400be40 con 0x7f086015f4e0 2024-05-09T04:33:26.583 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.584+0000 7f085fe00640 1 --2- 172.21.15.145:0/3831199777 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f08440039b0 0x7f0844003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:26.583 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.585+0000 7f085f400640 1 --2- 172.21.15.145:0/3831199777 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f086015f4e0 0x7f0844004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:26.584 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.585+0000 7f0864c00640 1 --2- 172.21.15.145:0/3831199777 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0844004d90 0x7f0844005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:26.584 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.586+0000 7f085fe00640 1 -- 172.21.15.145:0/3831199777 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f086015f4e0 msgr2=0x7f0844004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:26.584 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.586+0000 7f085fe00640 1 --2- 172.21.15.145:0/3831199777 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f086015f4e0 0x7f0844004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:26.585 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.586+0000 7f085fe00640 1 -- 172.21.15.145:0/3831199777 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0844004d90 msgr2=0x7f0844005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:26.585 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.586+0000 7f085fe00640 1 --2- 172.21.15.145:0/3831199777 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0844004d90 0x7f0844005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:26.585 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.586+0000 7f085fe00640 1 -- 172.21.15.145:0/3831199777 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0844003520 con 0x7f08440039b0 2024-05-09T04:33:26.587 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.589+0000 7f085fe00640 1 --2- 172.21.15.145:0/3831199777 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f08440039b0 0x7f0844003db0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0850007ca0 tx=0x7f0844005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85ee97f2cebb5885 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:26.589 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.590+0000 7f085ea00640 1 -- 172.21.15.145:0/3831199777 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0850039070 con 0x7f08440039b0 2024-05-09T04:33:26.589 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.590+0000 7f085ea00640 1 -- 172.21.15.145:0/3831199777 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f085002cb80 con 0x7f08440039b0 2024-05-09T04:33:26.589 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.590+0000 7f085ea00640 1 -- 172.21.15.145:0/3831199777 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0850005ca0 con 0x7f08440039b0 2024-05-09T04:33:26.591 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.591+0000 7f0865600640 1 -- 172.21.15.145:0/3831199777 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f08440039b0 msgr2=0x7f0844003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:26.591 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.592+0000 7f0865600640 1 --2- 172.21.15.145:0/3831199777 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f08440039b0 0x7f0844003db0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0850007ca0 tx=0x7f0844005830 comp rx=0 tx=0).stop 2024-05-09T04:33:26.591 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.592+0000 7f0865600640 1 -- 172.21.15.145:0/3831199777 shutdown_connections 2024-05-09T04:33:26.591 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.592+0000 7f0865600640 1 --2- 172.21.15.145:0/3831199777 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0844004d90 0x7f0844005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:26.591 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.592+0000 7f0865600640 1 --2- 172.21.15.145:0/3831199777 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f086015f4e0 0x7f0844004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:26.591 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.592+0000 7f0865600640 1 --2- 172.21.15.145:0/3831199777 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f08440039b0 0x7f0844003db0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:26.591 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.592+0000 7f0865600640 1 -- 172.21.15.145:0/3831199777 >> 172.21.15.145:0/3831199777 conn(0x7f086009e2e0 msgr2=0x7f08600d5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:26.592 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.592+0000 7f0865600640 1 -- 172.21.15.145:0/3831199777 shutdown_connections 2024-05-09T04:33:26.592 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.592+0000 7f0865600640 1 -- 172.21.15.145:0/3831199777 wait complete. 2024-05-09T04:33:26.595 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.597+0000 7f0865600640 1 Processor -- start 2024-05-09T04:33:26.596 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.597+0000 7f0865600640 1 -- start start 2024-05-09T04:33:26.597 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.598+0000 7f0865600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08440039b0 0x7f08601589c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:26.597 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.598+0000 7f0865600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0844004d90 0x7f0860161c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:26.597 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.598+0000 7f0865600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f086015f4e0 0x7f08601f4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:26.597 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.598+0000 7f0865600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f08600d0e40 con 0x7f086015f4e0 2024-05-09T04:33:26.598 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.598+0000 7f0865600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f08600d0920 con 0x7f08440039b0 2024-05-09T04:33:26.598 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.598+0000 7f0865600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f08601621d0 con 0x7f0844004d90 2024-05-09T04:33:26.598 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.598+0000 7f0864c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f086015f4e0 0x7f08601f4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:26.598 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.599+0000 7f085fe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08440039b0 0x7f08601589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:26.598 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.599+0000 7f0864c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f086015f4e0 0x7f08601f4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:52822/0 (socket says 172.21.15.145:52822) 2024-05-09T04:33:26.598 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.599+0000 7f0864c00640 1 -- 172.21.15.145:0/1822745203 learned_addr learned my addr 172.21.15.145:0/1822745203 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:26.598 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.599+0000 7f085f400640 1 --2- 172.21.15.145:0/1822745203 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0844004d90 0x7f0860161c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:26.599 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.599+0000 7f0864c00640 1 -- 172.21.15.145:0/1822745203 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0844004d90 msgr2=0x7f0860161c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:26.599 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.599+0000 7f0864c00640 1 --2- 172.21.15.145:0/1822745203 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0844004d90 0x7f0860161c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:26.599 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.599+0000 7f0864c00640 1 -- 172.21.15.145:0/1822745203 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08440039b0 msgr2=0x7f08601589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:26.599 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.599+0000 7f0864c00640 1 --2- 172.21.15.145:0/1822745203 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08440039b0 0x7f08601589c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:26.600 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.599+0000 7f0864c00640 1 -- 172.21.15.145:0/1822745203 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08600ccc70 con 0x7f086015f4e0 2024-05-09T04:33:26.600 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.600+0000 7f0864c00640 1 --2- 172.21.15.145:0/1822745203 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f086015f4e0 0x7f08601f4220 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f08600b6ad0 tx=0x7f0858019800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:26.600 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.600+0000 7f0857e00640 1 -- 172.21.15.145:0/1822745203 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0858009e50 con 0x7f086015f4e0 2024-05-09T04:33:26.600 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.600+0000 7f0857e00640 1 -- 172.21.15.145:0/1822745203 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0858018430 con 0x7f086015f4e0 2024-05-09T04:33:26.600 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.600+0000 7f0857e00640 1 -- 172.21.15.145:0/1822745203 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0858018820 con 0x7f086015f4e0 2024-05-09T04:33:26.600 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.600+0000 7f0865600640 1 -- 172.21.15.145:0/1822745203 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08600ccf50 con 0x7f086015f4e0 2024-05-09T04:33:26.600 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.600+0000 7f0865600640 1 -- 172.21.15.145:0/1822745203 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08601f4760 con 0x7f086015f4e0 2024-05-09T04:33:26.600 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.601+0000 7f0857e00640 1 -- 172.21.15.145:0/1822745203 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0858005d10 con 0x7f086015f4e0 2024-05-09T04:33:26.600 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.601+0000 7f0857e00640 1 -- 172.21.15.145:0/1822745203 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0858022700 con 0x7f086015f4e0 2024-05-09T04:33:26.600 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.602+0000 7f0865600640 1 -- 172.21.15.145:0/1822745203 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08600d4450 con 0x7f086015f4e0 2024-05-09T04:33:26.603 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.605+0000 7f0857e00640 1 -- 172.21.15.145:0/1822745203 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f08600d4450 con 0x7f086015f4e0 2024-05-09T04:33:26.677 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.679+0000 7f0865600640 1 -- 172.21.15.145:0/1822745203 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f086006a310 con 0x7f086015f4e0 2024-05-09T04:33:26.680 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.682+0000 7f0857e00640 1 -- 172.21.15.145:0/1822745203 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f0858029060 con 0x7f086015f4e0 2024-05-09T04:33:26.681 INFO:teuthology.orchestra.run.smithi145.stderr:adjusted tunables profile to default 2024-05-09T04:33:26.681 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.683+0000 7f0865600640 1 -- 172.21.15.145:0/1822745203 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f086015f4e0 msgr2=0x7f08601f4220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:26.681 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.683+0000 7f0865600640 1 --2- 172.21.15.145:0/1822745203 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f086015f4e0 0x7f08601f4220 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f08600b6ad0 tx=0x7f0858019800 comp rx=0 tx=0).stop 2024-05-09T04:33:26.682 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.683+0000 7f0865600640 1 -- 172.21.15.145:0/1822745203 shutdown_connections 2024-05-09T04:33:26.682 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.683+0000 7f0865600640 1 --2- 172.21.15.145:0/1822745203 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f086015f4e0 0x7f08601f4220 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:26.682 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.683+0000 7f0865600640 1 --2- 172.21.15.145:0/1822745203 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0844004d90 0x7f0860161c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:26.682 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.683+0000 7f0865600640 1 --2- 172.21.15.145:0/1822745203 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08440039b0 0x7f08601589c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:26.682 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.683+0000 7f0865600640 1 -- 172.21.15.145:0/1822745203 >> 172.21.15.145:0/1822745203 conn(0x7f086009e2e0 msgr2=0x7f08600b8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:26.682 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.683+0000 7f0865600640 1 -- 172.21.15.145:0/1822745203 shutdown_connections 2024-05-09T04:33:26.682 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:26.683+0000 7f0865600640 1 -- 172.21.15.145:0/1822745203 wait complete. 2024-05-09T04:33:26.700 INFO:tasks.ceph:check_enable_crimson: False 2024-05-09T04:33:26.700 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-05-09T04:33:26.701 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-05-09T04:33:26.701 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-05-09T04:33:26.741 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-09T04:33:26.739+0000 7f2e3f522040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T04:33:26.742 INFO:tasks.ceph.mgr.y.smithi145.stderr:2024-05-09T04:33:26.741+0000 7f5551720040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T04:33:26.772 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-05-09T04:33:26.772 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-05-09T04:33:26.887 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-05-09T04:33:26.887 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-05-09T04:33:26.924 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-09T04:33:26.924+0000 7f2e3f522040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T04:33:26.936 INFO:tasks.ceph.mgr.y.smithi145.stderr:2024-05-09T04:33:26.937+0000 7f5551720040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T04:33:26.984 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-09T04:33:26.985+0000 7f2e3f522040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T04:33:27.005 INFO:tasks.ceph.mgr.y.smithi145.stderr:2024-05-09T04:33:27.006+0000 7f5551720040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T04:33:27.011 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-05-09T04:33:27.011 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-05-09T04:33:27.127 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-09T04:33:27.128 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-05-09T04:33:27.205 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-09T04:33:27.205 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-05-09T04:33:27.277 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-09T04:33:27.277 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-05-09T04:33:27.278 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-09T04:33:27.278+0000 7f2e3f522040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T04:33:27.303 INFO:tasks.ceph.mgr.y.smithi145.stderr:2024-05-09T04:33:27.303+0000 7f5551720040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T04:33:27.334 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-09T04:33:27.335+0000 7f2e3f522040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T04:33:27.350 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-09T04:33:27.350 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-05-09T04:33:27.356 INFO:tasks.ceph.mgr.y.smithi145.stderr:2024-05-09T04:33:27.356+0000 7f5551720040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T04:33:27.466 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new 22e550ad-847b-4952-80e6-63d859ed1a0f 0 2024-05-09T04:33:27.644 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.645+0000 7f9b1be00640 1 Processor -- start 2024-05-09T04:33:27.644 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.645+0000 7f9b1be00640 1 -- start start 2024-05-09T04:33:27.646 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.646+0000 7f9b1be00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f9b1c167fc0 0x7f9b1c16a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:27.646 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.646+0000 7f9b1be00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b1c160380 con 0x7f9b1c06c480 2024-05-09T04:33:27.646 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.646+0000 7f9b1be00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b1c06f370 con 0x7f9b1c15f650 2024-05-09T04:33:27.647 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.646+0000 7f9b1be00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f9b1c06f9c0 con 0x7f9b1c167fc0 2024-05-09T04:33:27.647 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.647+0000 7f9b1a000640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f9b1c15f650 0x7f9b1c15fb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.192:60272/0 (socket says 172.21.15.192:60272) 2024-05-09T04:33:27.647 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.647+0000 7f9b1a000640 1 -- 172.21.15.192:0/3643690810 learned_addr learned my addr 172.21.15.192:0/3643690810 (peer_addr_for_me v1:172.21.15.192:0/0) 2024-05-09T04:33:27.647 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.647+0000 7f9b1b400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f9b1c167fc0 0x7f9b1c16a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:27.647 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.647+0000 7f9b1b400640 1 -- 172.21.15.192:0/3643690810 >> v1:172.21.15.192:6789/0 conn(0x7f9b1c15f650 legacy=0x7f9b1c15fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:27.647 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.647+0000 7f9b1b400640 1 -- 172.21.15.192:0/3643690810 >> v1:172.21.15.145:6789/0 conn(0x7f9b1c06c480 legacy=0x7f9b1c06c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:27.647 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.647+0000 7f9b1b400640 1 -- 172.21.15.192:0/3643690810 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b1c06f650 con 0x7f9b1c167fc0 2024-05-09T04:33:27.648 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.648+0000 7f9b1b400640 1 --2- 172.21.15.192:0/3643690810 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b1c167fc0 0x7f9b1c16a3b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c06fb40 tx=0x7f9b1402a640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b09d30937d4e893 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:27.648 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.648+0000 7f9b19600640 1 -- 172.21.15.192:0/3643690810 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b1402ae60 con 0x7f9b1c167fc0 2024-05-09T04:33:27.648 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.648+0000 7f9b19600640 1 -- 172.21.15.192:0/3643690810 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b14007740 con 0x7f9b1c167fc0 2024-05-09T04:33:27.648 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.648+0000 7f9b19600640 1 -- 172.21.15.192:0/3643690810 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b14007b10 con 0x7f9b1c167fc0 2024-05-09T04:33:27.649 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.649+0000 7f9b1be00640 1 -- 172.21.15.192:0/3643690810 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b1c167fc0 msgr2=0x7f9b1c16a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:27.649 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.649+0000 7f9b1be00640 1 --2- 172.21.15.192:0/3643690810 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b1c167fc0 0x7f9b1c16a3b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c06fb40 tx=0x7f9b1402a640 comp rx=0 tx=0).stop 2024-05-09T04:33:27.649 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.649+0000 7f9b1be00640 1 -- 172.21.15.192:0/3643690810 shutdown_connections 2024-05-09T04:33:27.649 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.649+0000 7f9b1be00640 1 --2- 172.21.15.192:0/3643690810 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b1c167fc0 0x7f9b1c16a3b0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:27.649 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.649+0000 7f9b1be00640 1 -- 172.21.15.192:0/3643690810 >> 172.21.15.192:0/3643690810 conn(0x7f9b1c06d590 msgr2=0x7f9b1c06d970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:27.649 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.649+0000 7f9b1be00640 1 -- 172.21.15.192:0/3643690810 shutdown_connections 2024-05-09T04:33:27.649 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.649+0000 7f9b1be00640 1 -- 172.21.15.192:0/3643690810 wait complete. 2024-05-09T04:33:27.650 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.651+0000 7f9b1be00640 1 Processor -- start 2024-05-09T04:33:27.650 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.651+0000 7f9b1be00640 1 -- start start 2024-05-09T04:33:27.652 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.652+0000 7f9b1be00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9b1c06c480 0x7f9b1c157240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:27.652 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.652+0000 7f9b1be00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9b1c15f650 0x7f9b1c141180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:27.652 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.652+0000 7f9b1be00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b1c167fc0 0x7f9b1c1416c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:27.652 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.652+0000 7f9b1be00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1c070e60 con 0x7f9b1c15f650 2024-05-09T04:33:27.652 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.652+0000 7f9b1be00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1c148c80 con 0x7f9b1c167fc0 2024-05-09T04:33:27.652 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.652+0000 7f9b1be00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9b1c070fe0 con 0x7f9b1c06c480 2024-05-09T04:33:27.652 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.652+0000 7f9b1b400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b1c167fc0 0x7f9b1c1416c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:27.652 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.652+0000 7f9b1aa00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9b1c06c480 0x7f9b1c157240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:27.652 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.652+0000 7f9b1b400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b1c167fc0 0x7f9b1c1416c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:39780/0 (socket says 172.21.15.192:39780) 2024-05-09T04:33:27.652 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.652+0000 7f9b1b400640 1 -- 172.21.15.192:0/4204603241 learned_addr learned my addr 172.21.15.192:0/4204603241 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-05-09T04:33:27.652 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.652+0000 7f9b1a000640 1 --2- 172.21.15.192:0/4204603241 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9b1c15f650 0x7f9b1c141180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:27.652 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.652+0000 7f9b1b400640 1 -- 172.21.15.192:0/4204603241 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9b1c06c480 msgr2=0x7f9b1c157240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:27.653 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.652+0000 7f9b1b400640 1 --2- 172.21.15.192:0/4204603241 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9b1c06c480 0x7f9b1c157240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:27.653 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.652+0000 7f9b1b400640 1 -- 172.21.15.192:0/4204603241 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9b1c15f650 msgr2=0x7f9b1c141180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:27.653 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.652+0000 7f9b1b400640 1 --2- 172.21.15.192:0/4204603241 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9b1c15f650 0x7f9b1c141180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:27.653 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.652+0000 7f9b1b400640 1 -- 172.21.15.192:0/4204603241 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b1c157c20 con 0x7f9b1c167fc0 2024-05-09T04:33:27.653 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.653+0000 7f9b1b400640 1 --2- 172.21.15.192:0/4204603241 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b1c167fc0 0x7f9b1c1416c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9b1402ac30 tx=0x7f9b14004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:27.654 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.653+0000 7f9b12a00640 1 -- 172.21.15.192:0/4204603241 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b1403f070 con 0x7f9b1c167fc0 2024-05-09T04:33:27.654 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.653+0000 7f9b12a00640 1 -- 172.21.15.192:0/4204603241 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b1402ae60 con 0x7f9b1c167fc0 2024-05-09T04:33:27.654 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.653+0000 7f9b12a00640 1 -- 172.21.15.192:0/4204603241 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b14044460 con 0x7f9b1c167fc0 2024-05-09T04:33:27.654 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.653+0000 7f9b1be00640 1 -- 172.21.15.192:0/4204603241 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b1c13e020 con 0x7f9b1c167fc0 2024-05-09T04:33:27.654 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.653+0000 7f9b1be00640 1 -- 172.21.15.192:0/4204603241 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b1c13e5e0 con 0x7f9b1c167fc0 2024-05-09T04:33:27.654 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.653+0000 7f9b12a00640 1 -- 172.21.15.192:0/4204603241 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9b14044600 con 0x7f9b1c167fc0 2024-05-09T04:33:27.654 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.653+0000 7f9b12a00640 1 -- 172.21.15.192:0/4204603241 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f9b1403b5e0 con 0x7f9b1c167fc0 2024-05-09T04:33:27.654 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.654+0000 7f9b1be00640 1 -- 172.21.15.192:0/4204603241 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b1c0694e0 con 0x7f9b1c167fc0 2024-05-09T04:33:27.658 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.659+0000 7f9b12a00640 1 -- 172.21.15.192:0/4204603241 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f9b14053080 con 0x7f9b1c167fc0 2024-05-09T04:33:27.737 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.738+0000 7f9b1be00640 1 -- 172.21.15.192:0/4204603241 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd new", "uuid": "22e550ad-847b-4952-80e6-63d859ed1a0f", "id": 0} v 0) -- 0x7f9b1c073b50 con 0x7f9b1c167fc0 2024-05-09T04:33:27.744 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.745+0000 7f9b12a00640 1 -- 172.21.15.192:0/4204603241 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "22e550ad-847b-4952-80e6-63d859ed1a0f", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f9b14044a80 con 0x7f9b1c167fc0 2024-05-09T04:33:27.744 INFO:teuthology.orchestra.run.smithi192.stdout:0 2024-05-09T04:33:27.745 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.746+0000 7f9b1be00640 1 -- 172.21.15.192:0/4204603241 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b1c167fc0 msgr2=0x7f9b1c1416c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:27.746 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.746+0000 7f9b1be00640 1 --2- 172.21.15.192:0/4204603241 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b1c167fc0 0x7f9b1c1416c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9b1402ac30 tx=0x7f9b14004a40 comp rx=0 tx=0).stop 2024-05-09T04:33:27.746 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.746+0000 7f9b1be00640 1 -- 172.21.15.192:0/4204603241 shutdown_connections 2024-05-09T04:33:27.746 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.746+0000 7f9b1be00640 1 --2- 172.21.15.192:0/4204603241 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b1c167fc0 0x7f9b1c1416c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:27.746 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.746+0000 7f9b1be00640 1 --2- 172.21.15.192:0/4204603241 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9b1c15f650 0x7f9b1c141180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:27.746 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.746+0000 7f9b1be00640 1 --2- 172.21.15.192:0/4204603241 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9b1c06c480 0x7f9b1c157240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:27.746 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.746+0000 7f9b1be00640 1 -- 172.21.15.192:0/4204603241 >> 172.21.15.192:0/4204603241 conn(0x7f9b1c06d590 msgr2=0x7f9b1c0b2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:27.746 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.747+0000 7f9b1be00640 1 -- 172.21.15.192:0/4204603241 shutdown_connections 2024-05-09T04:33:27.747 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.747+0000 7f9b1be00640 1 -- 172.21.15.192:0/4204603241 wait complete. 2024-05-09T04:33:27.762 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new 0f6b9fca-0c41-4145-8d74-598ddb42f2a3 1 2024-05-09T04:33:27.929 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.929+0000 7f50a2000640 1 Processor -- start 2024-05-09T04:33:27.929 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.929+0000 7f50a2000640 1 -- start start 2024-05-09T04:33:27.931 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.931+0000 7f50a2000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f509c15f3c0 0x7f509c0cf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:27.931 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.931+0000 7f50a2000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f509c0cedf0 0x7f509c0cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:27.931 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.931+0000 7f50a2000640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f509c0d2010 con 0x7f509c0d1b50 2024-05-09T04:33:27.931 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.931+0000 7f50a2000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f509c0cd390 con 0x7f509c15f3c0 2024-05-09T04:33:27.931 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.931+0000 7f50a2000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f509c0cd950 con 0x7f509c0cedf0 2024-05-09T04:33:27.931 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.931+0000 7f509be00640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f509c0d1b50 0x7f509c06d930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.192:60276/0 (socket says 172.21.15.192:60276) 2024-05-09T04:33:27.931 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.931+0000 7f509be00640 1 -- 172.21.15.192:0/2718089946 learned_addr learned my addr 172.21.15.192:0/2718089946 (peer_addr_for_me v1:172.21.15.192:0/0) 2024-05-09T04:33:27.931 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.931+0000 7f50a1600640 1 --2- 172.21.15.192:0/2718089946 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f509c0cedf0 0x7f509c0cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:27.932 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.931+0000 7f50a0c00640 1 --2- 172.21.15.192:0/2718089946 >> v2:172.21.15.145:3300/0 conn(0x7f509c15f3c0 0x7f509c0cf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:27.932 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.931+0000 7f509b400640 1 -- 172.21.15.192:0/2718089946 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3588535119 0 0) 0x7f509c0d2010 con 0x7f509c0d1b50 2024-05-09T04:33:27.932 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.931+0000 7f509b400640 1 -- 172.21.15.192:0/2718089946 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5084003520 con 0x7f509c0d1b50 2024-05-09T04:33:27.932 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.932+0000 7f50a1600640 1 -- 172.21.15.192:0/2718089946 >> v2:172.21.15.145:3300/0 conn(0x7f509c15f3c0 msgr2=0x7f509c0cf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:27.932 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.932+0000 7f50a1600640 1 --2- 172.21.15.192:0/2718089946 >> v2:172.21.15.145:3300/0 conn(0x7f509c15f3c0 0x7f509c0cf5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:27.932 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.932+0000 7f50a1600640 1 -- 172.21.15.192:0/2718089946 >> v1:172.21.15.192:6789/0 conn(0x7f509c0d1b50 legacy=0x7f509c06d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:27.932 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.932+0000 7f50a1600640 1 -- 172.21.15.192:0/2718089946 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f509c0d22f0 con 0x7f509c0cedf0 2024-05-09T04:33:27.933 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.933+0000 7f50a1600640 1 --2- 172.21.15.192:0/2718089946 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f509c0cedf0 0x7f509c0cfb00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5094009d90 tx=0x7f509402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6be6e5d2d85b1eeb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:27.933 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.933+0000 7f509b400640 1 -- 172.21.15.192:0/2718089946 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5094007740 con 0x7f509c0cedf0 2024-05-09T04:33:27.933 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.933+0000 7f509b400640 1 -- 172.21.15.192:0/2718089946 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50940078e0 con 0x7f509c0cedf0 2024-05-09T04:33:27.934 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.933+0000 7f509b400640 1 -- 172.21.15.192:0/2718089946 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5094007cb0 con 0x7f509c0cedf0 2024-05-09T04:33:27.934 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.933+0000 7f50a2000640 1 -- 172.21.15.192:0/2718089946 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f509c0cedf0 msgr2=0x7f509c0cfb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:27.934 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.933+0000 7f50a2000640 1 --2- 172.21.15.192:0/2718089946 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f509c0cedf0 0x7f509c0cfb00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5094009d90 tx=0x7f509402a660 comp rx=0 tx=0).stop 2024-05-09T04:33:27.934 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.933+0000 7f50a2000640 1 -- 172.21.15.192:0/2718089946 shutdown_connections 2024-05-09T04:33:27.934 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.933+0000 7f50a2000640 1 --2- 172.21.15.192:0/2718089946 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f509c0cedf0 0x7f509c0cfb00 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:27.934 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.933+0000 7f50a2000640 1 --2- 172.21.15.192:0/2718089946 >> v2:172.21.15.145:3300/0 conn(0x7f509c15f3c0 0x7f509c0cf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:27.934 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.933+0000 7f50a2000640 1 -- 172.21.15.192:0/2718089946 >> 172.21.15.192:0/2718089946 conn(0x7f509c0d2d40 msgr2=0x7f509c0d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:27.934 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.934+0000 7f50a2000640 1 -- 172.21.15.192:0/2718089946 shutdown_connections 2024-05-09T04:33:27.934 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.934+0000 7f50a2000640 1 -- 172.21.15.192:0/2718089946 wait complete. 2024-05-09T04:33:27.935 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.935+0000 7f50a2000640 1 Processor -- start 2024-05-09T04:33:27.935 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.935+0000 7f50a2000640 1 -- start start 2024-05-09T04:33:27.937 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.936+0000 7f50a2000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f509c0cedf0 0x7f509c1e68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:27.937 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.936+0000 7f50a2000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f509c0d1b50 0x7f509c1e4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:27.937 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.936+0000 7f50a2000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f509c15f3c0 0x7f509c14fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:27.937 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.936+0000 7f50a2000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f509c15f920 con 0x7f509c0d1b50 2024-05-09T04:33:27.937 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.936+0000 7f50a2000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f509c15f7a0 con 0x7f509c0cedf0 2024-05-09T04:33:27.938 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.936+0000 7f50a2000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f509c06f950 con 0x7f509c15f3c0 2024-05-09T04:33:27.938 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.936+0000 7f50a0c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f509c0cedf0 0x7f509c1e68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:27.938 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.936+0000 7f50a0c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f509c0cedf0 0x7f509c1e68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:39794/0 (socket says 172.21.15.192:39794) 2024-05-09T04:33:27.938 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.936+0000 7f50a0c00640 1 -- 172.21.15.192:0/2963504198 learned_addr learned my addr 172.21.15.192:0/2963504198 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-05-09T04:33:27.938 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.937+0000 7f50a1600640 1 --2- 172.21.15.192:0/2963504198 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f509c15f3c0 0x7f509c14fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:27.938 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.937+0000 7f509be00640 1 --2- 172.21.15.192:0/2963504198 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f509c0d1b50 0x7f509c1e4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:27.938 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.937+0000 7f50a0c00640 1 -- 172.21.15.192:0/2963504198 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f509c15f3c0 msgr2=0x7f509c14fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:27.938 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.937+0000 7f50a0c00640 1 --2- 172.21.15.192:0/2963504198 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f509c15f3c0 0x7f509c14fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:27.938 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.937+0000 7f50a0c00640 1 -- 172.21.15.192:0/2963504198 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f509c0d1b50 msgr2=0x7f509c1e4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:27.938 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.937+0000 7f50a0c00640 1 --2- 172.21.15.192:0/2963504198 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f509c0d1b50 0x7f509c1e4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:27.938 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.937+0000 7f50a0c00640 1 -- 172.21.15.192:0/2963504198 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f509c1e71d0 con 0x7f509c0cedf0 2024-05-09T04:33:27.938 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.937+0000 7f509be00640 1 --2- 172.21.15.192:0/2963504198 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f509c0d1b50 0x7f509c1e4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:33:27.938 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.937+0000 7f50a0c00640 1 --2- 172.21.15.192:0/2963504198 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f509c0cedf0 0x7f509c1e68f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f509000e9b0 tx=0x7f509000ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:27.938 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.937+0000 7f5098c00640 1 -- 172.21.15.192:0/2963504198 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5090017070 con 0x7f509c0cedf0 2024-05-09T04:33:27.939 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.937+0000 7f5098c00640 1 -- 172.21.15.192:0/2963504198 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f509000c9a0 con 0x7f509c0cedf0 2024-05-09T04:33:27.939 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.937+0000 7f5098c00640 1 -- 172.21.15.192:0/2963504198 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f509000cd90 con 0x7f509c0cedf0 2024-05-09T04:33:27.939 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.937+0000 7f50a2000640 1 -- 172.21.15.192:0/2963504198 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f509c1e74b0 con 0x7f509c0cedf0 2024-05-09T04:33:27.939 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.938+0000 7f50a2000640 1 -- 172.21.15.192:0/2963504198 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f509c1e7a70 con 0x7f509c0cedf0 2024-05-09T04:33:27.939 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.938+0000 7f5098c00640 1 -- 172.21.15.192:0/2963504198 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f509001a070 con 0x7f509c0cedf0 2024-05-09T04:33:27.939 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.938+0000 7f5098c00640 1 -- 172.21.15.192:0/2963504198 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f509001c980 con 0x7f509c0cedf0 2024-05-09T04:33:27.939 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.939+0000 7f50a2000640 1 -- 172.21.15.192:0/2963504198 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f509c0694e0 con 0x7f509c0cedf0 2024-05-09T04:33:27.941 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:27.942+0000 7f5098c00640 1 -- 172.21.15.192:0/2963504198 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f509001cb80 con 0x7f509c0cedf0 2024-05-09T04:33:28.019 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.019+0000 7f50a2000640 1 -- 172.21.15.192:0/2963504198 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd new", "uuid": "0f6b9fca-0c41-4145-8d74-598ddb42f2a3", "id": 1} v 0) -- 0x7f509c0c9ad0 con 0x7f509c0cedf0 2024-05-09T04:33:28.026 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.027+0000 7f5098c00640 1 -- 172.21.15.192:0/2963504198 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0f6b9fca-0c41-4145-8d74-598ddb42f2a3", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f5090005990 con 0x7f509c0cedf0 2024-05-09T04:33:28.026 INFO:teuthology.orchestra.run.smithi192.stdout:1 2024-05-09T04:33:28.027 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.028+0000 7f50a2000640 1 -- 172.21.15.192:0/2963504198 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f509c0cedf0 msgr2=0x7f509c1e68f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:28.027 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.028+0000 7f50a2000640 1 --2- 172.21.15.192:0/2963504198 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f509c0cedf0 0x7f509c1e68f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f509000e9b0 tx=0x7f509000ee70 comp rx=0 tx=0).stop 2024-05-09T04:33:28.027 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.028+0000 7f50a2000640 1 -- 172.21.15.192:0/2963504198 shutdown_connections 2024-05-09T04:33:28.027 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.028+0000 7f50a2000640 1 --2- 172.21.15.192:0/2963504198 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f509c15f3c0 0x7f509c14fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:28.027 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.028+0000 7f50a2000640 1 --2- 172.21.15.192:0/2963504198 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f509c0d1b50 0x7f509c1e4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:28.028 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.028+0000 7f50a2000640 1 --2- 172.21.15.192:0/2963504198 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f509c0cedf0 0x7f509c1e68f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:28.028 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.028+0000 7f50a2000640 1 -- 172.21.15.192:0/2963504198 >> 172.21.15.192:0/2963504198 conn(0x7f509c0d2d40 msgr2=0x7f509c06d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:28.028 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.028+0000 7f50a2000640 1 -- 172.21.15.192:0/2963504198 shutdown_connections 2024-05-09T04:33:28.028 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.028+0000 7f50a2000640 1 -- 172.21.15.192:0/2963504198 wait complete. 2024-05-09T04:33:28.043 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new b3b1f67e-5790-483f-8610-e05a5bce4c30 2 2024-05-09T04:33:28.224 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.225+0000 7fc8b9a00640 1 Processor -- start 2024-05-09T04:33:28.224 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.225+0000 7fc8b9a00640 1 -- start start 2024-05-09T04:33:28.226 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.227+0000 7fc8b9a00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc8b415d0b0 0x7fc8b40b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:28.226 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.227+0000 7fc8b9a00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8b406fcf0 con 0x7fc8b40add40 2024-05-09T04:33:28.226 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.227+0000 7fc8b9a00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8b406ff90 con 0x7fc8b40cfef0 2024-05-09T04:33:28.226 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.227+0000 7fc8b9a00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fc8b4072850 con 0x7fc8b415d0b0 2024-05-09T04:33:28.227 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.227+0000 7fc8b3e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc8b415d0b0 0x7fc8b40b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:28.227 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.227+0000 7fc8b9000640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7fc8b40cfef0 0x7fc8b406eae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.192:60278/0 (socket says 172.21.15.192:60278) 2024-05-09T04:33:28.227 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.227+0000 7fc8b3e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc8b415d0b0 0x7fc8b40b02b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:39796/0 (socket says 172.21.15.192:39796) 2024-05-09T04:33:28.227 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.227+0000 7fc8b9000640 1 -- 172.21.15.192:0/1567686781 learned_addr learned my addr 172.21.15.192:0/1567686781 (peer_addr_for_me v1:172.21.15.192:0/0) 2024-05-09T04:33:28.227 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.227+0000 7fc8b3e00640 1 -- 172.21.15.192:0/1567686781 >> v1:172.21.15.192:6789/0 conn(0x7fc8b40cfef0 legacy=0x7fc8b406eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:28.227 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.227+0000 7fc8b3e00640 1 -- 172.21.15.192:0/1567686781 >> v1:172.21.15.145:6789/0 conn(0x7fc8b40add40 legacy=0x7fc8b4072140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:28.227 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.227+0000 7fc8b3e00640 1 -- 172.21.15.192:0/1567686781 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8b4070270 con 0x7fc8b415d0b0 2024-05-09T04:33:28.228 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.229+0000 7fc8b3e00640 1 --2- 172.21.15.192:0/1567686781 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8b415d0b0 0x7fc8b40b02b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc8a40097a0 tx=0x7fc8a402a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a81d7e65b0373379 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:28.228 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.229+0000 7fc8b2a00640 1 -- 172.21.15.192:0/1567686781 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a4037070 con 0x7fc8b415d0b0 2024-05-09T04:33:28.228 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.229+0000 7fc8b2a00640 1 -- 172.21.15.192:0/1567686781 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8a402ae50 con 0x7fc8b415d0b0 2024-05-09T04:33:28.228 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.229+0000 7fc8b2a00640 1 -- 172.21.15.192:0/1567686781 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a4032660 con 0x7fc8b415d0b0 2024-05-09T04:33:28.229 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.230+0000 7fc8b9a00640 1 -- 172.21.15.192:0/1567686781 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8b415d0b0 msgr2=0x7fc8b40b02b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:28.229 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.230+0000 7fc8b9a00640 1 --2- 172.21.15.192:0/1567686781 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8b415d0b0 0x7fc8b40b02b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc8a40097a0 tx=0x7fc8a402a100 comp rx=0 tx=0).stop 2024-05-09T04:33:28.229 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.230+0000 7fc8b9a00640 1 -- 172.21.15.192:0/1567686781 shutdown_connections 2024-05-09T04:33:28.229 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.230+0000 7fc8b9a00640 1 --2- 172.21.15.192:0/1567686781 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8b415d0b0 0x7fc8b40b02b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:28.229 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.230+0000 7fc8b9a00640 1 -- 172.21.15.192:0/1567686781 >> 172.21.15.192:0/1567686781 conn(0x7fc8b40b28f0 msgr2=0x7fc8b40b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:28.229 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.230+0000 7fc8b9a00640 1 -- 172.21.15.192:0/1567686781 shutdown_connections 2024-05-09T04:33:28.229 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.230+0000 7fc8b9a00640 1 -- 172.21.15.192:0/1567686781 wait complete. 2024-05-09T04:33:28.230 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.231+0000 7fc8b9a00640 1 Processor -- start 2024-05-09T04:33:28.231 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.232+0000 7fc8b9a00640 1 -- start start 2024-05-09T04:33:28.233 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.233+0000 7fc8b9a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8b40add40 0x7fc8b41f7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:28.233 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.233+0000 7fc8b9a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc8b40cfef0 0x7fc8b4155b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:28.233 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.233+0000 7fc8b9a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8b415d0b0 0x7fc8b41609f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:28.234 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.233+0000 7fc8b9a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b406f1f0 con 0x7fc8b40cfef0 2024-05-09T04:33:28.234 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.233+0000 7fc8b9a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b406e5c0 con 0x7fc8b415d0b0 2024-05-09T04:33:28.234 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.233+0000 7fc8b9a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc8b406f370 con 0x7fc8b40add40 2024-05-09T04:33:28.234 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.234+0000 7fc8b3e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8b40add40 0x7fc8b41f7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:28.234 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.234+0000 7fc8b9000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8b415d0b0 0x7fc8b41609f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:28.234 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.234+0000 7fc8b3e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8b40add40 0x7fc8b41f7c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.192:54730/0 (socket says 172.21.15.192:54730) 2024-05-09T04:33:28.234 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.234+0000 7fc8b3e00640 1 -- 172.21.15.192:0/2129409417 learned_addr learned my addr 172.21.15.192:0/2129409417 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-05-09T04:33:28.234 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.234+0000 7fc8b3400640 1 --2- 172.21.15.192:0/2129409417 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc8b40cfef0 0x7fc8b4155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:28.234 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.234+0000 7fc8b3e00640 1 -- 172.21.15.192:0/2129409417 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8b415d0b0 msgr2=0x7fc8b41609f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:28.234 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.234+0000 7fc8b3e00640 1 --2- 172.21.15.192:0/2129409417 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8b415d0b0 0x7fc8b41609f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:28.234 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.234+0000 7fc8b3e00640 1 -- 172.21.15.192:0/2129409417 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc8b40cfef0 msgr2=0x7fc8b4155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:28.234 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.234+0000 7fc8b3e00640 1 --2- 172.21.15.192:0/2129409417 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc8b40cfef0 0x7fc8b4155b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:28.234 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.234+0000 7fc8b3e00640 1 -- 172.21.15.192:0/2129409417 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8b41564f0 con 0x7fc8b40add40 2024-05-09T04:33:28.235 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.235+0000 7fc8b3e00640 1 --2- 172.21.15.192:0/2129409417 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8b40add40 0x7fc8b41f7c80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4035040 tx=0x7fc8a40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:28.236 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.235+0000 7fc8abe00640 1 -- 172.21.15.192:0/2129409417 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a4041070 con 0x7fc8b40add40 2024-05-09T04:33:28.236 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.235+0000 7fc8abe00640 1 -- 172.21.15.192:0/2129409417 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8a403c670 con 0x7fc8b40add40 2024-05-09T04:33:28.236 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.235+0000 7fc8abe00640 1 -- 172.21.15.192:0/2129409417 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a4037040 con 0x7fc8b40add40 2024-05-09T04:33:28.237 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.235+0000 7fc8b9a00640 1 -- 172.21.15.192:0/2129409417 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8b41567d0 con 0x7fc8b40add40 2024-05-09T04:33:28.237 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.235+0000 7fc8b9a00640 1 -- 172.21.15.192:0/2129409417 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8b4156d90 con 0x7fc8b40add40 2024-05-09T04:33:28.237 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.235+0000 7fc8abe00640 1 -- 172.21.15.192:0/2129409417 <== mon.2 v2:172.21.15.192:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc8a403a070 con 0x7fc8b40add40 2024-05-09T04:33:28.237 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.235+0000 7fc8abe00640 1 -- 172.21.15.192:0/2129409417 <== mon.2 v2:172.21.15.192:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7fc8a4045400 con 0x7fc8b40add40 2024-05-09T04:33:28.237 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.236+0000 7fc8b9a00640 1 -- 172.21.15.192:0/2129409417 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8b406a310 con 0x7fc8b40add40 2024-05-09T04:33:28.238 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.239+0000 7fc8abe00640 1 -- 172.21.15.192:0/2129409417 <== mon.2 v2:172.21.15.192:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fc8a403ca80 con 0x7fc8b40add40 2024-05-09T04:33:28.320 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.320+0000 7fc8b9a00640 1 -- 172.21.15.192:0/2129409417 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "osd new", "uuid": "b3b1f67e-5790-483f-8610-e05a5bce4c30", "id": 2} v 0) -- 0x7fc8b4072350 con 0x7fc8b40add40 2024-05-09T04:33:28.322 INFO:tasks.ceph.mgr.y.smithi145.stderr:2024-05-09T04:33:28.322+0000 7f5551720040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T04:33:28.327 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.328+0000 7fc8abe00640 1 -- 172.21.15.192:0/2129409417 <== mon.2 v2:172.21.15.192:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b3b1f67e-5790-483f-8610-e05a5bce4c30", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7fc8a403cd20 con 0x7fc8b40add40 2024-05-09T04:33:28.327 INFO:teuthology.orchestra.run.smithi192.stdout:2 2024-05-09T04:33:28.329 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.329+0000 7fc8b9a00640 1 -- 172.21.15.192:0/2129409417 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8b40add40 msgr2=0x7fc8b41f7c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:28.329 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.329+0000 7fc8b9a00640 1 --2- 172.21.15.192:0/2129409417 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8b40add40 0x7fc8b41f7c80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4035040 tx=0x7fc8a40046c0 comp rx=0 tx=0).stop 2024-05-09T04:33:28.329 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.330+0000 7fc8b9a00640 1 -- 172.21.15.192:0/2129409417 shutdown_connections 2024-05-09T04:33:28.329 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.330+0000 7fc8b9a00640 1 --2- 172.21.15.192:0/2129409417 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8b415d0b0 0x7fc8b41609f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:28.330 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.330+0000 7fc8b9a00640 1 --2- 172.21.15.192:0/2129409417 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc8b40cfef0 0x7fc8b4155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:28.330 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.330+0000 7fc8b9a00640 1 --2- 172.21.15.192:0/2129409417 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8b40add40 0x7fc8b41f7c80 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:28.330 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.330+0000 7fc8b9a00640 1 -- 172.21.15.192:0/2129409417 >> 172.21.15.192:0/2129409417 conn(0x7fc8b40b28f0 msgr2=0x7fc8b40b39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:28.330 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.330+0000 7fc8b9a00640 1 -- 172.21.15.192:0/2129409417 shutdown_connections 2024-05-09T04:33:28.330 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.330+0000 7fc8b9a00640 1 -- 172.21.15.192:0/2129409417 wait complete. 2024-05-09T04:33:28.347 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new 884787a5-e037-4d31-9aae-743030eba4ce 3 2024-05-09T04:33:28.367 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-09T04:33:28.368+0000 7f2e3f522040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T04:33:28.425 INFO:tasks.ceph.mgr.y.smithi145.stderr:2024-05-09T04:33:28.426+0000 7f5551720040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T04:33:28.475 INFO:tasks.ceph.mgr.y.smithi145.stderr:2024-05-09T04:33:28.476+0000 7f5551720040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T04:33:28.483 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-09T04:33:28.484+0000 7f2e3f522040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T04:33:28.519 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.520+0000 7f2315a00640 1 Processor -- start 2024-05-09T04:33:28.519 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.520+0000 7f2315a00640 1 -- start start 2024-05-09T04:33:28.522 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.522+0000 7f2315a00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f23100d58a0 0x7f23100d2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:28.522 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.522+0000 7f2315a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23100b5d70 0x7f23100d0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:28.522 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.522+0000 7f2315a00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f231015f9f0 con 0x7f231015f610 2024-05-09T04:33:28.522 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.522+0000 7f2315a00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f23100d40c0 con 0x7f23100d58a0 2024-05-09T04:33:28.522 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.522+0000 7f2315a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f23100b6200 con 0x7f23100b5d70 2024-05-09T04:33:28.523 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.522+0000 7f230f400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23100b5d70 0x7f23100d0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:28.523 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.522+0000 7f2307e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f23100d58a0 0x7f23100d2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:28.523 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.522+0000 7f230f400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23100b5d70 0x7f23100d0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.192:54744/0 (socket says 172.21.15.192:54744) 2024-05-09T04:33:28.523 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.522+0000 7f230f400640 1 -- 172.21.15.192:0/3164882679 learned_addr learned my addr 172.21.15.192:0/3164882679 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-05-09T04:33:28.523 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.522+0000 7f2307e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f23100d58a0 0x7f23100d2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:39820/0 (socket says 172.21.15.192:39820) 2024-05-09T04:33:28.523 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.523+0000 7f230f400640 1 -- 172.21.15.192:0/3164882679 >> v2:172.21.15.192:3300/0 conn(0x7f23100d58a0 msgr2=0x7f23100d2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:28.523 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.523+0000 7f230f400640 1 --2- 172.21.15.192:0/3164882679 >> v2:172.21.15.192:3300/0 conn(0x7f23100d58a0 0x7f23100d2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:28.523 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.523+0000 7f230f400640 1 -- 172.21.15.192:0/3164882679 >> v1:172.21.15.145:6789/0 conn(0x7f231015f610 legacy=0x7f23100d5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:28.523 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.523+0000 7f230f400640 1 -- 172.21.15.192:0/3164882679 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23100d34b0 con 0x7f23100b5d70 2024-05-09T04:33:28.523 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.523+0000 7f2307e00640 1 --2- 172.21.15.192:0/3164882679 >> v2:172.21.15.192:3300/0 conn(0x7f23100d58a0 0x7f23100d2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:33:28.524 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.524+0000 7f230f400640 1 --2- 172.21.15.192:0/3164882679 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23100b5d70 0x7f23100d0050 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2308009e50 tx=0x7f230802a7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef3a01fdd5cd54f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:28.524 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.524+0000 7f230e000640 1 -- 172.21.15.192:0/3164882679 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2308036070 con 0x7f23100b5d70 2024-05-09T04:33:28.524 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.524+0000 7f230e000640 1 -- 172.21.15.192:0/3164882679 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2308031600 con 0x7f23100b5d70 2024-05-09T04:33:28.524 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.524+0000 7f230e000640 1 -- 172.21.15.192:0/3164882679 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23080319d0 con 0x7f23100b5d70 2024-05-09T04:33:28.525 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.525+0000 7f2315a00640 1 -- 172.21.15.192:0/3164882679 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23100b5d70 msgr2=0x7f23100d0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:28.525 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.525+0000 7f2315a00640 1 --2- 172.21.15.192:0/3164882679 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23100b5d70 0x7f23100d0050 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2308009e50 tx=0x7f230802a7a0 comp rx=0 tx=0).stop 2024-05-09T04:33:28.525 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.525+0000 7f2315a00640 1 -- 172.21.15.192:0/3164882679 shutdown_connections 2024-05-09T04:33:28.525 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.525+0000 7f2315a00640 1 --2- 172.21.15.192:0/3164882679 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23100b5d70 0x7f23100d0050 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:28.525 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.525+0000 7f2315a00640 1 --2- 172.21.15.192:0/3164882679 >> v2:172.21.15.192:3300/0 conn(0x7f23100d58a0 0x7f23100d2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:28.525 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.525+0000 7f2315a00640 1 -- 172.21.15.192:0/3164882679 >> 172.21.15.192:0/3164882679 conn(0x7f231008ec20 msgr2=0x7f231009e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:28.525 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.525+0000 7f2315a00640 1 -- 172.21.15.192:0/3164882679 shutdown_connections 2024-05-09T04:33:28.525 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.525+0000 7f2315a00640 1 -- 172.21.15.192:0/3164882679 wait complete. 2024-05-09T04:33:28.527 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.527+0000 7f2315a00640 1 Processor -- start 2024-05-09T04:33:28.527 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.527+0000 7f2315a00640 1 -- start start 2024-05-09T04:33:28.528 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.528+0000 7f2315a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23100b5d70 0x7f231015b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:28.529 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.528+0000 7f2315a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f23100d58a0 0x7f2310160670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:28.529 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.528+0000 7f2315a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f231015f610 0x7f23100b9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:28.529 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.528+0000 7f2315a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f23100d3d50 con 0x7f23100d58a0 2024-05-09T04:33:28.529 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.528+0000 7f2315a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f23100d3a70 con 0x7f231015f610 2024-05-09T04:33:28.529 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.528+0000 7f2315a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f23100d0590 con 0x7f23100b5d70 2024-05-09T04:33:28.529 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.529+0000 7f230f400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f231015f610 0x7f23100b9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:28.529 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.529+0000 7f230ea00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23100b5d70 0x7f231015b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:28.529 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.529+0000 7f230ea00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23100b5d70 0x7f231015b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.192:54752/0 (socket says 172.21.15.192:54752) 2024-05-09T04:33:28.529 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.529+0000 7f230ea00640 1 -- 172.21.15.192:0/2857081703 learned_addr learned my addr 172.21.15.192:0/2857081703 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-05-09T04:33:28.529 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.529+0000 7f230f400640 1 --2- 172.21.15.192:0/2857081703 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f231015f610 0x7f23100b9fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:39828/0 (socket says 172.21.15.192:39828) 2024-05-09T04:33:28.529 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.529+0000 7f2307e00640 1 --2- 172.21.15.192:0/2857081703 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f23100d58a0 0x7f2310160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:28.529 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.529+0000 7f230ea00640 1 -- 172.21.15.192:0/2857081703 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f231015f610 msgr2=0x7f23100b9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:28.529 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.529+0000 7f230ea00640 1 --2- 172.21.15.192:0/2857081703 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f231015f610 0x7f23100b9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:28.530 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.529+0000 7f230ea00640 1 -- 172.21.15.192:0/2857081703 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f23100d58a0 msgr2=0x7f2310160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:28.530 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.529+0000 7f230ea00640 1 --2- 172.21.15.192:0/2857081703 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f23100d58a0 0x7f2310160670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:28.530 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.529+0000 7f230ea00640 1 -- 172.21.15.192:0/2857081703 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23100b91f0 con 0x7f23100b5d70 2024-05-09T04:33:28.530 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.529+0000 7f230f400640 1 --2- 172.21.15.192:0/2857081703 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f231015f610 0x7f23100b9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:33:28.530 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.529+0000 7f230ea00640 1 --2- 172.21.15.192:0/2857081703 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23100b5d70 0x7f231015b890 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f230000c3a0 tx=0x7f230000c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:28.531 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.530+0000 7f2306a00640 1 -- 172.21.15.192:0/2857081703 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2300017070 con 0x7f23100b5d70 2024-05-09T04:33:28.531 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.530+0000 7f2306a00640 1 -- 172.21.15.192:0/2857081703 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23000052d0 con 0x7f23100b5d70 2024-05-09T04:33:28.531 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.530+0000 7f2306a00640 1 -- 172.21.15.192:0/2857081703 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2300003830 con 0x7f23100b5d70 2024-05-09T04:33:28.531 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.530+0000 7f2315a00640 1 -- 172.21.15.192:0/2857081703 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23100b94d0 con 0x7f23100b5d70 2024-05-09T04:33:28.531 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.530+0000 7f2315a00640 1 -- 172.21.15.192:0/2857081703 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f23100b9a90 con 0x7f23100b5d70 2024-05-09T04:33:28.531 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.530+0000 7f2306a00640 1 -- 172.21.15.192:0/2857081703 <== mon.2 v2:172.21.15.192:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f230001a070 con 0x7f23100b5d70 2024-05-09T04:33:28.531 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.530+0000 7f2306a00640 1 -- 172.21.15.192:0/2857081703 <== mon.2 v2:172.21.15.192:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f2300011590 con 0x7f23100b5d70 2024-05-09T04:33:28.531 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.531+0000 7f2315a00640 1 -- 172.21.15.192:0/2857081703 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f231006a310 con 0x7f23100b5d70 2024-05-09T04:33:28.533 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.534+0000 7f2306a00640 1 -- 172.21.15.192:0/2857081703 <== mon.2 v2:172.21.15.192:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f2300021070 con 0x7f23100b5d70 2024-05-09T04:33:28.542 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-09T04:33:28.543+0000 7f2e3f522040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T04:33:28.610 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.611+0000 7f2315a00640 1 -- 172.21.15.192:0/2857081703 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "osd new", "uuid": "884787a5-e037-4d31-9aae-743030eba4ce", "id": 3} v 0) -- 0x7f231015e150 con 0x7f23100b5d70 2024-05-09T04:33:28.620 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.621+0000 7f2306a00640 1 -- 172.21.15.192:0/2857081703 <== mon.2 v2:172.21.15.192:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "884787a5-e037-4d31-9aae-743030eba4ce", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f2300011b30 con 0x7f23100b5d70 2024-05-09T04:33:28.620 INFO:teuthology.orchestra.run.smithi192.stdout:3 2024-05-09T04:33:28.622 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.622+0000 7f2315a00640 1 -- 172.21.15.192:0/2857081703 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23100b5d70 msgr2=0x7f231015b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:28.622 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.622+0000 7f2315a00640 1 --2- 172.21.15.192:0/2857081703 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23100b5d70 0x7f231015b890 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f230000c3a0 tx=0x7f230000c860 comp rx=0 tx=0).stop 2024-05-09T04:33:28.622 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.622+0000 7f2315a00640 1 -- 172.21.15.192:0/2857081703 shutdown_connections 2024-05-09T04:33:28.622 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.622+0000 7f2315a00640 1 --2- 172.21.15.192:0/2857081703 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f231015f610 0x7f23100b9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:28.622 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.622+0000 7f2315a00640 1 --2- 172.21.15.192:0/2857081703 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f23100d58a0 0x7f2310160670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:28.622 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.622+0000 7f2315a00640 1 --2- 172.21.15.192:0/2857081703 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23100b5d70 0x7f231015b890 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:28.622 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.622+0000 7f2315a00640 1 -- 172.21.15.192:0/2857081703 >> 172.21.15.192:0/2857081703 conn(0x7f231008ec20 msgr2=0x7f23100d4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:28.622 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.622+0000 7f2315a00640 1 -- 172.21.15.192:0/2857081703 shutdown_connections 2024-05-09T04:33:28.622 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.622+0000 7f2315a00640 1 -- 172.21.15.192:0/2857081703 wait complete. 2024-05-09T04:33:28.638 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new 0759fc1e-c5fc-4a2f-86bb-e8eccd83f833 4 2024-05-09T04:33:28.668 INFO:tasks.ceph.mgr.y.smithi145.stderr:2024-05-09T04:33:28.668+0000 7f5551720040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T04:33:28.733 INFO:tasks.ceph.mgr.y.smithi145.stderr:2024-05-09T04:33:28.733+0000 7f5551720040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T04:33:28.745 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-09T04:33:28.746+0000 7f2e3f522040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T04:33:28.794 INFO:tasks.ceph.mgr.y.smithi145.stderr:2024-05-09T04:33:28.794+0000 7f5551720040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T04:33:28.804 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-09T04:33:28.804+0000 7f2e3f522040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T04:33:28.868 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.868+0000 7f59f5e00640 1 Processor -- start 2024-05-09T04:33:28.868 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.869+0000 7f59f5e00640 1 -- start start 2024-05-09T04:33:28.868 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-09T04:33:28.869+0000 7f2e3f522040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T04:33:28.870 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.870+0000 7f59f5e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f59f00b8960 0x7f59f0071250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:28.870 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.870+0000 7f59f5e00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59f015e920 con 0x7f59f00cd7c0 2024-05-09T04:33:28.870 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.870+0000 7f59f5e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59f00b8d40 con 0x7f59f00d2ac0 2024-05-09T04:33:28.870 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.870+0000 7f59f5e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f59f015e490 con 0x7f59f00b8960 2024-05-09T04:33:28.870 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.870+0000 7f59eea00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f59f00b8960 0x7f59f0071250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:28.870 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.870+0000 7f59ef400640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f59f00d2ac0 0x7f59f0153d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.192:60286/0 (socket says 172.21.15.192:60286) 2024-05-09T04:33:28.870 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.870+0000 7f59ef400640 1 -- 172.21.15.192:0/108906611 learned_addr learned my addr 172.21.15.192:0/108906611 (peer_addr_for_me v1:172.21.15.192:0/0) 2024-05-09T04:33:28.871 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.871+0000 7f59eea00640 1 -- 172.21.15.192:0/108906611 >> v1:172.21.15.192:6789/0 conn(0x7f59f00d2ac0 legacy=0x7f59f0153d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:28.871 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.871+0000 7f59eea00640 1 -- 172.21.15.192:0/108906611 >> v1:172.21.15.145:6789/0 conn(0x7f59f00cd7c0 legacy=0x7f59f00d23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:28.871 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.871+0000 7f59eea00640 1 -- 172.21.15.192:0/108906611 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59f00cdf80 con 0x7f59f00b8960 2024-05-09T04:33:28.871 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.871+0000 7f59ee000640 1 -- 172.21.15.192:0/108906611 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4288470447 0 0) 0x7f59e8001490 con 0x7f59f00d2ac0 2024-05-09T04:33:28.872 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.872+0000 7f59eea00640 1 --2- 172.21.15.192:0/108906611 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f59f00b8960 0x7f59f0071250 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f59e00097d0 tx=0x7f59e002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73fd1216876a7d39 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:28.872 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.872+0000 7f59ee000640 1 -- 172.21.15.192:0/108906611 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e0037070 con 0x7f59f00b8960 2024-05-09T04:33:28.872 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.872+0000 7f59ee000640 1 -- 172.21.15.192:0/108906611 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59e0032440 con 0x7f59f00b8960 2024-05-09T04:33:28.872 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.872+0000 7f59ee000640 1 -- 172.21.15.192:0/108906611 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e0032810 con 0x7f59f00b8960 2024-05-09T04:33:28.873 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.873+0000 7f59f5e00640 1 -- 172.21.15.192:0/108906611 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f59f00b8960 msgr2=0x7f59f0071250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:28.873 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.873+0000 7f59f5e00640 1 --2- 172.21.15.192:0/108906611 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f59f00b8960 0x7f59f0071250 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f59e00097d0 tx=0x7f59e002a210 comp rx=0 tx=0).stop 2024-05-09T04:33:28.873 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.873+0000 7f59f5e00640 1 -- 172.21.15.192:0/108906611 shutdown_connections 2024-05-09T04:33:28.873 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.873+0000 7f59f5e00640 1 --2- 172.21.15.192:0/108906611 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f59f00b8960 0x7f59f0071250 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:28.873 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.873+0000 7f59f5e00640 1 -- 172.21.15.192:0/108906611 >> 172.21.15.192:0/108906611 conn(0x7f59f00d3170 msgr2=0x7f59f00d3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:28.873 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.873+0000 7f59f5e00640 1 -- 172.21.15.192:0/108906611 shutdown_connections 2024-05-09T04:33:28.873 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.873+0000 7f59f5e00640 1 -- 172.21.15.192:0/108906611 wait complete. 2024-05-09T04:33:28.874 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.875+0000 7f59f5e00640 1 Processor -- start 2024-05-09T04:33:28.874 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.875+0000 7f59f5e00640 1 -- start start 2024-05-09T04:33:28.876 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.876+0000 7f59f5e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f59f00b8960 0x7f59f0152590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:28.876 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.876+0000 7f59f5e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f59f00cd7c0 0x7f59f00b3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:28.876 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.876+0000 7f59f5e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f59f00d2ac0 0x7f59f00b3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:28.877 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.876+0000 7f59f5e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f59f0071d30 con 0x7f59f00d2ac0 2024-05-09T04:33:28.877 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.876+0000 7f59f5e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f59f015d900 con 0x7f59f00cd7c0 2024-05-09T04:33:28.877 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.876+0000 7f59f5e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f59f015de40 con 0x7f59f00b8960 2024-05-09T04:33:28.877 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.876+0000 7f59eea00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f59f00b8960 0x7f59f0152590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:28.877 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.876+0000 7f59e7e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f59f00cd7c0 0x7f59f00b3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:28.877 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.876+0000 7f59eea00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f59f00b8960 0x7f59f0152590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.192:54780/0 (socket says 172.21.15.192:54780) 2024-05-09T04:33:28.877 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.876+0000 7f59eea00640 1 -- 172.21.15.192:0/248673327 learned_addr learned my addr 172.21.15.192:0/248673327 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-05-09T04:33:28.877 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.877+0000 7f59ef400640 1 --2- 172.21.15.192:0/248673327 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f59f00d2ac0 0x7f59f00b3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:28.877 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.877+0000 7f59eea00640 1 -- 172.21.15.192:0/248673327 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f59f00cd7c0 msgr2=0x7f59f00b3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:28.877 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.877+0000 7f59eea00640 1 --2- 172.21.15.192:0/248673327 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f59f00cd7c0 0x7f59f00b3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:28.877 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.877+0000 7f59eea00640 1 -- 172.21.15.192:0/248673327 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f59f00d2ac0 msgr2=0x7f59f00b3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:28.877 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.877+0000 7f59eea00640 1 --2- 172.21.15.192:0/248673327 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f59f00d2ac0 0x7f59f00b3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:28.877 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.877+0000 7f59eea00640 1 -- 172.21.15.192:0/248673327 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59f00c6ca0 con 0x7f59f00b8960 2024-05-09T04:33:28.877 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.877+0000 7f59eea00640 1 --2- 172.21.15.192:0/248673327 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f59f00b8960 0x7f59f0152590 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f59e00098f0 tx=0x7f59e00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:28.879 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.877+0000 7f59e7400640 1 -- 172.21.15.192:0/248673327 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e0037040 con 0x7f59f00b8960 2024-05-09T04:33:28.879 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.877+0000 7f59e7400640 1 -- 172.21.15.192:0/248673327 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59e0042070 con 0x7f59f00b8960 2024-05-09T04:33:28.879 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.877+0000 7f59f5e00640 1 -- 172.21.15.192:0/248673327 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59f00c6f80 con 0x7f59f00b8960 2024-05-09T04:33:28.879 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.877+0000 7f59e7400640 1 -- 172.21.15.192:0/248673327 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e0030bd0 con 0x7f59f00b8960 2024-05-09T04:33:28.879 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.878+0000 7f59f5e00640 1 -- 172.21.15.192:0/248673327 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f59f00c7540 con 0x7f59f00b8960 2024-05-09T04:33:28.879 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.878+0000 7f59e7400640 1 -- 172.21.15.192:0/248673327 <== mon.2 v2:172.21.15.192:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f59e0047460 con 0x7f59f00b8960 2024-05-09T04:33:28.879 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.878+0000 7f59e7400640 1 -- 172.21.15.192:0/248673327 <== mon.2 v2:172.21.15.192:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f59e003d950 con 0x7f59f00b8960 2024-05-09T04:33:28.880 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.879+0000 7f59f5e00640 1 -- 172.21.15.192:0/248673327 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59f00694e0 con 0x7f59f00b8960 2024-05-09T04:33:28.882 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.883+0000 7f59e7400640 1 -- 172.21.15.192:0/248673327 <== mon.2 v2:172.21.15.192:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f59e0032e20 con 0x7f59f00b8960 2024-05-09T04:33:28.921 INFO:tasks.ceph.mgr.y.smithi145.stderr:2024-05-09T04:33:28.921+0000 7f5551720040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T04:33:28.963 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.963+0000 7f59f5e00640 1 -- 172.21.15.192:0/248673327 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "osd new", "uuid": "0759fc1e-c5fc-4a2f-86bb-e8eccd83f833", "id": 4} v 0) -- 0x7f59f00d4080 con 0x7f59f00b8960 2024-05-09T04:33:28.971 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.971+0000 7f59e7400640 1 -- 172.21.15.192:0/248673327 <== mon.2 v2:172.21.15.192:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0759fc1e-c5fc-4a2f-86bb-e8eccd83f833", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f59e003dbe0 con 0x7f59f00b8960 2024-05-09T04:33:28.971 INFO:teuthology.orchestra.run.smithi192.stdout:4 2024-05-09T04:33:28.973 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.973+0000 7f59f5e00640 1 -- 172.21.15.192:0/248673327 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f59f00b8960 msgr2=0x7f59f0152590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:28.973 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.973+0000 7f59f5e00640 1 --2- 172.21.15.192:0/248673327 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f59f00b8960 0x7f59f0152590 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f59e00098f0 tx=0x7f59e00046c0 comp rx=0 tx=0).stop 2024-05-09T04:33:28.973 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.973+0000 7f59f5e00640 1 -- 172.21.15.192:0/248673327 shutdown_connections 2024-05-09T04:33:28.973 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.973+0000 7f59f5e00640 1 --2- 172.21.15.192:0/248673327 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f59f00d2ac0 0x7f59f00b3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:28.973 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.973+0000 7f59f5e00640 1 --2- 172.21.15.192:0/248673327 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f59f00cd7c0 0x7f59f00b3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:28.973 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.973+0000 7f59f5e00640 1 --2- 172.21.15.192:0/248673327 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f59f00b8960 0x7f59f0152590 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:28.973 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.973+0000 7f59f5e00640 1 -- 172.21.15.192:0/248673327 >> 172.21.15.192:0/248673327 conn(0x7f59f00d3170 msgr2=0x7f59f00d21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:28.973 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.973+0000 7f59f5e00640 1 -- 172.21.15.192:0/248673327 shutdown_connections 2024-05-09T04:33:28.973 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:28.973+0000 7f59f5e00640 1 -- 172.21.15.192:0/248673327 wait complete. 2024-05-09T04:33:28.990 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new 9443cc22-9379-46e3-9173-98b592bf2c16 5 2024-05-09T04:33:29.006 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-09T04:33:29.006+0000 7f2e3f522040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T04:33:29.169 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.169+0000 7feb66000640 1 Processor -- start 2024-05-09T04:33:29.170 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.170+0000 7feb66000640 1 -- start start 2024-05-09T04:33:29.171 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.171+0000 7feb66000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7feb601628e0 0x7feb600b5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:29.171 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.171+0000 7feb66000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feb600d1920 0x7feb600d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:29.171 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.171+0000 7feb66000640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb600d22e0 con 0x7feb600c8aa0 2024-05-09T04:33:29.171 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.171+0000 7feb66000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7feb600d40b0 con 0x7feb601628e0 2024-05-09T04:33:29.171 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.171+0000 7feb66000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7feb600d2580 con 0x7feb600d1920 2024-05-09T04:33:29.171 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.171+0000 7feb5fe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feb600d1920 0x7feb600d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:29.171 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.171+0000 7feb65600640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7feb600c8aa0 0x7feb600c8f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.192:60296/0 (socket says 172.21.15.192:60296) 2024-05-09T04:33:29.172 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.172+0000 7feb65600640 1 -- 172.21.15.192:0/437677288 learned_addr learned my addr 172.21.15.192:0/437677288 (peer_addr_for_me v1:172.21.15.192:0/0) 2024-05-09T04:33:29.172 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.172+0000 7feb64c00640 1 --2- 172.21.15.192:0/437677288 >> v2:172.21.15.145:3300/0 conn(0x7feb601628e0 0x7feb600b5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:29.174 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.172+0000 7feb5fe00640 1 -- 172.21.15.192:0/437677288 >> v2:172.21.15.145:3300/0 conn(0x7feb601628e0 msgr2=0x7feb600b5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:29.174 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.172+0000 7feb5fe00640 1 --2- 172.21.15.192:0/437677288 >> v2:172.21.15.145:3300/0 conn(0x7feb601628e0 0x7feb600b5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:29.174 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.172+0000 7feb5f400640 1 -- 172.21.15.192:0/437677288 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2400742320 0 0) 0x7feb600d22e0 con 0x7feb600c8aa0 2024-05-09T04:33:29.174 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.172+0000 7feb5fe00640 1 -- 172.21.15.192:0/437677288 >> v1:172.21.15.192:6789/0 conn(0x7feb600c8aa0 legacy=0x7feb600c8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:29.175 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.172+0000 7feb5fe00640 1 -- 172.21.15.192:0/437677288 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb600d35d0 con 0x7feb600d1920 2024-05-09T04:33:29.175 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.173+0000 7feb5fe00640 1 --2- 172.21.15.192:0/437677288 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feb600d1920 0x7feb600d1da0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7feb4c0097a0 tx=0x7feb4c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81d89fa36250ded9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:29.175 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.173+0000 7feb5f400640 1 -- 172.21.15.192:0/437677288 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb4c037070 con 0x7feb600d1920 2024-05-09T04:33:29.175 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.173+0000 7feb5f400640 1 -- 172.21.15.192:0/437677288 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb4c032440 con 0x7feb600d1920 2024-05-09T04:33:29.175 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.173+0000 7feb5f400640 1 -- 172.21.15.192:0/437677288 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb4c032810 con 0x7feb600d1920 2024-05-09T04:33:29.175 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.174+0000 7feb66000640 1 -- 172.21.15.192:0/437677288 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feb600d1920 msgr2=0x7feb600d1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:29.175 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.174+0000 7feb66000640 1 --2- 172.21.15.192:0/437677288 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feb600d1920 0x7feb600d1da0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7feb4c0097a0 tx=0x7feb4c02a180 comp rx=0 tx=0).stop 2024-05-09T04:33:29.175 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.174+0000 7feb66000640 1 -- 172.21.15.192:0/437677288 shutdown_connections 2024-05-09T04:33:29.175 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.174+0000 7feb66000640 1 --2- 172.21.15.192:0/437677288 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feb600d1920 0x7feb600d1da0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:29.175 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.174+0000 7feb66000640 1 --2- 172.21.15.192:0/437677288 >> v2:172.21.15.145:3300/0 conn(0x7feb601628e0 0x7feb600b5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:29.175 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.174+0000 7feb66000640 1 -- 172.21.15.192:0/437677288 >> 172.21.15.192:0/437677288 conn(0x7feb600d4f50 msgr2=0x7feb600d5330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:29.175 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.174+0000 7feb66000640 1 -- 172.21.15.192:0/437677288 shutdown_connections 2024-05-09T04:33:29.175 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.174+0000 7feb66000640 1 -- 172.21.15.192:0/437677288 wait complete. 2024-05-09T04:33:29.178 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.176+0000 7feb66000640 1 Processor -- start 2024-05-09T04:33:29.178 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.176+0000 7feb66000640 1 -- start start 2024-05-09T04:33:29.178 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.177+0000 7feb66000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feb600c8aa0 0x7feb600ba660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:29.178 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.177+0000 7feb66000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feb600d1920 0x7feb600ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:29.178 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.177+0000 7feb66000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feb601628e0 0x7feb6014c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:29.178 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.177+0000 7feb66000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7feb600b6580 con 0x7feb601628e0 2024-05-09T04:33:29.178 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.177+0000 7feb66000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7feb600d3b90 con 0x7feb600d1920 2024-05-09T04:33:29.179 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.177+0000 7feb66000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7feb60162cc0 con 0x7feb600c8aa0 2024-05-09T04:33:29.179 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.177+0000 7feb5fe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feb600d1920 0x7feb600ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:29.179 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.177+0000 7feb64c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feb600c8aa0 0x7feb600ba660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:29.179 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.177+0000 7feb5fe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feb600d1920 0x7feb600ce8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:39834/0 (socket says 172.21.15.192:39834) 2024-05-09T04:33:29.179 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.177+0000 7feb5fe00640 1 -- 172.21.15.192:0/2039835424 learned_addr learned my addr 172.21.15.192:0/2039835424 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-05-09T04:33:29.179 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.177+0000 7feb65600640 1 --2- 172.21.15.192:0/2039835424 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feb601628e0 0x7feb6014c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:29.179 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.178+0000 7feb5fe00640 1 -- 172.21.15.192:0/2039835424 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feb600c8aa0 msgr2=0x7feb600ba660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:29.180 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.178+0000 7feb5fe00640 1 --2- 172.21.15.192:0/2039835424 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feb600c8aa0 0x7feb600ba660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:29.180 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.178+0000 7feb5fe00640 1 -- 172.21.15.192:0/2039835424 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feb601628e0 msgr2=0x7feb6014c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:29.180 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.178+0000 7feb5fe00640 1 --2- 172.21.15.192:0/2039835424 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feb601628e0 0x7feb6014c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:29.180 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.178+0000 7feb5fe00640 1 -- 172.21.15.192:0/2039835424 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb600cd810 con 0x7feb600d1920 2024-05-09T04:33:29.180 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.178+0000 7feb64c00640 1 --2- 172.21.15.192:0/2039835424 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feb600c8aa0 0x7feb600ba660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:33:29.180 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.178+0000 7feb65600640 1 --2- 172.21.15.192:0/2039835424 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feb601628e0 0x7feb6014c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:33:29.181 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.178+0000 7feb5fe00640 1 --2- 172.21.15.192:0/2039835424 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feb600d1920 0x7feb600ce8b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7feb4c000c00 tx=0x7feb4c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:29.181 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.178+0000 7feb5cc00640 1 -- 172.21.15.192:0/2039835424 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb4c037040 con 0x7feb600d1920 2024-05-09T04:33:29.181 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.178+0000 7feb5cc00640 1 -- 172.21.15.192:0/2039835424 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb4c004230 con 0x7feb600d1920 2024-05-09T04:33:29.181 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.178+0000 7feb5cc00640 1 -- 172.21.15.192:0/2039835424 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb4c003610 con 0x7feb600d1920 2024-05-09T04:33:29.181 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.178+0000 7feb66000640 1 -- 172.21.15.192:0/2039835424 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb600cdaf0 con 0x7feb600d1920 2024-05-09T04:33:29.181 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.178+0000 7feb66000640 1 -- 172.21.15.192:0/2039835424 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb600ce0b0 con 0x7feb600d1920 2024-05-09T04:33:29.182 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.179+0000 7feb5cc00640 1 -- 172.21.15.192:0/2039835424 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7feb4c003b40 con 0x7feb600d1920 2024-05-09T04:33:29.182 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.179+0000 7feb5cc00640 1 -- 172.21.15.192:0/2039835424 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7feb4c03dd00 con 0x7feb600d1920 2024-05-09T04:33:29.183 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.180+0000 7feb66000640 1 -- 172.21.15.192:0/2039835424 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb600694e0 con 0x7feb600d1920 2024-05-09T04:33:29.183 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.182+0000 7feb5cc00640 1 -- 172.21.15.192:0/2039835424 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7feb4c03d530 con 0x7feb600d1920 2024-05-09T04:33:29.260 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.261+0000 7feb66000640 1 -- 172.21.15.192:0/2039835424 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9443cc22-9379-46e3-9173-98b592bf2c16", "id": 5} v 0) -- 0x7feb6006d060 con 0x7feb600d1920 2024-05-09T04:33:29.267 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.268+0000 7feb5cc00640 1 -- 172.21.15.192:0/2039835424 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9443cc22-9379-46e3-9173-98b592bf2c16", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7feb4c03d710 con 0x7feb600d1920 2024-05-09T04:33:29.267 INFO:teuthology.orchestra.run.smithi192.stdout:5 2024-05-09T04:33:29.269 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.269+0000 7feb66000640 1 -- 172.21.15.192:0/2039835424 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feb600d1920 msgr2=0x7feb600ce8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:29.269 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.269+0000 7feb66000640 1 --2- 172.21.15.192:0/2039835424 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feb600d1920 0x7feb600ce8b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7feb4c000c00 tx=0x7feb4c0046c0 comp rx=0 tx=0).stop 2024-05-09T04:33:29.269 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.269+0000 7feb66000640 1 -- 172.21.15.192:0/2039835424 shutdown_connections 2024-05-09T04:33:29.269 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.269+0000 7feb66000640 1 --2- 172.21.15.192:0/2039835424 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7feb601628e0 0x7feb6014c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:29.269 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.269+0000 7feb66000640 1 --2- 172.21.15.192:0/2039835424 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feb600d1920 0x7feb600ce8b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:29.269 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.269+0000 7feb66000640 1 --2- 172.21.15.192:0/2039835424 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feb600c8aa0 0x7feb600ba660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:29.269 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.269+0000 7feb66000640 1 -- 172.21.15.192:0/2039835424 >> 172.21.15.192:0/2039835424 conn(0x7feb600d4f50 msgr2=0x7feb6015ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:29.269 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.269+0000 7feb66000640 1 -- 172.21.15.192:0/2039835424 shutdown_connections 2024-05-09T04:33:29.270 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.269+0000 7feb66000640 1 -- 172.21.15.192:0/2039835424 wait complete. 2024-05-09T04:33:29.286 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new d05a3311-505a-45b5-85b9-aa3113e96987 6 2024-05-09T04:33:29.375 INFO:tasks.ceph.mgr.y.smithi145.stderr:2024-05-09T04:33:29.374+0000 7f5551720040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T04:33:29.484 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-09T04:33:29.485+0000 7f2e3f522040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T04:33:29.510 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.511+0000 7f432a400640 1 Processor -- start 2024-05-09T04:33:29.511 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.511+0000 7f432a400640 1 -- start start 2024-05-09T04:33:29.512 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.512+0000 7f432a400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f43240d2030 0x7f43240b5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:29.512 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.512+0000 7f432a400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f43240cfb60 0x7f43240cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:29.512 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.512+0000 7f432a400640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43240d2580 con 0x7f43241628e0 2024-05-09T04:33:29.512 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.512+0000 7f432a400640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f43240d40b0 con 0x7f43240d2030 2024-05-09T04:33:29.512 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.512+0000 7f432a400640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f43240d35d0 con 0x7f43240cfb60 2024-05-09T04:33:29.512 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.513+0000 7f4329a00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f43240cfb60 0x7f43240cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:29.512 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.513+0000 7f4329a00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f43240cfb60 0x7f43240cffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:39850/0 (socket says 172.21.15.192:39850) 2024-05-09T04:33:29.512 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.513+0000 7f4329a00640 1 -- 172.21.15.192:0/2734249024 learned_addr learned my addr 172.21.15.192:0/2734249024 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-05-09T04:33:29.513 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.513+0000 7f4329a00640 1 -- 172.21.15.192:0/2734249024 >> v2:172.21.15.145:3300/0 conn(0x7f43240d2030 msgr2=0x7f43240b5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:29.513 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.513+0000 7f4329a00640 1 --2- 172.21.15.192:0/2734249024 >> v2:172.21.15.145:3300/0 conn(0x7f43240d2030 0x7f43240b5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:29.513 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.513+0000 7f4329a00640 1 -- 172.21.15.192:0/2734249024 >> v1:172.21.15.145:6789/0 conn(0x7f43241628e0 legacy=0x7f43240d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:29.513 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.513+0000 7f4329a00640 1 -- 172.21.15.192:0/2734249024 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43240d38b0 con 0x7f43240cfb60 2024-05-09T04:33:29.514 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.515+0000 7f4329a00640 1 --2- 172.21.15.192:0/2734249024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43240cfb60 0x7f43240cffd0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f432408d760 tx=0x7f431c02a6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34040adc1aba703f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:29.514 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.515+0000 7f4323400640 1 -- 172.21.15.192:0/2734249024 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431c036070 con 0x7f43240cfb60 2024-05-09T04:33:29.514 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.515+0000 7f4323400640 1 -- 172.21.15.192:0/2734249024 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f431c031600 con 0x7f43240cfb60 2024-05-09T04:33:29.515 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.515+0000 7f4323400640 1 -- 172.21.15.192:0/2734249024 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431c0319d0 con 0x7f43240cfb60 2024-05-09T04:33:29.515 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.516+0000 7f432a400640 1 -- 172.21.15.192:0/2734249024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43240cfb60 msgr2=0x7f43240cffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:29.515 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.516+0000 7f432a400640 1 --2- 172.21.15.192:0/2734249024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43240cfb60 0x7f43240cffd0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f432408d760 tx=0x7f431c02a6a0 comp rx=0 tx=0).stop 2024-05-09T04:33:29.515 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.516+0000 7f432a400640 1 -- 172.21.15.192:0/2734249024 shutdown_connections 2024-05-09T04:33:29.516 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.516+0000 7f432a400640 1 --2- 172.21.15.192:0/2734249024 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43240cfb60 0x7f43240cffd0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:29.516 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.516+0000 7f432a400640 1 --2- 172.21.15.192:0/2734249024 >> v2:172.21.15.145:3300/0 conn(0x7f43240d2030 0x7f43240b5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:29.516 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.516+0000 7f432a400640 1 -- 172.21.15.192:0/2734249024 >> 172.21.15.192:0/2734249024 conn(0x7f43240d4f50 msgr2=0x7f43240d5330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:29.516 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.516+0000 7f432a400640 1 -- 172.21.15.192:0/2734249024 shutdown_connections 2024-05-09T04:33:29.516 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.516+0000 7f432a400640 1 -- 172.21.15.192:0/2734249024 wait complete. 2024-05-09T04:33:29.517 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.518+0000 7f432a400640 1 Processor -- start 2024-05-09T04:33:29.517 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.518+0000 7f432a400640 1 -- start start 2024-05-09T04:33:29.520 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.519+0000 7f432a400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43240cfb60 0x7f43240b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:29.521 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.519+0000 7f432a400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f43240d2030 0x7f43240b2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:29.521 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.519+0000 7f432a400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f43241628e0 0x7f43240b2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:29.521 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.519+0000 7f432a400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4324162cc0 con 0x7f43241628e0 2024-05-09T04:33:29.521 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.520+0000 7f432a400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f43240b6580 con 0x7f43240cfb60 2024-05-09T04:33:29.521 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.520+0000 7f432a400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f43240d2860 con 0x7f43240d2030 2024-05-09T04:33:29.521 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.520+0000 7f4323e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f43240d2030 0x7f43240b2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:29.521 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.520+0000 7f4323e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f43240d2030 0x7f43240b2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.192:54814/0 (socket says 172.21.15.192:54814) 2024-05-09T04:33:29.521 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.520+0000 7f4323e00640 1 -- 172.21.15.192:0/3021284665 learned_addr learned my addr 172.21.15.192:0/3021284665 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-05-09T04:33:29.521 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.520+0000 7f4329a00640 1 --2- 172.21.15.192:0/3021284665 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f43241628e0 0x7f43240b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:29.521 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.520+0000 7f4323e00640 1 -- 172.21.15.192:0/3021284665 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43240cfb60 msgr2=0x7f43240b8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:29.521 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.520+0000 7f4329000640 1 --2- 172.21.15.192:0/3021284665 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43240cfb60 0x7f43240b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:29.521 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.520+0000 7f4323e00640 1 --2- 172.21.15.192:0/3021284665 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43240cfb60 0x7f43240b8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:29.521 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.520+0000 7f4323e00640 1 -- 172.21.15.192:0/3021284665 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f43241628e0 msgr2=0x7f43240b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:29.521 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.520+0000 7f4323e00640 1 --2- 172.21.15.192:0/3021284665 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f43241628e0 0x7f43240b2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:29.522 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.520+0000 7f4323e00640 1 -- 172.21.15.192:0/3021284665 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43240b9d50 con 0x7f43240d2030 2024-05-09T04:33:29.522 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.520+0000 7f4329a00640 1 --2- 172.21.15.192:0/3021284665 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f43241628e0 0x7f43240b2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:33:29.522 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.520+0000 7f4329000640 1 --2- 172.21.15.192:0/3021284665 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43240cfb60 0x7f43240b8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:33:29.522 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.521+0000 7f4323e00640 1 --2- 172.21.15.192:0/3021284665 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f43240d2030 0x7f43240b2310 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f431000b4a0 tx=0x7f431000b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:29.522 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.521+0000 7f4320c00640 1 -- 172.21.15.192:0/3021284665 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4310014070 con 0x7f43240d2030 2024-05-09T04:33:29.522 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.521+0000 7f4320c00640 1 -- 172.21.15.192:0/3021284665 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4310010610 con 0x7f43240d2030 2024-05-09T04:33:29.522 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.521+0000 7f4320c00640 1 -- 172.21.15.192:0/3021284665 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4310010a00 con 0x7f43240d2030 2024-05-09T04:33:29.522 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.521+0000 7f432a400640 1 -- 172.21.15.192:0/3021284665 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43240ba030 con 0x7f43240d2030 2024-05-09T04:33:29.522 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.521+0000 7f432a400640 1 -- 172.21.15.192:0/3021284665 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f43240ba5f0 con 0x7f43240d2030 2024-05-09T04:33:29.523 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.521+0000 7f4320c00640 1 -- 172.21.15.192:0/3021284665 <== mon.2 v2:172.21.15.192:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4310017070 con 0x7f43240d2030 2024-05-09T04:33:29.523 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.521+0000 7f4320c00640 1 -- 172.21.15.192:0/3021284665 <== mon.2 v2:172.21.15.192:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f4310010ba0 con 0x7f43240d2030 2024-05-09T04:33:29.524 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.522+0000 7f432a400640 1 -- 172.21.15.192:0/3021284665 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43240694e0 con 0x7f43240d2030 2024-05-09T04:33:29.524 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.524+0000 7f4320c00640 1 -- 172.21.15.192:0/3021284665 <== mon.2 v2:172.21.15.192:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f4310019450 con 0x7f43240d2030 2024-05-09T04:33:29.530 INFO:tasks.ceph.mgr.y.smithi145.stderr:2024-05-09T04:33:29.527+0000 7f5551720040 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-09T04:33:29.603 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.603+0000 7f432a400640 1 -- 172.21.15.192:0/3021284665 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "osd new", "uuid": "d05a3311-505a-45b5-85b9-aa3113e96987", "id": 6} v 0) -- 0x7f43240b9080 con 0x7f43240d2030 2024-05-09T04:33:29.614 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.615+0000 7f4320c00640 1 -- 172.21.15.192:0/3021284665 <== mon.2 v2:172.21.15.192:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d05a3311-505a-45b5-85b9-aa3113e96987", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f4310019630 con 0x7f43240d2030 2024-05-09T04:33:29.614 INFO:teuthology.orchestra.run.smithi192.stdout:6 2024-05-09T04:33:29.617 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.616+0000 7f432a400640 1 -- 172.21.15.192:0/3021284665 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f43240d2030 msgr2=0x7f43240b2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:29.617 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.616+0000 7f432a400640 1 --2- 172.21.15.192:0/3021284665 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f43240d2030 0x7f43240b2310 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f431000b4a0 tx=0x7f431000b960 comp rx=0 tx=0).stop 2024-05-09T04:33:29.617 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.616+0000 7f432a400640 1 -- 172.21.15.192:0/3021284665 shutdown_connections 2024-05-09T04:33:29.617 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.616+0000 7f432a400640 1 --2- 172.21.15.192:0/3021284665 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f43241628e0 0x7f43240b2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:29.617 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.616+0000 7f432a400640 1 --2- 172.21.15.192:0/3021284665 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f43240d2030 0x7f43240b2310 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:29.617 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.616+0000 7f432a400640 1 --2- 172.21.15.192:0/3021284665 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43240cfb60 0x7f43240b8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:29.617 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.616+0000 7f432a400640 1 -- 172.21.15.192:0/3021284665 >> 172.21.15.192:0/3021284665 conn(0x7f43240d4f50 msgr2=0x7f43240d5330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:29.617 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.616+0000 7f432a400640 1 -- 172.21.15.192:0/3021284665 shutdown_connections 2024-05-09T04:33:29.617 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.616+0000 7f432a400640 1 -- 172.21.15.192:0/3021284665 wait complete. 2024-05-09T04:33:29.631 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new c7c23dbc-2029-43ec-9f53-25f3d58019a5 7 2024-05-09T04:33:29.649 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-09T04:33:29.649+0000 7f2e3f522040 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-09T04:33:29.807 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.808+0000 7fea01200640 1 Processor -- start 2024-05-09T04:33:29.808 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.808+0000 7fea01200640 1 -- start start 2024-05-09T04:33:29.809 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.809+0000 7fea01200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe9fc0b3110 0x7fe9fc0d1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:29.809 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.809+0000 7fea01200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe9fc0b34f0 0x7fe9fc0ce960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:29.809 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.810+0000 7fea01200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9fc0b3ae0 con 0x7fe9fc0b3dc0 2024-05-09T04:33:29.809 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.810+0000 7fea01200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fe9fc0d3a70 con 0x7fe9fc0b34f0 2024-05-09T04:33:29.809 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.810+0000 7fea01200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe9fc0d0ee0 con 0x7fe9fc0b3110 2024-05-09T04:33:29.810 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.810+0000 7fe9fbe00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe9fc0b34f0 0x7fe9fc0ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:29.810 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.810+0000 7fe9fb400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe9fc0b3110 0x7fe9fc0d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:29.810 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.810+0000 7fe9fb400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe9fc0b3110 0x7fe9fc0d1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.192:54818/0 (socket says 172.21.15.192:54818) 2024-05-09T04:33:29.810 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.810+0000 7fe9fbe00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe9fc0b34f0 0x7fe9fc0ce960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:39860/0 (socket says 172.21.15.192:39860) 2024-05-09T04:33:29.810 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.810+0000 7fe9fb400640 1 -- 172.21.15.192:0/1064764704 learned_addr learned my addr 172.21.15.192:0/1064764704 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-05-09T04:33:29.810 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.810+0000 7fe9faa00640 1 --1- 172.21.15.192:0/1064764704 >> v1:172.21.15.192:6789/0 conn(0x7fe9fc0b3dc0 0x7fe9fc0c6890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.192:60298/0 (socket says 172.21.15.192:60298) 2024-05-09T04:33:29.811 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.810+0000 7fe9fbe00640 1 -- 172.21.15.192:0/1064764704 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe9fc0b3110 msgr2=0x7fe9fc0d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:29.811 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.810+0000 7fe9fbe00640 1 --2- 172.21.15.192:0/1064764704 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe9fc0b3110 0x7fe9fc0d1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:29.811 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.810+0000 7fe9fbe00640 1 -- 172.21.15.192:0/1064764704 >> v1:172.21.15.192:6789/0 conn(0x7fe9fc0b3dc0 legacy=0x7fe9fc0c6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:29.811 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.810+0000 7fe9fbe00640 1 -- 172.21.15.192:0/1064764704 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9fc0cfe80 con 0x7fe9fc0b34f0 2024-05-09T04:33:29.811 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.810+0000 7fe9fb400640 1 --2- 172.21.15.192:0/1064764704 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe9fc0b3110 0x7fe9fc0d1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:33:29.811 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.812+0000 7fe9fbe00640 1 --2- 172.21.15.192:0/1064764704 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9fc0b34f0 0x7fe9fc0ce960 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4009d90 tx=0x7fe9f402a4c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f62fd7afa53fadf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:29.811 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.812+0000 7fe9fa000640 1 -- 172.21.15.192:0/1064764704 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f4007740 con 0x7fe9fc0b34f0 2024-05-09T04:33:29.811 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.812+0000 7fe9fa000640 1 -- 172.21.15.192:0/1064764704 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9f40078e0 con 0x7fe9fc0b34f0 2024-05-09T04:33:29.812 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.812+0000 7fe9fa000640 1 -- 172.21.15.192:0/1064764704 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f4007cb0 con 0x7fe9fc0b34f0 2024-05-09T04:33:29.812 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.812+0000 7fea01200640 1 -- 172.21.15.192:0/1064764704 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9fc0b34f0 msgr2=0x7fe9fc0ce960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:29.812 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.812+0000 7fea01200640 1 --2- 172.21.15.192:0/1064764704 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9fc0b34f0 0x7fe9fc0ce960 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4009d90 tx=0x7fe9f402a4c0 comp rx=0 tx=0).stop 2024-05-09T04:33:29.813 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.813+0000 7fea01200640 1 -- 172.21.15.192:0/1064764704 shutdown_connections 2024-05-09T04:33:29.813 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.813+0000 7fea01200640 1 --2- 172.21.15.192:0/1064764704 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9fc0b34f0 0x7fe9fc0ce960 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:29.813 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.813+0000 7fea01200640 1 --2- 172.21.15.192:0/1064764704 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe9fc0b3110 0x7fe9fc0d1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:29.813 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.813+0000 7fea01200640 1 -- 172.21.15.192:0/1064764704 >> 172.21.15.192:0/1064764704 conn(0x7fe9fc0d2d40 msgr2=0x7fe9fc0d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:29.813 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.813+0000 7fea01200640 1 -- 172.21.15.192:0/1064764704 shutdown_connections 2024-05-09T04:33:29.813 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.813+0000 7fea01200640 1 -- 172.21.15.192:0/1064764704 wait complete. 2024-05-09T04:33:29.814 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.815+0000 7fea01200640 1 Processor -- start 2024-05-09T04:33:29.814 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.815+0000 7fea01200640 1 -- start start 2024-05-09T04:33:29.818 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.816+0000 7fea01200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9fc0b3110 0x7fe9fc0b0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:29.818 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.816+0000 7fea01200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe9fc0b34f0 0x7fe9fc06eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:29.819 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.816+0000 7fea01200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe9fc06efe0 0x7fe9fc1f4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:29.819 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.816+0000 7fea01200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe9fc0d1c10 con 0x7fe9fc0b34f0 2024-05-09T04:33:29.819 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.816+0000 7fea01200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe9fc0d1a90 con 0x7fe9fc0b3110 2024-05-09T04:33:29.819 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.816+0000 7fea01200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe9fc0d10e0 con 0x7fe9fc06efe0 2024-05-09T04:33:29.819 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.816+0000 7fe9fb400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9fc0b3110 0x7fe9fc0b0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:29.819 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.816+0000 7fe9fb400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9fc0b3110 0x7fe9fc0b0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:39864/0 (socket says 172.21.15.192:39864) 2024-05-09T04:33:29.819 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.816+0000 7fe9fb400640 1 -- 172.21.15.192:0/3590090073 learned_addr learned my addr 172.21.15.192:0/3590090073 (peer_addr_for_me v2:172.21.15.192:0/0) 2024-05-09T04:33:29.819 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.816+0000 7fe9fbe00640 1 --2- 172.21.15.192:0/3590090073 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe9fc06efe0 0x7fe9fc1f4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:29.819 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.816+0000 7fe9faa00640 1 --2- 172.21.15.192:0/3590090073 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe9fc0b34f0 0x7fe9fc06eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:29.820 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.817+0000 7fe9fb400640 1 -- 172.21.15.192:0/3590090073 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe9fc06efe0 msgr2=0x7fe9fc1f4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:29.820 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.817+0000 7fe9fb400640 1 --2- 172.21.15.192:0/3590090073 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe9fc06efe0 0x7fe9fc1f4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:29.820 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.817+0000 7fe9fb400640 1 -- 172.21.15.192:0/3590090073 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe9fc0b34f0 msgr2=0x7fe9fc06eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:29.820 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.817+0000 7fe9fb400640 1 --2- 172.21.15.192:0/3590090073 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe9fc0b34f0 0x7fe9fc06eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:29.820 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.817+0000 7fe9fb400640 1 -- 172.21.15.192:0/3590090073 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9fc06f5a0 con 0x7fe9fc0b3110 2024-05-09T04:33:29.820 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.817+0000 7fe9faa00640 1 --2- 172.21.15.192:0/3590090073 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe9fc0b34f0 0x7fe9fc06eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:33:29.820 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.817+0000 7fe9fb400640 1 --2- 172.21.15.192:0/3590090073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9fc0b3110 0x7fe9fc0b0c50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec00e9b0 tx=0x7fe9ec00ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:29.820 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.817+0000 7fe9f3400640 1 -- 172.21.15.192:0/3590090073 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec017070 con 0x7fe9fc0b3110 2024-05-09T04:33:29.820 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.817+0000 7fe9f3400640 1 -- 172.21.15.192:0/3590090073 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9ec00c9a0 con 0x7fe9fc0b3110 2024-05-09T04:33:29.820 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.817+0000 7fe9f3400640 1 -- 172.21.15.192:0/3590090073 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec00cd50 con 0x7fe9fc0b3110 2024-05-09T04:33:29.820 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.817+0000 7fea01200640 1 -- 172.21.15.192:0/3590090073 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9fc09cc40 con 0x7fe9fc0b3110 2024-05-09T04:33:29.821 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.817+0000 7fea01200640 1 -- 172.21.15.192:0/3590090073 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9fc09d200 con 0x7fe9fc0b3110 2024-05-09T04:33:29.821 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.818+0000 7fe9f3400640 1 -- 172.21.15.192:0/3590090073 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe9ec01a070 con 0x7fe9fc0b3110 2024-05-09T04:33:29.821 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.818+0000 7fe9f3400640 1 -- 172.21.15.192:0/3590090073 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7fe9ec013ad0 con 0x7fe9fc0b3110 2024-05-09T04:33:29.821 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.818+0000 7fea01200640 1 -- 172.21.15.192:0/3590090073 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9fc09d640 con 0x7fe9fc0b3110 2024-05-09T04:33:29.822 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.821+0000 7fe9f3400640 1 -- 172.21.15.192:0/3590090073 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fe9ec013470 con 0x7fe9fc0b3110 2024-05-09T04:33:29.826 INFO:tasks.ceph.mgr.y.smithi145.stderr:2024-05-09T04:33:29.826+0000 7f5551720040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T04:33:29.888 INFO:tasks.ceph.mgr.y.smithi145.stderr:2024-05-09T04:33:29.888+0000 7f5551720040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T04:33:29.898 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.899+0000 7fea01200640 1 -- 172.21.15.192:0/3590090073 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c7c23dbc-2029-43ec-9f53-25f3d58019a5", "id": 7} v 0) -- 0x7fe9fc0694e0 con 0x7fe9fc0b3110 2024-05-09T04:33:29.910 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.911+0000 7fe9f3400640 1 -- 172.21.15.192:0/3590090073 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c7c23dbc-2029-43ec-9f53-25f3d58019a5", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7fe9ec013650 con 0x7fe9fc0b3110 2024-05-09T04:33:29.911 INFO:teuthology.orchestra.run.smithi192.stdout:7 2024-05-09T04:33:29.911 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.912+0000 7fea01200640 1 -- 172.21.15.192:0/3590090073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9fc0b3110 msgr2=0x7fe9fc0b0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:29.912 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.912+0000 7fea01200640 1 --2- 172.21.15.192:0/3590090073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9fc0b3110 0x7fe9fc0b0c50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec00e9b0 tx=0x7fe9ec00ee70 comp rx=0 tx=0).stop 2024-05-09T04:33:29.912 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.912+0000 7fea01200640 1 -- 172.21.15.192:0/3590090073 shutdown_connections 2024-05-09T04:33:29.912 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.912+0000 7fea01200640 1 --2- 172.21.15.192:0/3590090073 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe9fc06efe0 0x7fe9fc1f4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:29.912 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.912+0000 7fea01200640 1 --2- 172.21.15.192:0/3590090073 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe9fc0b34f0 0x7fe9fc06eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:29.912 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.912+0000 7fea01200640 1 --2- 172.21.15.192:0/3590090073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9fc0b3110 0x7fe9fc0b0c50 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:29.912 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.912+0000 7fea01200640 1 -- 172.21.15.192:0/3590090073 >> 172.21.15.192:0/3590090073 conn(0x7fe9fc0d2d40 msgr2=0x7fe9fc071b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:29.912 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.912+0000 7fea01200640 1 -- 172.21.15.192:0/3590090073 shutdown_connections 2024-05-09T04:33:29.912 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:33:29.912+0000 7fea01200640 1 -- 172.21.15.192:0/3590090073 wait complete. 2024-05-09T04:33:29.928 INFO:tasks.ceph.osd.0:Restarting daemon 2024-05-09T04:33:29.928 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-05-09T04:33:29.935 INFO:tasks.ceph.osd.0:Started 2024-05-09T04:33:29.935 INFO:tasks.ceph.osd.1:Restarting daemon 2024-05-09T04:33:29.935 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-05-09T04:33:29.942 INFO:tasks.ceph.osd.1:Started 2024-05-09T04:33:29.942 INFO:tasks.ceph.osd.2:Restarting daemon 2024-05-09T04:33:29.942 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-05-09T04:33:29.948 INFO:tasks.ceph.osd.2:Started 2024-05-09T04:33:29.948 INFO:tasks.ceph.osd.3:Restarting daemon 2024-05-09T04:33:29.948 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-05-09T04:33:29.953 INFO:tasks.ceph.osd.3:Started 2024-05-09T04:33:29.953 INFO:tasks.ceph.osd.4:Restarting daemon 2024-05-09T04:33:29.953 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-05-09T04:33:29.964 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-09T04:33:29.965+0000 7f2e3f522040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T04:33:29.968 INFO:tasks.ceph.osd.4:Started 2024-05-09T04:33:29.969 INFO:tasks.ceph.osd.5:Restarting daemon 2024-05-09T04:33:29.969 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-05-09T04:33:29.973 INFO:tasks.ceph.osd.5:Started 2024-05-09T04:33:29.973 INFO:tasks.ceph.osd.6:Restarting daemon 2024-05-09T04:33:29.973 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-05-09T04:33:29.977 INFO:tasks.ceph.osd.6:Started 2024-05-09T04:33:29.977 INFO:tasks.ceph.osd.7:Restarting daemon 2024-05-09T04:33:29.977 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-05-09T04:33:29.982 INFO:tasks.ceph.osd.7:Started 2024-05-09T04:33:29.982 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T04:33:30.044 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-09T04:33:30.045+0000 7f2e3f522040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T04:33:30.064 INFO:tasks.ceph.mgr.y.smithi145.stderr:2024-05-09T04:33:30.064+0000 7f5551720040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T04:33:30.202 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.202+0000 7f0583e00640 1 Processor -- start 2024-05-09T04:33:30.202 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.202+0000 7f0583e00640 1 -- start start 2024-05-09T04:33:30.204 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.203+0000 7f0583e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f05840d3180 0x7f05840d5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:30.204 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.203+0000 7f0583e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f05840b4760 0x7f05840b4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:30.204 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.203+0000 7f0583e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05840d3e70 con 0x7f05840d22f0 2024-05-09T04:33:30.204 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.203+0000 7f0583e00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f05840d4150 con 0x7f05840d3180 2024-05-09T04:33:30.204 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.203+0000 7f0583e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f05840b6200 con 0x7f05840b4760 2024-05-09T04:33:30.204 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.203+0000 7f0582a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f05840d3180 0x7f05840d5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:30.204 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.203+0000 7f0582a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f05840d3180 0x7f05840d5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:52870/0 (socket says 172.21.15.145:52870) 2024-05-09T04:33:30.204 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.203+0000 7f0582a00640 1 -- 172.21.15.145:0/3973532099 learned_addr learned my addr 172.21.15.145:0/3973532099 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:30.204 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.203+0000 7f0583400640 1 --2- 172.21.15.145:0/3973532099 >> v2:172.21.15.192:3300/0 conn(0x7f05840b4760 0x7f05840b4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:30.204 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.204+0000 7f0582a00640 1 -- 172.21.15.145:0/3973532099 >> v2:172.21.15.192:3300/0 conn(0x7f05840b4760 msgr2=0x7f05840b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:30.204 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.204+0000 7f0582a00640 1 --2- 172.21.15.145:0/3973532099 >> v2:172.21.15.192:3300/0 conn(0x7f05840b4760 0x7f05840b4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:30.204 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.204+0000 7f0582a00640 1 -- 172.21.15.145:0/3973532099 >> v1:172.21.15.192:6789/0 conn(0x7f05840d22f0 legacy=0x7f05840d3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:30.205 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.204+0000 7f0582a00640 1 -- 172.21.15.145:0/3973532099 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05840b6450 con 0x7f05840d3180 2024-05-09T04:33:30.206 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.205+0000 7f0582a00640 1 --2- 172.21.15.145:0/3973532099 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f05840d3180 0x7f05840d5ad0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f05840d4b10 tx=0x7f057402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9162daad9ca7f79 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:30.206 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.205+0000 7f0581600640 1 -- 172.21.15.145:0/3973532099 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0574037070 con 0x7f05840d3180 2024-05-09T04:33:30.206 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.205+0000 7f0581600640 1 -- 172.21.15.145:0/3973532099 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0574032440 con 0x7f05840d3180 2024-05-09T04:33:30.206 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.205+0000 7f0581600640 1 -- 172.21.15.145:0/3973532099 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0574032810 con 0x7f05840d3180 2024-05-09T04:33:30.206 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.205+0000 7f0583e00640 1 -- 172.21.15.145:0/3973532099 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f05840d3180 msgr2=0x7f05840d5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:30.206 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.205+0000 7f0583e00640 1 --2- 172.21.15.145:0/3973532099 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f05840d3180 0x7f05840d5ad0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f05840d4b10 tx=0x7f057402a1e0 comp rx=0 tx=0).stop 2024-05-09T04:33:30.206 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.206+0000 7f0583e00640 1 -- 172.21.15.145:0/3973532099 shutdown_connections 2024-05-09T04:33:30.206 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.206+0000 7f0583e00640 1 --2- 172.21.15.145:0/3973532099 >> v2:172.21.15.192:3300/0 conn(0x7f05840b4760 0x7f05840b4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:30.207 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.206+0000 7f0583e00640 1 --2- 172.21.15.145:0/3973532099 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f05840d3180 0x7f05840d5ad0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:30.207 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.206+0000 7f0583e00640 1 -- 172.21.15.145:0/3973532099 >> 172.21.15.145:0/3973532099 conn(0x7f05840d4fb0 msgr2=0x7f05840d53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:30.207 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.206+0000 7f0583e00640 1 -- 172.21.15.145:0/3973532099 shutdown_connections 2024-05-09T04:33:30.207 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.206+0000 7f0583e00640 1 -- 172.21.15.145:0/3973532099 wait complete. 2024-05-09T04:33:30.209 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.207+0000 7f0583e00640 1 Processor -- start 2024-05-09T04:33:30.209 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.207+0000 7f0583e00640 1 -- start start 2024-05-09T04:33:30.209 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.208+0000 7f0583e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05840b4760 0x7f0584153750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:30.209 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.208+0000 7f0583e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05840d22f0 0x7f05840cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:30.209 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.208+0000 7f0583e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f05840d3180 0x7f05840c8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:30.209 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.208+0000 7f0583e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f05840d2780 con 0x7f05840d3180 2024-05-09T04:33:30.209 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.208+0000 7f0583e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f05840d1e60 con 0x7f05840b4760 2024-05-09T04:33:30.210 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.208+0000 7f0583e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f05840b3fa0 con 0x7f05840d22f0 2024-05-09T04:33:30.210 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.209+0000 7f0582a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05840b4760 0x7f0584153750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:30.210 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.209+0000 7f0583400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f05840d3180 0x7f05840c8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:30.210 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.209+0000 7f0583400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f05840d3180 0x7f05840c8ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:52884/0 (socket says 172.21.15.145:52884) 2024-05-09T04:33:30.210 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.209+0000 7f0583400640 1 -- 172.21.15.145:0/627856538 learned_addr learned my addr 172.21.15.145:0/627856538 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:30.210 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.209+0000 7f0582000640 1 --2- 172.21.15.145:0/627856538 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05840d22f0 0x7f05840cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:30.210 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.209+0000 7f0583400640 1 -- 172.21.15.145:0/627856538 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05840d22f0 msgr2=0x7f05840cb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:30.210 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.209+0000 7f0583400640 1 --2- 172.21.15.145:0/627856538 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05840d22f0 0x7f05840cb5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:30.210 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.209+0000 7f0583400640 1 -- 172.21.15.145:0/627856538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05840b4760 msgr2=0x7f0584153750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:30.210 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.209+0000 7f0583400640 1 --2- 172.21.15.145:0/627856538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05840b4760 0x7f0584153750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:30.210 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.209+0000 7f0583400640 1 -- 172.21.15.145:0/627856538 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05840c9210 con 0x7f05840d3180 2024-05-09T04:33:30.211 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.209+0000 7f0583400640 1 --2- 172.21.15.145:0/627856538 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f05840d3180 0x7f05840c8ae0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f057c009dd0 tx=0x7f057c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:30.211 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-09T04:33:30.211+0000 7f2e3f522040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T04:33:30.212 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.210+0000 7f057aa00640 1 -- 172.21.15.145:0/627856538 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057c018020 con 0x7f05840d3180 2024-05-09T04:33:30.212 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.210+0000 7f057aa00640 1 -- 172.21.15.145:0/627856538 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f057c014480 con 0x7f05840d3180 2024-05-09T04:33:30.212 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.210+0000 7f057aa00640 1 -- 172.21.15.145:0/627856538 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057c014870 con 0x7f05840d3180 2024-05-09T04:33:30.212 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.210+0000 7f0583e00640 1 -- 172.21.15.145:0/627856538 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05840c94f0 con 0x7f05840d3180 2024-05-09T04:33:30.212 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.210+0000 7f0583e00640 1 -- 172.21.15.145:0/627856538 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05840c9ab0 con 0x7f05840d3180 2024-05-09T04:33:30.212 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.210+0000 7f057aa00640 1 -- 172.21.15.145:0/627856538 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f057c01b020 con 0x7f05840d3180 2024-05-09T04:33:30.212 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.210+0000 7f057aa00640 1 -- 172.21.15.145:0/627856538 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f057c014da0 con 0x7f05840d3180 2024-05-09T04:33:30.212 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.211+0000 7f0583e00640 1 -- 172.21.15.145:0/627856538 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f058406a310 con 0x7f05840d3180 2024-05-09T04:33:30.213 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.213+0000 7f057aa00640 1 -- 172.21.15.145:0/627856538 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f057c022020 con 0x7f05840d3180 2024-05-09T04:33:30.280 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.281+0000 7f0583e00640 1 -- 172.21.15.145:0/627856538 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f05840d0770 con 0x7f05840d3180 2024-05-09T04:33:30.281 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.281+0000 7f057aa00640 1 -- 172.21.15.145:0/627856538 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f057c010040 con 0x7f05840d3180 2024-05-09T04:33:30.281 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:33:30.282 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":10,"fsid":"957ee6c1-073b-44cf-9b93-9491b3fdfc1d","created":"2024-05-09T04:33:26.544367+0000","modified":"2024-05-09T04:33:29.902361+0000","last_up_change":"0.000000","last_in_change":"2024-05-09T04:33:29.902361+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"22e550ad-847b-4952-80e6-63d859ed1a0f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"0f6b9fca-0c41-4145-8d74-598ddb42f2a3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b3b1f67e-5790-483f-8610-e05a5bce4c30","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"884787a5-e037-4d31-9aae-743030eba4ce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"0759fc1e-c5fc-4a2f-86bb-e8eccd83f833","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9443cc22-9379-46e3-9173-98b592bf2c16","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d05a3311-505a-45b5-85b9-aa3113e96987","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"c7c23dbc-2029-43ec-9f53-25f3d58019a5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T04:33:30.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.283+0000 7f0583e00640 1 -- 172.21.15.145:0/627856538 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f05840d3180 msgr2=0x7f05840c8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:30.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.283+0000 7f0583e00640 1 --2- 172.21.15.145:0/627856538 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f05840d3180 0x7f05840c8ae0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f057c009dd0 tx=0x7f057c00c490 comp rx=0 tx=0).stop 2024-05-09T04:33:30.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.283+0000 7f0583e00640 1 -- 172.21.15.145:0/627856538 shutdown_connections 2024-05-09T04:33:30.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.283+0000 7f0583e00640 1 --2- 172.21.15.145:0/627856538 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f05840d3180 0x7f05840c8ae0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:30.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.283+0000 7f0583e00640 1 --2- 172.21.15.145:0/627856538 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05840d22f0 0x7f05840cb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:30.285 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.283+0000 7f0583e00640 1 --2- 172.21.15.145:0/627856538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05840b4760 0x7f0584153750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:30.285 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.283+0000 7f0583e00640 1 -- 172.21.15.145:0/627856538 >> 172.21.15.145:0/627856538 conn(0x7f05840d4fb0 msgr2=0x7f05840cfc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:30.285 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.283+0000 7f0583e00640 1 -- 172.21.15.145:0/627856538 shutdown_connections 2024-05-09T04:33:30.285 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:30.283+0000 7f0583e00640 1 -- 172.21.15.145:0/627856538 wait complete. 2024-05-09T04:33:30.299 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-05-09T04:33:30.299 INFO:tasks.ceph:Waiting for OSDs to come up 2024-05-09T04:33:30.427 INFO:tasks.ceph.mgr.y.smithi145.stderr:2024-05-09T04:33:30.428+0000 7f5551720040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T04:33:30.484 INFO:tasks.ceph.mgr.y.smithi145.stderr:2024-05-09T04:33:30.484+0000 7f5551720040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T04:33:30.547 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-09T04:33:30.547+0000 7f2e3f522040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T04:33:30.604 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-09T04:33:30.604+0000 7f2e3f522040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T04:33:30.638 INFO:tasks.ceph.osd.0.smithi145.stderr:2024-05-09T04:33:30.638+0000 7f5e0f414640 -1 Falling back to public interface 2024-05-09T04:33:30.643 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-09T04:33:30.643+0000 7f66643fc640 -1 Falling back to public interface 2024-05-09T04:33:30.644 INFO:tasks.ceph.osd.3.smithi145.stderr:2024-05-09T04:33:30.645+0000 7f8d7981b640 -1 Falling back to public interface 2024-05-09T04:33:30.648 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-05-09T04:33:30.649+0000 7f688e613640 -1 Falling back to public interface 2024-05-09T04:33:30.654 INFO:tasks.ceph.osd.2.smithi145.stderr:2024-05-09T04:33:30.654+0000 7f2fa7c3c640 -1 Falling back to public interface 2024-05-09T04:33:30.664 INFO:tasks.ceph.osd.1.smithi145.stderr:2024-05-09T04:33:30.665+0000 7f252c21a640 -1 Falling back to public interface 2024-05-09T04:33:30.671 INFO:tasks.ceph.mgr.y.smithi145.stderr:2024-05-09T04:33:30.672+0000 7f5551720040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T04:33:30.686 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-05-09T04:33:30.686+0000 7f8c00157640 -1 Falling back to public interface 2024-05-09T04:33:30.686 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-09T04:33:30.686+0000 7fc9eae0f640 -1 Falling back to public interface 2024-05-09T04:33:30.795 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-09T04:33:30.795+0000 7f2e3f522040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T04:33:30.842 INFO:tasks.ceph.mgr.y.smithi145.stderr:2024-05-09T04:33:30.840+0000 7f5551720040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T04:33:30.899 INFO:tasks.ceph.mgr.y.smithi145.stderr:2024-05-09T04:33:30.899+0000 7f5551720040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T04:33:30.961 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-09T04:33:30.962+0000 7f2e3f522040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T04:33:30.982 INFO:tasks.ceph.mgr.y.smithi145.stderr:2024-05-09T04:33:30.982+0000 7f5551720040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T04:33:31.022 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-09T04:33:31.023+0000 7f2e3f522040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T04:33:31.037 INFO:tasks.ceph.mgr.y.smithi145.stderr:2024-05-09T04:33:31.037+0000 7f5551720040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T04:33:31.101 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-09T04:33:31.102+0000 7f2e3f522040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T04:33:31.103 DEBUG:teuthology.orchestra.run.smithi145:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-09T04:33:31.153 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-09T04:33:31.154+0000 7f2e3f522040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T04:33:31.263 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.263+0000 7f7853200640 1 Processor -- start 2024-05-09T04:33:31.264 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.264+0000 7f7853200640 1 -- start start 2024-05-09T04:33:31.266 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.266+0000 7f7853200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f784c06fc90 0x7f784c09ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:31.266 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.266+0000 7f7853200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f784c0cc8b0 0x7f784c06e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:31.267 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.266+0000 7f7853200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f784c074350 con 0x7f784c070070 2024-05-09T04:33:31.267 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.266+0000 7f7853200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f784c0cf510 con 0x7f784c06fc90 2024-05-09T04:33:31.267 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.266+0000 7f7853200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f784c0cef50 con 0x7f784c0cc8b0 2024-05-09T04:33:31.267 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.266+0000 7f784b400640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f784c070070 0x7f784c0cc1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:40726/0 (socket says 172.21.15.145:40726) 2024-05-09T04:33:31.267 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.266+0000 7f784b400640 1 -- 172.21.15.145:0/3029792370 learned_addr learned my addr 172.21.15.145:0/3029792370 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:33:31.267 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.266+0000 7f7851200640 1 --2- 172.21.15.145:0/3029792370 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f784c0cc8b0 0x7f784c06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:31.267 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.266+0000 7f784be00640 1 --2- 172.21.15.145:0/3029792370 >> v2:172.21.15.192:3300/0 conn(0x7f784c06fc90 0x7f784c09ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:31.267 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.266+0000 7f784aa00640 1 -- 172.21.15.145:0/3029792370 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 899315533 0 0) 0x7f784c074350 con 0x7f784c070070 2024-05-09T04:33:31.267 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.266+0000 7f784aa00640 1 -- 172.21.15.145:0/3029792370 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7830003520 con 0x7f784c070070 2024-05-09T04:33:31.267 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.267+0000 7f7851200640 1 -- 172.21.15.145:0/3029792370 >> v2:172.21.15.192:3300/0 conn(0x7f784c06fc90 msgr2=0x7f784c09ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:31.267 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.267+0000 7f7851200640 1 --2- 172.21.15.145:0/3029792370 >> v2:172.21.15.192:3300/0 conn(0x7f784c06fc90 0x7f784c09ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:31.267 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.267+0000 7f7851200640 1 -- 172.21.15.145:0/3029792370 >> v1:172.21.15.145:6789/0 conn(0x7f784c070070 legacy=0x7f784c0cc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:31.267 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.267+0000 7f7851200640 1 -- 172.21.15.145:0/3029792370 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f784c0cf230 con 0x7f784c0cc8b0 2024-05-09T04:33:31.269 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.268+0000 7f7851200640 1 --2- 172.21.15.145:0/3029792370 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f784c0cc8b0 0x7f784c06e370 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7844009d90 tx=0x7f784402a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41cd7c66ac0fb339 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:31.269 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.268+0000 7f784aa00640 1 -- 172.21.15.145:0/3029792370 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7844031600 con 0x7f784c0cc8b0 2024-05-09T04:33:31.269 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.268+0000 7f784aa00640 1 -- 172.21.15.145:0/3029792370 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78440317a0 con 0x7f784c0cc8b0 2024-05-09T04:33:31.269 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.268+0000 7f784aa00640 1 -- 172.21.15.145:0/3029792370 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7844031b70 con 0x7f784c0cc8b0 2024-05-09T04:33:31.269 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.268+0000 7f7853200640 1 -- 172.21.15.145:0/3029792370 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f784c0cc8b0 msgr2=0x7f784c06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:31.269 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.268+0000 7f7853200640 1 --2- 172.21.15.145:0/3029792370 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f784c0cc8b0 0x7f784c06e370 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7844009d90 tx=0x7f784402a5d0 comp rx=0 tx=0).stop 2024-05-09T04:33:31.269 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.268+0000 7f7853200640 1 -- 172.21.15.145:0/3029792370 shutdown_connections 2024-05-09T04:33:31.269 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.268+0000 7f7853200640 1 --2- 172.21.15.145:0/3029792370 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f784c0cc8b0 0x7f784c06e370 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:31.269 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.268+0000 7f7853200640 1 --2- 172.21.15.145:0/3029792370 >> v2:172.21.15.192:3300/0 conn(0x7f784c06fc90 0x7f784c09ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:31.269 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.268+0000 7f7853200640 1 -- 172.21.15.145:0/3029792370 >> 172.21.15.145:0/3029792370 conn(0x7f784c0d0fb0 msgr2=0x7f784c0d1390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:31.270 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.269+0000 7f7853200640 1 -- 172.21.15.145:0/3029792370 shutdown_connections 2024-05-09T04:33:31.270 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.269+0000 7f7853200640 1 -- 172.21.15.145:0/3029792370 wait complete. 2024-05-09T04:33:31.270 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.270+0000 7f7853200640 1 Processor -- start 2024-05-09T04:33:31.270 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.270+0000 7f7853200640 1 -- start start 2024-05-09T04:33:31.272 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.271+0000 7f7853200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f784c06fc90 0x7f784c0c8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:31.272 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.271+0000 7f7853200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f784c070070 0x7f784c0b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:31.272 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.271+0000 7f7853200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f784c0cc8b0 0x7f784c0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:31.272 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.271+0000 7f7853200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f784c0ccf20 con 0x7f784c0cc8b0 2024-05-09T04:33:31.272 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.271+0000 7f7853200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f784c073e30 con 0x7f784c06fc90 2024-05-09T04:33:31.272 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.271+0000 7f7853200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f784c0cb920 con 0x7f784c070070 2024-05-09T04:33:31.273 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.271+0000 7f7851200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f784c0cc8b0 0x7f784c0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:31.273 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.271+0000 7f7851200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f784c0cc8b0 0x7f784c0b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:46588/0 (socket says 172.21.15.145:46588) 2024-05-09T04:33:31.273 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.271+0000 7f7851200640 1 -- 172.21.15.145:0/3511631729 learned_addr learned my addr 172.21.15.145:0/3511631729 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:31.273 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.271+0000 7f784b400640 1 --2- 172.21.15.145:0/3511631729 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f784c070070 0x7f784c0b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:31.273 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.271+0000 7f784be00640 1 --2- 172.21.15.145:0/3511631729 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f784c06fc90 0x7f784c0c8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:31.273 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.272+0000 7f7851200640 1 -- 172.21.15.145:0/3511631729 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f784c070070 msgr2=0x7f784c0b3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:31.273 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.272+0000 7f7851200640 1 --2- 172.21.15.145:0/3511631729 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f784c070070 0x7f784c0b3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:31.273 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.272+0000 7f7851200640 1 -- 172.21.15.145:0/3511631729 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f784c06fc90 msgr2=0x7f784c0c8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:31.273 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.272+0000 7f7851200640 1 --2- 172.21.15.145:0/3511631729 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f784c06fc90 0x7f784c0c8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:31.273 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.272+0000 7f7851200640 1 -- 172.21.15.145:0/3511631729 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f784c0b4510 con 0x7f784c0cc8b0 2024-05-09T04:33:31.273 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.272+0000 7f7851200640 1 --2- 172.21.15.145:0/3511631729 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f784c0cc8b0 0x7f784c0b5050 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f784402abc0 tx=0x7f7844004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:31.273 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.272+0000 7f7843e00640 1 -- 172.21.15.145:0/3511631729 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7844040070 con 0x7f784c0cc8b0 2024-05-09T04:33:31.273 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.272+0000 7f7843e00640 1 -- 172.21.15.145:0/3511631729 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f784403c830 con 0x7f784c0cc8b0 2024-05-09T04:33:31.274 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.272+0000 7f7853200640 1 -- 172.21.15.145:0/3511631729 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f784c0b47f0 con 0x7f784c0cc8b0 2024-05-09T04:33:31.274 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.272+0000 7f7843e00640 1 -- 172.21.15.145:0/3511631729 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784403cbe0 con 0x7f784c0cc8b0 2024-05-09T04:33:31.274 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.273+0000 7f7853200640 1 -- 172.21.15.145:0/3511631729 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f784c0caaf0 con 0x7f784c0cc8b0 2024-05-09T04:33:31.275 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.273+0000 7f7843e00640 1 -- 172.21.15.145:0/3511631729 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7844039040 con 0x7f784c0cc8b0 2024-05-09T04:33:31.275 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.273+0000 7f7843e00640 1 -- 172.21.15.145:0/3511631729 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f7844045c30 con 0x7f784c0cc8b0 2024-05-09T04:33:31.275 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.274+0000 7f7853200640 1 -- 172.21.15.145:0/3511631729 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f784c0768f0 con 0x7f784c0cc8b0 2024-05-09T04:33:31.276 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.276+0000 7f7843e00640 1 -- 172.21.15.145:0/3511631729 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f7844031730 con 0x7f784c0cc8b0 2024-05-09T04:33:31.341 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.341+0000 7f7853200640 1 -- 172.21.15.145:0/3511631729 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f784c15c540 con 0x7f784c0cc8b0 2024-05-09T04:33:31.342 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.342+0000 7f7843e00640 1 -- 172.21.15.145:0/3511631729 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f784403cd80 con 0x7f784c0cc8b0 2024-05-09T04:33:31.342 INFO:teuthology.misc.health.smithi145.stdout: 2024-05-09T04:33:31.342 INFO:teuthology.misc.health.smithi145.stdout:{"epoch":10,"fsid":"957ee6c1-073b-44cf-9b93-9491b3fdfc1d","created":"2024-05-09T04:33:26.544367+0000","modified":"2024-05-09T04:33:29.902361+0000","last_up_change":"0.000000","last_in_change":"2024-05-09T04:33:29.902361+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"22e550ad-847b-4952-80e6-63d859ed1a0f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"0f6b9fca-0c41-4145-8d74-598ddb42f2a3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b3b1f67e-5790-483f-8610-e05a5bce4c30","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"884787a5-e037-4d31-9aae-743030eba4ce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"0759fc1e-c5fc-4a2f-86bb-e8eccd83f833","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9443cc22-9379-46e3-9173-98b592bf2c16","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d05a3311-505a-45b5-85b9-aa3113e96987","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"c7c23dbc-2029-43ec-9f53-25f3d58019a5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T04:33:31.343 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.343+0000 7f7853200640 1 -- 172.21.15.145:0/3511631729 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f784c0cc8b0 msgr2=0x7f784c0b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:31.343 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.343+0000 7f7853200640 1 --2- 172.21.15.145:0/3511631729 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f784c0cc8b0 0x7f784c0b5050 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f784402abc0 tx=0x7f7844004a40 comp rx=0 tx=0).stop 2024-05-09T04:33:31.343 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.343+0000 7f7853200640 1 -- 172.21.15.145:0/3511631729 shutdown_connections 2024-05-09T04:33:31.344 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.344+0000 7f7853200640 1 --2- 172.21.15.145:0/3511631729 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f784c0cc8b0 0x7f784c0b5050 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:31.344 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.344+0000 7f7853200640 1 --2- 172.21.15.145:0/3511631729 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f784c070070 0x7f784c0b3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:31.344 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.344+0000 7f7853200640 1 --2- 172.21.15.145:0/3511631729 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f784c06fc90 0x7f784c0c8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:31.344 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.344+0000 7f7853200640 1 -- 172.21.15.145:0/3511631729 >> 172.21.15.145:0/3511631729 conn(0x7f784c0d0fb0 msgr2=0x7f784c15f570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:31.344 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.344+0000 7f7853200640 1 -- 172.21.15.145:0/3511631729 shutdown_connections 2024-05-09T04:33:31.344 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:31.344+0000 7f7853200640 1 -- 172.21.15.145:0/3511631729 wait complete. 2024-05-09T04:33:31.356 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-05-09T04:33:31.598 INFO:tasks.ceph.mgr.y.smithi145.stderr:2024-05-09T04:33:31.598+0000 7f5551720040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T04:33:31.696 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-09T04:33:31.697+0000 7f2e3f522040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T04:33:31.728 INFO:tasks.ceph.mgr.y.smithi145.stderr:2024-05-09T04:33:31.728+0000 7f5551720040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T04:33:31.837 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-05-09T04:33:31.837+0000 7f2e3f522040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T04:33:32.768 INFO:tasks.ceph.mgr.y.smithi145.stderr:2024-05-09T04:33:32.768+0000 7f54f0a00640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-05-09T04:33:32.769 INFO:tasks.ceph.mgr.y.smithi145.stderr:2024-05-09T04:33:32.770+0000 7f54f0a00640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-05-09T04:33:33.463 DEBUG:teuthology.orchestra.run.smithi145:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T04:33:33.469 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T04:33:33.816 INFO:tasks.ceph.osd.2.smithi145.stderr:2024-05-09T04:33:33.817+0000 7f2fa7c3c640 -1 osd.2 0 log_to_monitors true 2024-05-09T04:33:34.034 INFO:tasks.ceph.osd.1.smithi145.stderr:2024-05-09T04:33:34.034+0000 7f252c21a640 -1 osd.1 0 log_to_monitors true 2024-05-09T04:33:34.058 INFO:tasks.ceph.osd.0.smithi145.stderr:2024-05-09T04:33:34.058+0000 7f5e0f414640 -1 osd.0 0 log_to_monitors true 2024-05-09T04:33:34.268 INFO:tasks.ceph.osd.3.smithi145.stderr:2024-05-09T04:33:34.268+0000 7f8d7981b640 -1 osd.3 0 log_to_monitors true 2024-05-09T04:33:34.305 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-05-09T04:33:34.305+0000 7f688e613640 -1 osd.7 0 log_to_monitors true 2024-05-09T04:33:34.494 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-09T04:33:34.494+0000 7f66643fc640 -1 osd.5 0 log_to_monitors true 2024-05-09T04:33:34.568 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-05-09T04:33:34.568+0000 7f8c00157640 -1 osd.6 0 log_to_monitors true 2024-05-09T04:33:34.568 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-09T04:33:34.569+0000 7fc9eae0f640 -1 osd.4 0 log_to_monitors true 2024-05-09T04:33:38.031 INFO:tasks.ceph.osd.2.smithi145.stderr:2024-05-09T04:33:38.032+0000 7f2fa2a00640 -1 osd.2 0 waiting for initial osdmap 2024-05-09T04:33:38.034 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-09T04:33:38.035+0000 7f665f200640 -1 osd.5 0 waiting for initial osdmap 2024-05-09T04:33:38.041 INFO:tasks.ceph.osd.2.smithi145.stderr:2024-05-09T04:33:38.041+0000 7f2f98a00640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T04:33:38.045 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-05-09T04:33:38.046+0000 7f6655200640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T04:33:38.071 INFO:tasks.ceph.osd.0.smithi145.stderr:2024-05-09T04:33:38.072+0000 7f5e09e00640 -1 osd.0 0 waiting for initial osdmap 2024-05-09T04:33:38.076 INFO:tasks.ceph.osd.0.smithi145.stderr:2024-05-09T04:33:38.077+0000 7f5dffe00640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T04:33:38.088 INFO:tasks.ceph.osd.3.smithi145.stderr:2024-05-09T04:33:38.089+0000 7f8d74600640 -1 osd.3 0 waiting for initial osdmap 2024-05-09T04:33:38.094 INFO:tasks.ceph.osd.1.smithi145.stderr:2024-05-09T04:33:38.095+0000 7f2526c00640 -1 osd.1 0 waiting for initial osdmap 2024-05-09T04:33:38.100 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-05-09T04:33:38.101+0000 7f6888e00640 -1 osd.7 0 waiting for initial osdmap 2024-05-09T04:33:38.100 INFO:tasks.ceph.osd.3.smithi145.stderr:2024-05-09T04:33:38.101+0000 7f8d6a600640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T04:33:38.104 INFO:tasks.ceph.osd.1.smithi145.stderr:2024-05-09T04:33:38.105+0000 7f251cc00640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T04:33:38.109 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-05-09T04:33:38.110+0000 7f687ee00640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T04:33:38.129 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-09T04:33:38.130+0000 7fc9e6000640 -1 osd.4 0 waiting for initial osdmap 2024-05-09T04:33:38.134 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-05-09T04:33:38.135+0000 7f8bfb000640 -1 osd.6 0 waiting for initial osdmap 2024-05-09T04:33:38.136 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-05-09T04:33:38.137+0000 7fc9db600640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T04:33:38.151 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-05-09T04:33:38.151+0000 7f8bf1000640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T04:33:38.161 DEBUG:teuthology.orchestra.run.smithi145:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-09T04:33:38.310 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.310+0000 7f12ed400640 1 Processor -- start 2024-05-09T04:33:38.310 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.310+0000 7f12ed400640 1 -- start start 2024-05-09T04:33:38.311 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.311+0000 7f12ed400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f12e80d18e0 0x7f12e815f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:38.311 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.311+0000 7f12ed400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f12e815f590 0x7f12e80d1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:38.311 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.312+0000 7f12ed400640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12e80b42d0 con 0x7f12e80d2200 2024-05-09T04:33:38.311 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.312+0000 7f12ed400640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f12e80b1c60 con 0x7f12e80d18e0 2024-05-09T04:33:38.311 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.312+0000 7f12ed400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f12e80d0b40 con 0x7f12e815f590 2024-05-09T04:33:38.312 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.312+0000 7f12e7e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f12e80d18e0 0x7f12e815f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:38.312 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.312+0000 7f12e7400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f12e815f590 0x7f12e80d1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:38.312 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.312+0000 7f12e7400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f12e815f590 0x7f12e80d1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.145:49806/0 (socket says 172.21.15.145:49806) 2024-05-09T04:33:38.312 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.312+0000 7f12e7e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f12e80d18e0 0x7f12e815f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.145:46082/0 (socket says 172.21.15.145:46082) 2024-05-09T04:33:38.312 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.312+0000 7f12e7400640 1 -- 172.21.15.145:0/3759320209 learned_addr learned my addr 172.21.15.145:0/3759320209 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:38.313 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.313+0000 7f12e7e00640 1 -- 172.21.15.145:0/3759320209 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f12e815f590 msgr2=0x7f12e80d1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:38.313 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.313+0000 7f12e7e00640 1 --2- 172.21.15.145:0/3759320209 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f12e815f590 0x7f12e80d1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:38.313 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.313+0000 7f12e7e00640 1 -- 172.21.15.145:0/3759320209 >> v1:172.21.15.192:6789/0 conn(0x7f12e80d2200 legacy=0x7f12e80d2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:38.313 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.313+0000 7f12e7e00640 1 -- 172.21.15.145:0/3759320209 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12e80d0e20 con 0x7f12e80d18e0 2024-05-09T04:33:38.313 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.313+0000 7f12e7400640 1 --2- 172.21.15.145:0/3759320209 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f12e815f590 0x7f12e80d1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:33:38.314 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.314+0000 7f12e7e00640 1 --2- 172.21.15.145:0/3759320209 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12e80d18e0 0x7f12e815f050 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f12d80097a0 tx=0x7f12d802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=374a292b6e81e516 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:38.315 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.314+0000 7f12e6a00640 1 -- 172.21.15.145:0/3759320209 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d8037070 con 0x7f12e80d18e0 2024-05-09T04:33:38.315 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.315+0000 7f12e6a00640 1 -- 172.21.15.145:0/3759320209 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12d8032440 con 0x7f12e80d18e0 2024-05-09T04:33:38.315 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.315+0000 7f12e6a00640 1 -- 172.21.15.145:0/3759320209 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d8032810 con 0x7f12e80d18e0 2024-05-09T04:33:38.315 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.315+0000 7f12ed400640 1 -- 172.21.15.145:0/3759320209 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12e80d18e0 msgr2=0x7f12e815f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:38.315 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.315+0000 7f12ed400640 1 --2- 172.21.15.145:0/3759320209 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12e80d18e0 0x7f12e815f050 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f12d80097a0 tx=0x7f12d802a180 comp rx=0 tx=0).stop 2024-05-09T04:33:38.315 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.315+0000 7f12ed400640 1 -- 172.21.15.145:0/3759320209 shutdown_connections 2024-05-09T04:33:38.316 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.315+0000 7f12ed400640 1 --2- 172.21.15.145:0/3759320209 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f12e815f590 0x7f12e80d1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:38.316 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.315+0000 7f12ed400640 1 --2- 172.21.15.145:0/3759320209 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12e80d18e0 0x7f12e815f050 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:38.316 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.315+0000 7f12ed400640 1 -- 172.21.15.145:0/3759320209 >> 172.21.15.145:0/3759320209 conn(0x7f12e80d4fb0 msgr2=0x7f12e80d53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:38.316 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.315+0000 7f12ed400640 1 -- 172.21.15.145:0/3759320209 shutdown_connections 2024-05-09T04:33:38.316 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.315+0000 7f12ed400640 1 -- 172.21.15.145:0/3759320209 wait complete. 2024-05-09T04:33:38.316 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.317+0000 7f12ed400640 1 Processor -- start 2024-05-09T04:33:38.317 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.317+0000 7f12ed400640 1 -- start start 2024-05-09T04:33:38.318 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.318+0000 7f12ed400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f12e80d18e0 0x7f12e80c8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:38.318 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.318+0000 7f12ed400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f12e80d2200 0x7f12e81f5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:38.319 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.318+0000 7f12ed400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12e815f590 0x7f12e81543b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:38.319 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.318+0000 7f12ed400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f12e815f9f0 con 0x7f12e80d18e0 2024-05-09T04:33:38.319 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.318+0000 7f12ed400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f12e80b1eb0 con 0x7f12e815f590 2024-05-09T04:33:38.319 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.318+0000 7f12ed400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f12e80d2690 con 0x7f12e80d2200 2024-05-09T04:33:38.319 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.318+0000 7f12e7e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f12e80d18e0 0x7f12e80c8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:38.319 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.318+0000 7f12e7e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f12e80d18e0 0x7f12e80c8ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:46708/0 (socket says 172.21.15.145:46708) 2024-05-09T04:33:38.319 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.318+0000 7f12e7e00640 1 -- 172.21.15.145:0/1526650583 learned_addr learned my addr 172.21.15.145:0/1526650583 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:38.319 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.319+0000 7f12eca00640 1 --2- 172.21.15.145:0/1526650583 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12e815f590 0x7f12e81543b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:38.319 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.319+0000 7f12e7400640 1 --2- 172.21.15.145:0/1526650583 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f12e80d2200 0x7f12e81f5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:38.319 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.319+0000 7f12e7e00640 1 -- 172.21.15.145:0/1526650583 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f12e80d2200 msgr2=0x7f12e81f5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:38.319 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.319+0000 7f12e7e00640 1 --2- 172.21.15.145:0/1526650583 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f12e80d2200 0x7f12e81f5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:38.319 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.319+0000 7f12e7e00640 1 -- 172.21.15.145:0/1526650583 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12e815f590 msgr2=0x7f12e81543b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:38.319 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.319+0000 7f12e7e00640 1 --2- 172.21.15.145:0/1526650583 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12e815f590 0x7f12e81543b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:38.319 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.319+0000 7f12e7e00640 1 -- 172.21.15.145:0/1526650583 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12e80cd550 con 0x7f12e80d18e0 2024-05-09T04:33:38.320 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.319+0000 7f12e7e00640 1 --2- 172.21.15.145:0/1526650583 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f12e80d18e0 0x7f12e80c8ae0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f12d8035040 tx=0x7f12d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:38.320 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.319+0000 7f12dfe00640 1 -- 172.21.15.145:0/1526650583 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d8037040 con 0x7f12e80d18e0 2024-05-09T04:33:38.320 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.319+0000 7f12dfe00640 1 -- 172.21.15.145:0/1526650583 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12d8030500 con 0x7f12e80d18e0 2024-05-09T04:33:38.321 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.319+0000 7f12ed400640 1 -- 172.21.15.145:0/1526650583 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12e81f56d0 con 0x7f12e80d18e0 2024-05-09T04:33:38.321 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.319+0000 7f12dfe00640 1 -- 172.21.15.145:0/1526650583 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d80308d0 con 0x7f12e80d18e0 2024-05-09T04:33:38.321 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.319+0000 7f12ed400640 1 -- 172.21.15.145:0/1526650583 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12e81f5c50 con 0x7f12e80d18e0 2024-05-09T04:33:38.321 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.321+0000 7f12ed400640 1 -- 172.21.15.145:0/1526650583 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12e8076770 con 0x7f12e80d18e0 2024-05-09T04:33:38.321 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.321+0000 7f12dfe00640 1 -- 172.21.15.145:0/1526650583 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 4) ==== 98145+0+0 (secure 0 0 0) 0x7f12d8044020 con 0x7f12e80d18e0 2024-05-09T04:33:38.322 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.323+0000 7f12dfe00640 1 --2- 172.21.15.145:0/1526650583 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f12c8076240 0x7f12c8078700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:38.323 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.323+0000 7f12dfe00640 1 -- 172.21.15.145:0/1526650583 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7f12d80b6a30 con 0x7f12e80d18e0 2024-05-09T04:33:38.323 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.323+0000 7f12e7400640 1 --2- 172.21.15.145:0/1526650583 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f12c8076240 0x7f12c8078700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:38.323 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.324+0000 7f12e7400640 1 --2- 172.21.15.145:0/1526650583 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f12c8076240 0x7f12c8078700 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f12e8162770 tx=0x7f12d4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:38.328 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.328+0000 7f12dfe00640 1 -- 172.21.15.145:0/1526650583 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f12e80cd550 con 0x7f12e80d18e0 2024-05-09T04:33:38.517 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.517+0000 7f12ed400640 1 -- 172.21.15.145:0/1526650583 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f12e80d1dc0 con 0x7f12e80d18e0 2024-05-09T04:33:38.518 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.519+0000 7f12dfe00640 1 -- 172.21.15.145:0/1526650583 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7058 (secure 0 0 0) 0x7f12d80811f0 con 0x7f12e80d18e0 2024-05-09T04:33:38.518 INFO:teuthology.misc.health.smithi145.stdout: 2024-05-09T04:33:38.519 INFO:teuthology.misc.health.smithi145.stdout:{"epoch":12,"fsid":"957ee6c1-073b-44cf-9b93-9491b3fdfc1d","created":"2024-05-09T04:33:26.544367+0000","modified":"2024-05-09T04:33:35.765839+0000","last_up_change":"0.000000","last_in_change":"2024-05-09T04:33:29.902361+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"22e550ad-847b-4952-80e6-63d859ed1a0f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"0f6b9fca-0c41-4145-8d74-598ddb42f2a3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b3b1f67e-5790-483f-8610-e05a5bce4c30","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"884787a5-e037-4d31-9aae-743030eba4ce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"0759fc1e-c5fc-4a2f-86bb-e8eccd83f833","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9443cc22-9379-46e3-9173-98b592bf2c16","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d05a3311-505a-45b5-85b9-aa3113e96987","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"c7c23dbc-2029-43ec-9f53-25f3d58019a5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T04:33:38.522 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.522+0000 7f12ed400640 1 -- 172.21.15.145:0/1526650583 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f12c8076240 msgr2=0x7f12c8078700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:38.522 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.522+0000 7f12ed400640 1 --2- 172.21.15.145:0/1526650583 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f12c8076240 0x7f12c8078700 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f12e8162770 tx=0x7f12d4008040 comp rx=0 tx=0).stop 2024-05-09T04:33:38.522 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.522+0000 7f12ed400640 1 -- 172.21.15.145:0/1526650583 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f12e80d18e0 msgr2=0x7f12e80c8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:38.522 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.522+0000 7f12ed400640 1 --2- 172.21.15.145:0/1526650583 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f12e80d18e0 0x7f12e80c8ae0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f12d8035040 tx=0x7f12d80046c0 comp rx=0 tx=0).stop 2024-05-09T04:33:38.522 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.522+0000 7f12ed400640 1 -- 172.21.15.145:0/1526650583 shutdown_connections 2024-05-09T04:33:38.522 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.522+0000 7f12ed400640 1 --2- 172.21.15.145:0/1526650583 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f12c8076240 0x7f12c8078700 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:38.522 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.522+0000 7f12ed400640 1 --2- 172.21.15.145:0/1526650583 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12e815f590 0x7f12e81543b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:38.522 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.522+0000 7f12ed400640 1 --2- 172.21.15.145:0/1526650583 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f12e80d2200 0x7f12e81f5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:38.522 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.522+0000 7f12ed400640 1 --2- 172.21.15.145:0/1526650583 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f12e80d18e0 0x7f12e80c8ae0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:38.522 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.522+0000 7f12ed400640 1 -- 172.21.15.145:0/1526650583 >> 172.21.15.145:0/1526650583 conn(0x7f12e80d4fb0 msgr2=0x7f12e80b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:38.523 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.522+0000 7f12ed400640 1 -- 172.21.15.145:0/1526650583 shutdown_connections 2024-05-09T04:33:38.523 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:38.522+0000 7f12ed400640 1 -- 172.21.15.145:0/1526650583 wait complete. 2024-05-09T04:33:38.535 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-05-09T04:33:45.342 DEBUG:teuthology.orchestra.run.smithi145:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-09T04:33:45.528 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.528+0000 7ff4ea000640 1 Processor -- start 2024-05-09T04:33:45.528 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.529+0000 7ff4ea000640 1 -- start start 2024-05-09T04:33:45.530 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.530+0000 7ff4ea000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff4e415aa20 0x7ff4e4071f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:45.530 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.530+0000 7ff4ea000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff4e406df40 0x7ff4e4072480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:45.530 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.530+0000 7ff4ea000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4e4074230 con 0x7ff4e4072ae0 2024-05-09T04:33:45.530 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.530+0000 7ff4ea000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7ff4e415dca0 con 0x7ff4e415aa20 2024-05-09T04:33:45.530 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.530+0000 7ff4ea000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7ff4e415d690 con 0x7ff4e406df40 2024-05-09T04:33:45.530 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.530+0000 7ff4e8c00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff4e415aa20 0x7ff4e4071f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:45.530 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.530+0000 7ff4e8c00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff4e415aa20 0x7ff4e4071f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:49186/0 (socket says 172.21.15.145:49186) 2024-05-09T04:33:45.530 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.530+0000 7ff4e3e00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7ff4e4072ae0 0x7ff4e40cb1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:40440/0 (socket says 172.21.15.145:40440) 2024-05-09T04:33:45.531 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.530+0000 7ff4e8c00640 1 -- 172.21.15.145:0/1835738953 learned_addr learned my addr 172.21.15.145:0/1835738953 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:45.531 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.531+0000 7ff4e9600640 1 --2- 172.21.15.145:0/1835738953 >> v2:172.21.15.192:3300/0 conn(0x7ff4e406df40 0x7ff4e4072480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:45.532 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.531+0000 7ff4e8c00640 1 -- 172.21.15.145:0/1835738953 >> v2:172.21.15.192:3300/0 conn(0x7ff4e406df40 msgr2=0x7ff4e4072480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:45.532 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.531+0000 7ff4e8c00640 1 --2- 172.21.15.145:0/1835738953 >> v2:172.21.15.192:3300/0 conn(0x7ff4e406df40 0x7ff4e4072480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:45.532 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.531+0000 7ff4e8c00640 1 -- 172.21.15.145:0/1835738953 >> v1:172.21.15.145:6789/0 conn(0x7ff4e4072ae0 legacy=0x7ff4e40cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:45.532 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.531+0000 7ff4e8c00640 1 -- 172.21.15.145:0/1835738953 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4e4073f50 con 0x7ff4e415aa20 2024-05-09T04:33:45.532 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.532+0000 7ff4e8c00640 1 --2- 172.21.15.145:0/1835738953 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff4e415aa20 0x7ff4e4071f40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff4e4070450 tx=0x7ff4d802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f99dd70becbec44 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:45.532 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.532+0000 7ff4e3400640 1 -- 172.21.15.145:0/1835738953 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d8037070 con 0x7ff4e415aa20 2024-05-09T04:33:45.532 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.532+0000 7ff4e3400640 1 -- 172.21.15.145:0/1835738953 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4d8032440 con 0x7ff4e415aa20 2024-05-09T04:33:45.532 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.532+0000 7ff4e3400640 1 -- 172.21.15.145:0/1835738953 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d8032810 con 0x7ff4e415aa20 2024-05-09T04:33:45.533 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.533+0000 7ff4ea000640 1 -- 172.21.15.145:0/1835738953 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff4e415aa20 msgr2=0x7ff4e4071f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:45.533 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.533+0000 7ff4ea000640 1 --2- 172.21.15.145:0/1835738953 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff4e415aa20 0x7ff4e4071f40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff4e4070450 tx=0x7ff4d802a270 comp rx=0 tx=0).stop 2024-05-09T04:33:45.533 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.533+0000 7ff4ea000640 1 -- 172.21.15.145:0/1835738953 shutdown_connections 2024-05-09T04:33:45.533 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.533+0000 7ff4ea000640 1 --2- 172.21.15.145:0/1835738953 >> v2:172.21.15.192:3300/0 conn(0x7ff4e406df40 0x7ff4e4072480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:45.533 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.533+0000 7ff4ea000640 1 --2- 172.21.15.145:0/1835738953 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff4e415aa20 0x7ff4e4071f40 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:45.533 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.533+0000 7ff4ea000640 1 -- 172.21.15.145:0/1835738953 >> 172.21.15.145:0/1835738953 conn(0x7ff4e40d0b60 msgr2=0x7ff4e40d0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:45.533 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.533+0000 7ff4ea000640 1 -- 172.21.15.145:0/1835738953 shutdown_connections 2024-05-09T04:33:45.533 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.533+0000 7ff4ea000640 1 -- 172.21.15.145:0/1835738953 wait complete. 2024-05-09T04:33:45.534 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.535+0000 7ff4ea000640 1 Processor -- start 2024-05-09T04:33:45.534 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.535+0000 7ff4ea000640 1 -- start start 2024-05-09T04:33:45.535 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.536+0000 7ff4ea000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff4e406df40 0x7ff4e40c7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:45.535 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.536+0000 7ff4ea000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff4e4072ae0 0x7ff4e414fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:45.536 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.536+0000 7ff4ea000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4e415aa20 0x7ff4e40cf180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:45.536 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.536+0000 7ff4ea000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e415ae00 con 0x7ff4e406df40 2024-05-09T04:33:45.536 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.536+0000 7ff4ea000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e4073b50 con 0x7ff4e415aa20 2024-05-09T04:33:45.536 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.536+0000 7ff4ea000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff4e40b3680 con 0x7ff4e4072ae0 2024-05-09T04:33:45.536 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.536+0000 7ff4e8c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff4e406df40 0x7ff4e40c7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:45.536 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.536+0000 7ff4e8c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff4e406df40 0x7ff4e40c7530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:49190/0 (socket says 172.21.15.145:49190) 2024-05-09T04:33:45.536 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.536+0000 7ff4e8c00640 1 -- 172.21.15.145:0/865681863 learned_addr learned my addr 172.21.15.145:0/865681863 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:45.538 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.536+0000 7ff4e8c00640 1 -- 172.21.15.145:0/865681863 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff4e4072ae0 msgr2=0x7ff4e414fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:45.538 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.536+0000 7ff4e8c00640 1 --2- 172.21.15.145:0/865681863 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff4e4072ae0 0x7ff4e414fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:45.538 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.536+0000 7ff4e9600640 1 --2- 172.21.15.145:0/865681863 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4e415aa20 0x7ff4e40cf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:45.538 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.536+0000 7ff4e8c00640 1 -- 172.21.15.145:0/865681863 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4e415aa20 msgr2=0x7ff4e40cf180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:45.538 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.536+0000 7ff4e8c00640 1 --2- 172.21.15.145:0/865681863 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4e415aa20 0x7ff4e40cf180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:45.538 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.536+0000 7ff4e8c00640 1 -- 172.21.15.145:0/865681863 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4e40b3960 con 0x7ff4e406df40 2024-05-09T04:33:45.538 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.537+0000 7ff4e8c00640 1 --2- 172.21.15.145:0/865681863 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff4e406df40 0x7ff4e40c7530 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff4d802a2b0 tx=0x7ff4d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:45.538 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.537+0000 7ff4e0c00640 1 -- 172.21.15.145:0/865681863 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d8041020 con 0x7ff4e406df40 2024-05-09T04:33:45.539 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.537+0000 7ff4e0c00640 1 -- 172.21.15.145:0/865681863 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4d80306c0 con 0x7ff4e406df40 2024-05-09T04:33:45.539 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.537+0000 7ff4ea000640 1 -- 172.21.15.145:0/865681863 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4e40b3c40 con 0x7ff4e406df40 2024-05-09T04:33:45.539 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.537+0000 7ff4e0c00640 1 -- 172.21.15.145:0/865681863 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d8037040 con 0x7ff4e406df40 2024-05-09T04:33:45.539 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.537+0000 7ff4ea000640 1 -- 172.21.15.145:0/865681863 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4e40b4200 con 0x7ff4e406df40 2024-05-09T04:33:45.539 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.538+0000 7ff4ea000640 1 -- 172.21.15.145:0/865681863 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4e4076770 con 0x7ff4e406df40 2024-05-09T04:33:45.539 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.539+0000 7ff4e0c00640 1 -- 172.21.15.145:0/865681863 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff4d8044020 con 0x7ff4e406df40 2024-05-09T04:33:45.540 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.540+0000 7ff4e0c00640 1 --2- 172.21.15.145:0/865681863 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff4c40762d0 0x7ff4c4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:45.540 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.540+0000 7ff4e0c00640 1 -- 172.21.15.145:0/865681863 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7ff4d80b7ab0 con 0x7ff4e406df40 2024-05-09T04:33:45.540 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.540+0000 7ff4e3e00640 1 --2- 172.21.15.145:0/865681863 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff4c40762d0 0x7ff4c4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:45.541 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.541+0000 7ff4e3e00640 1 --2- 172.21.15.145:0/865681863 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff4c40762d0 0x7ff4c4078790 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff4e40d0910 tx=0x7ff4d00079c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:45.545 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.546+0000 7ff4e0c00640 1 -- 172.21.15.145:0/865681863 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff4d804b030 con 0x7ff4e406df40 2024-05-09T04:33:45.752 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.753+0000 7ff4ea000640 1 -- 172.21.15.145:0/865681863 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff4e40732d0 con 0x7ff4e406df40 2024-05-09T04:33:45.754 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.754+0000 7ff4e0c00640 1 -- 172.21.15.145:0/865681863 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12932 (secure 0 0 0) 0x7ff4d8081750 con 0x7ff4e406df40 2024-05-09T04:33:45.754 INFO:teuthology.misc.health.smithi145.stdout: 2024-05-09T04:33:45.755 INFO:teuthology.misc.health.smithi145.stdout:{"epoch":16,"fsid":"957ee6c1-073b-44cf-9b93-9491b3fdfc1d","created":"2024-05-09T04:33:26.544367+0000","modified":"2024-05-09T04:33:42.070628+0000","last_up_change":"2024-05-09T04:33:39.039318+0000","last_in_change":"2024-05-09T04:33:29.902361+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T04:33:39.795344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22e550ad-847b-4952-80e6-63d859ed1a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6801","nonce":3182572912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6803","nonce":3182572912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6807","nonce":3182572912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6805","nonce":3182572912}]},"public_addr":"172.21.15.145:6801/3182572912","cluster_addr":"172.21.15.145:6803/3182572912","heartbeat_back_addr":"172.21.15.145:6807/3182572912","heartbeat_front_addr":"172.21.15.145:6805/3182572912","state":["exists","up"]},{"osd":1,"uuid":"0f6b9fca-0c41-4145-8d74-598ddb42f2a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6825","nonce":1845225555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6827","nonce":1845225555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6831","nonce":1845225555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6829","nonce":1845225555}]},"public_addr":"172.21.15.145:6825/1845225555","cluster_addr":"172.21.15.145:6827/1845225555","heartbeat_back_addr":"172.21.15.145:6831/1845225555","heartbeat_front_addr":"172.21.15.145:6829/1845225555","state":["exists","up"]},{"osd":2,"uuid":"b3b1f67e-5790-483f-8610-e05a5bce4c30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6817","nonce":2168958993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6819","nonce":2168958993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6823","nonce":2168958993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6821","nonce":2168958993}]},"public_addr":"172.21.15.145:6817/2168958993","cluster_addr":"172.21.15.145:6819/2168958993","heartbeat_back_addr":"172.21.15.145:6823/2168958993","heartbeat_front_addr":"172.21.15.145:6821/2168958993","state":["exists","up"]},{"osd":3,"uuid":"884787a5-e037-4d31-9aae-743030eba4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6809","nonce":337009826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6811","nonce":337009826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6815","nonce":337009826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6813","nonce":337009826}]},"public_addr":"172.21.15.145:6809/337009826","cluster_addr":"172.21.15.145:6811/337009826","heartbeat_back_addr":"172.21.15.145:6815/337009826","heartbeat_front_addr":"172.21.15.145:6813/337009826","state":["exists","up"]},{"osd":4,"uuid":"0759fc1e-c5fc-4a2f-86bb-e8eccd83f833","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6818","nonce":3117700995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6821","nonce":3117700995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6827","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6829","nonce":3117700995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6823","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6825","nonce":3117700995}]},"public_addr":"172.21.15.192:6818/3117700995","cluster_addr":"172.21.15.192:6821/3117700995","heartbeat_back_addr":"172.21.15.192:6829/3117700995","heartbeat_front_addr":"172.21.15.192:6825/3117700995","state":["exists","up"]},{"osd":5,"uuid":"9443cc22-9379-46e3-9173-98b592bf2c16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6801","nonce":1328373268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6803","nonce":1328373268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6807","nonce":1328373268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6805","nonce":1328373268}]},"public_addr":"172.21.15.192:6801/1328373268","cluster_addr":"172.21.15.192:6803/1328373268","heartbeat_back_addr":"172.21.15.192:6807/1328373268","heartbeat_front_addr":"172.21.15.192:6805/1328373268","state":["exists","up"]},{"osd":6,"uuid":"d05a3311-505a-45b5-85b9-aa3113e96987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6817","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6820","nonce":1868011057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6824","nonce":1868011057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6831","nonce":1868011057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6828","nonce":1868011057}]},"public_addr":"172.21.15.192:6820/1868011057","cluster_addr":"172.21.15.192:6824/1868011057","heartbeat_back_addr":"172.21.15.192:6831/1868011057","heartbeat_front_addr":"172.21.15.192:6828/1868011057","state":["exists","up"]},{"osd":7,"uuid":"c7c23dbc-2029-43ec-9f53-25f3d58019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6809","nonce":3368512157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6811","nonce":3368512157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6815","nonce":3368512157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6813","nonce":3368512157}]},"public_addr":"172.21.15.192:6809/3368512157","cluster_addr":"172.21.15.192:6811/3368512157","heartbeat_back_addr":"172.21.15.192:6815/3368512157","heartbeat_front_addr":"172.21.15.192:6813/3368512157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.091188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.032567+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:34.797468+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.228839+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.622400+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.473103+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.566092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.349395+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T04:33:45.757 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.757+0000 7ff4ea000640 1 -- 172.21.15.145:0/865681863 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff4c40762d0 msgr2=0x7ff4c4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:45.757 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.757+0000 7ff4ea000640 1 --2- 172.21.15.145:0/865681863 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff4c40762d0 0x7ff4c4078790 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff4e40d0910 tx=0x7ff4d00079c0 comp rx=0 tx=0).stop 2024-05-09T04:33:45.757 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.757+0000 7ff4ea000640 1 -- 172.21.15.145:0/865681863 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff4e406df40 msgr2=0x7ff4e40c7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:45.757 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.757+0000 7ff4ea000640 1 --2- 172.21.15.145:0/865681863 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff4e406df40 0x7ff4e40c7530 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff4d802a2b0 tx=0x7ff4d80046c0 comp rx=0 tx=0).stop 2024-05-09T04:33:45.758 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.757+0000 7ff4ea000640 1 -- 172.21.15.145:0/865681863 shutdown_connections 2024-05-09T04:33:45.758 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.757+0000 7ff4ea000640 1 --2- 172.21.15.145:0/865681863 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff4c40762d0 0x7ff4c4078790 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:45.758 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.757+0000 7ff4ea000640 1 --2- 172.21.15.145:0/865681863 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4e415aa20 0x7ff4e40cf180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:45.758 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.757+0000 7ff4ea000640 1 --2- 172.21.15.145:0/865681863 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff4e4072ae0 0x7ff4e414fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:45.758 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.757+0000 7ff4ea000640 1 --2- 172.21.15.145:0/865681863 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff4e406df40 0x7ff4e40c7530 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:45.758 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.757+0000 7ff4ea000640 1 -- 172.21.15.145:0/865681863 >> 172.21.15.145:0/865681863 conn(0x7ff4e40d0b60 msgr2=0x7ff4e4160900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:45.758 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.758+0000 7ff4ea000640 1 -- 172.21.15.145:0/865681863 shutdown_connections 2024-05-09T04:33:45.758 INFO:teuthology.misc.health.smithi145.stderr:2024-05-09T04:33:45.758+0000 7ff4ea000640 1 -- 172.21.15.145:0/865681863 wait complete. 2024-05-09T04:33:45.772 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-05-09T04:33:45.772 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-05-09T04:33:45.773 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-09T04:33:45.773 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-09T04:33:45.780 INFO:tasks.ceph.mds.a:Started 2024-05-09T04:33:45.780 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-09T04:33:45.780 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-09T04:33:45.787 INFO:tasks.ceph.mds.c:Started 2024-05-09T04:33:45.787 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-09T04:33:45.788 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-09T04:33:45.795 INFO:tasks.ceph.mds.b:Started 2024-05-09T04:33:45.795 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-09T04:33:45.796 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-09T04:33:45.803 INFO:tasks.ceph.mds.d:Started 2024-05-09T04:33:45.804 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-05-09T04:33:45.804 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T04:33:45.924 INFO:tasks.ceph.mds.a.smithi145.stdout:starting mds.a at 2024-05-09T04:33:45.931 INFO:tasks.ceph.mds.c.smithi145.stdout:starting mds.c at 2024-05-09T04:33:45.948 INFO:tasks.ceph.mds.b.smithi192.stdout:starting mds.b at 2024-05-09T04:33:45.951 INFO:tasks.ceph.mds.d.smithi192.stdout:starting mds.d at 2024-05-09T04:33:45.984 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:45.984+0000 7ff532600640 1 Processor -- start 2024-05-09T04:33:45.984 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:45.984+0000 7ff532600640 1 -- start start 2024-05-09T04:33:45.986 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:45.985+0000 7ff532600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff52c0b40f0 0x7ff52c0d37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:45.986 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:45.985+0000 7ff532600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff52c0d3d10 0x7ff52c0b22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:45.986 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:45.985+0000 7ff532600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff52c0d09b0 con 0x7ff52c15f490 2024-05-09T04:33:45.986 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:45.985+0000 7ff532600640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7ff52c0b1e20 con 0x7ff52c0b40f0 2024-05-09T04:33:45.986 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:45.985+0000 7ff532600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7ff52c0d4150 con 0x7ff52c0d3d10 2024-05-09T04:33:45.987 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:45.985+0000 7ff52b400640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7ff52c15f490 0x7ff52c0d5900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:40446/0 (socket says 172.21.15.145:40446) 2024-05-09T04:33:45.987 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:45.985+0000 7ff52b400640 1 -- 172.21.15.145:0/3529255068 learned_addr learned my addr 172.21.15.145:0/3529255068 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:33:45.987 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:45.986+0000 7ff52be00640 1 --2- 172.21.15.145:0/3529255068 >> v2:172.21.15.192:3300/0 conn(0x7ff52c0d3d10 0x7ff52c0b22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:45.987 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:45.986+0000 7ff52aa00640 1 --2- 172.21.15.145:0/3529255068 >> v2:172.21.15.145:3300/0 conn(0x7ff52c0b40f0 0x7ff52c0d37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:45.987 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:45.986+0000 7ff52a000640 1 -- 172.21.15.145:0/3529255068 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3934791995 0 0) 0x7ff52c0d09b0 con 0x7ff52c15f490 2024-05-09T04:33:45.987 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:45.986+0000 7ff52a000640 1 -- 172.21.15.145:0/3529255068 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff510003520 con 0x7ff52c15f490 2024-05-09T04:33:45.987 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:45.986+0000 7ff52be00640 1 -- 172.21.15.145:0/3529255068 >> v2:172.21.15.145:3300/0 conn(0x7ff52c0b40f0 msgr2=0x7ff52c0d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:45.987 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:45.986+0000 7ff52be00640 1 --2- 172.21.15.145:0/3529255068 >> v2:172.21.15.145:3300/0 conn(0x7ff52c0b40f0 0x7ff52c0d37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:45.987 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:45.986+0000 7ff52be00640 1 -- 172.21.15.145:0/3529255068 >> v1:172.21.15.145:6789/0 conn(0x7ff52c15f490 legacy=0x7ff52c0d5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:45.987 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:45.986+0000 7ff52be00640 1 -- 172.21.15.145:0/3529255068 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff52c0d0c90 con 0x7ff52c0d3d10 2024-05-09T04:33:45.988 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:45.987+0000 7ff52be00640 1 --2- 172.21.15.145:0/3529255068 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff52c0d3d10 0x7ff52c0b22b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff52c08f4a0 tx=0x7ff52402a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14549a5d2b728c63 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:45.988 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:45.988+0000 7ff52a000640 1 -- 172.21.15.145:0/3529255068 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff524007740 con 0x7ff52c0d3d10 2024-05-09T04:33:45.988 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:45.988+0000 7ff52a000640 1 -- 172.21.15.145:0/3529255068 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5240078e0 con 0x7ff52c0d3d10 2024-05-09T04:33:45.989 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:45.988+0000 7ff52a000640 1 -- 172.21.15.145:0/3529255068 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff524007cb0 con 0x7ff52c0d3d10 2024-05-09T04:33:45.989 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:45.988+0000 7ff532600640 1 -- 172.21.15.145:0/3529255068 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff52c0d3d10 msgr2=0x7ff52c0b22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:45.989 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:45.988+0000 7ff532600640 1 --2- 172.21.15.145:0/3529255068 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff52c0d3d10 0x7ff52c0b22b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff52c08f4a0 tx=0x7ff52402a710 comp rx=0 tx=0).stop 2024-05-09T04:33:45.989 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:45.988+0000 7ff532600640 1 -- 172.21.15.145:0/3529255068 shutdown_connections 2024-05-09T04:33:45.989 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:45.988+0000 7ff532600640 1 --2- 172.21.15.145:0/3529255068 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff52c0d3d10 0x7ff52c0b22b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:45.989 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:45.988+0000 7ff532600640 1 --2- 172.21.15.145:0/3529255068 >> v2:172.21.15.145:3300/0 conn(0x7ff52c0b40f0 0x7ff52c0d37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:45.989 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:45.988+0000 7ff532600640 1 -- 172.21.15.145:0/3529255068 >> 172.21.15.145:0/3529255068 conn(0x7ff52c09e530 msgr2=0x7ff52c0d51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:45.989 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:45.988+0000 7ff532600640 1 -- 172.21.15.145:0/3529255068 shutdown_connections 2024-05-09T04:33:45.989 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:45.988+0000 7ff532600640 1 -- 172.21.15.145:0/3529255068 wait complete. 2024-05-09T04:33:45.990 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:45.990+0000 7ff532600640 1 Processor -- start 2024-05-09T04:33:45.990 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:45.990+0000 7ff532600640 1 -- start start 2024-05-09T04:33:45.992 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:45.991+0000 7ff532600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff52c0b40f0 0x7ff52c1f90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:45.992 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:45.991+0000 7ff532600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff52c0d3d10 0x7ff52c0b9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:45.992 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:45.991+0000 7ff532600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff52c15f490 0x7ff52c0cb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:45.992 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:45.991+0000 7ff532600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff52c15f9f0 con 0x7ff52c0b40f0 2024-05-09T04:33:45.992 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:45.991+0000 7ff532600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff52c15f870 con 0x7ff52c0d3d10 2024-05-09T04:33:45.992 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:45.991+0000 7ff532600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff52c0b27f0 con 0x7ff52c15f490 2024-05-09T04:33:45.993 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:45.991+0000 7ff52b400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff52c0b40f0 0x7ff52c1f90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:45.993 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:45.991+0000 7ff52b400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff52c0b40f0 0x7ff52c1f90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:49258/0 (socket says 172.21.15.145:49258) 2024-05-09T04:33:45.993 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:45.991+0000 7ff52b400640 1 -- 172.21.15.145:0/1805642511 learned_addr learned my addr 172.21.15.145:0/1805642511 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:45.993 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:45.992+0000 7ff52aa00640 1 --2- 172.21.15.145:0/1805642511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff52c0d3d10 0x7ff52c0b9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:45.993 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:45.992+0000 7ff52be00640 1 --2- 172.21.15.145:0/1805642511 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff52c15f490 0x7ff52c0cb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:45.993 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:45.992+0000 7ff52b400640 1 -- 172.21.15.145:0/1805642511 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff52c15f490 msgr2=0x7ff52c0cb700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:45.993 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:45.992+0000 7ff52b400640 1 --2- 172.21.15.145:0/1805642511 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff52c15f490 0x7ff52c0cb700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:45.993 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:45.992+0000 7ff52b400640 1 -- 172.21.15.145:0/1805642511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff52c0d3d10 msgr2=0x7ff52c0b9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:45.993 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:45.992+0000 7ff52b400640 1 --2- 172.21.15.145:0/1805642511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff52c0d3d10 0x7ff52c0b9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:45.993 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:45.992+0000 7ff52b400640 1 -- 172.21.15.145:0/1805642511 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff52c0ba230 con 0x7ff52c0b40f0 2024-05-09T04:33:45.993 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:45.992+0000 7ff52aa00640 1 --2- 172.21.15.145:0/1805642511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff52c0d3d10 0x7ff52c0b9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:33:45.993 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:45.992+0000 7ff52b400640 1 --2- 172.21.15.145:0/1805642511 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff52c0b40f0 0x7ff52c1f90a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff51c00c410 tx=0x7ff51c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:45.993 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:45.992+0000 7ff523400640 1 -- 172.21.15.145:0/1805642511 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff51c017070 con 0x7ff52c0b40f0 2024-05-09T04:33:45.993 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:45.992+0000 7ff523400640 1 -- 172.21.15.145:0/1805642511 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff51c005ce0 con 0x7ff52c0b40f0 2024-05-09T04:33:45.993 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:45.992+0000 7ff532600640 1 -- 172.21.15.145:0/1805642511 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff52c0ba4d0 con 0x7ff52c0b40f0 2024-05-09T04:33:45.994 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:45.992+0000 7ff523400640 1 -- 172.21.15.145:0/1805642511 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff51c0072d0 con 0x7ff52c0b40f0 2024-05-09T04:33:45.994 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:45.993+0000 7ff532600640 1 -- 172.21.15.145:0/1805642511 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff52c0baa90 con 0x7ff52c0b40f0 2024-05-09T04:33:45.994 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:45.994+0000 7ff532600640 1 -- 172.21.15.145:0/1805642511 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff52c06a310 con 0x7ff52c0b40f0 2024-05-09T04:33:45.994 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:45.995+0000 7ff523400640 1 -- 172.21.15.145:0/1805642511 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff51c01a070 con 0x7ff52c0b40f0 2024-05-09T04:33:45.997 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:45.996+0000 7ff523400640 1 --2- 172.21.15.145:0/1805642511 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff50c0762d0 0x7ff50c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:45.997 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:45.997+0000 7ff523400640 1 -- 172.21.15.145:0/1805642511 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7ff51c096590 con 0x7ff52c0b40f0 2024-05-09T04:33:45.998 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:45.997+0000 7ff52aa00640 1 --2- 172.21.15.145:0/1805642511 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff50c0762d0 0x7ff50c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:45.998 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:45.997+0000 7ff52aa00640 1 --2- 172.21.15.145:0/1805642511 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff50c0762d0 0x7ff50c078790 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff52c1f95e0 tx=0x7ff518008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:46.001 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:46.001+0000 7ff523400640 1 -- 172.21.15.145:0/1805642511 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff51c05fbe0 con 0x7ff52c0b40f0 2024-05-09T04:33:46.244 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:46.244+0000 7ff532600640 1 -- 172.21.15.145:0/1805642511 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff52c0d4fd0 con 0x7ff52c0b40f0 2024-05-09T04:33:46.245 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:46.245+0000 7ff523400640 1 -- 172.21.15.145:0/1805642511 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1395 (secure 0 0 0) 0x7ff51c063ee0 con 0x7ff52c0b40f0 2024-05-09T04:33:46.245 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:33:46.245 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/1284540932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":1284540932},{"type":"v1","addr":"172.21.15.145:6835","nonce":1284540932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-09T04:33:46.245 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 2 2024-05-09T04:33:46.248 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:46.247+0000 7ff532600640 1 -- 172.21.15.145:0/1805642511 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff50c0762d0 msgr2=0x7ff50c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:46.248 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:46.247+0000 7ff532600640 1 --2- 172.21.15.145:0/1805642511 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff50c0762d0 0x7ff50c078790 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff52c1f95e0 tx=0x7ff518008040 comp rx=0 tx=0).stop 2024-05-09T04:33:46.248 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:46.247+0000 7ff532600640 1 -- 172.21.15.145:0/1805642511 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff52c0b40f0 msgr2=0x7ff52c1f90a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:46.248 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:46.247+0000 7ff532600640 1 --2- 172.21.15.145:0/1805642511 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff52c0b40f0 0x7ff52c1f90a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff51c00c410 tx=0x7ff51c00c8d0 comp rx=0 tx=0).stop 2024-05-09T04:33:46.248 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:46.248+0000 7ff532600640 1 -- 172.21.15.145:0/1805642511 shutdown_connections 2024-05-09T04:33:46.248 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:46.248+0000 7ff532600640 1 --2- 172.21.15.145:0/1805642511 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff50c0762d0 0x7ff50c078790 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:46.248 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:46.248+0000 7ff532600640 1 --2- 172.21.15.145:0/1805642511 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff52c15f490 0x7ff52c0cb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:46.248 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:46.248+0000 7ff532600640 1 --2- 172.21.15.145:0/1805642511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff52c0d3d10 0x7ff52c0b9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:46.248 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:46.248+0000 7ff532600640 1 --2- 172.21.15.145:0/1805642511 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff52c0b40f0 0x7ff52c1f90a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:46.248 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:46.248+0000 7ff532600640 1 -- 172.21.15.145:0/1805642511 >> 172.21.15.145:0/1805642511 conn(0x7ff52c09e530 msgr2=0x7ff52c0b6ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:46.248 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:46.248+0000 7ff532600640 1 -- 172.21.15.145:0/1805642511 shutdown_connections 2024-05-09T04:33:46.249 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:46.248+0000 7ff532600640 1 -- 172.21.15.145:0/1805642511 wait complete. 2024-05-09T04:33:48.266 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T04:33:48.473 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.473+0000 7f624d000640 1 Processor -- start 2024-05-09T04:33:48.473 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.473+0000 7f624d000640 1 -- start start 2024-05-09T04:33:48.474 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.475+0000 7f624d000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6248148c20 0x7f62480affc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:48.475 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.475+0000 7f624d000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6248072350 con 0x7f6248145880 2024-05-09T04:33:48.475 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.475+0000 7f624d000640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6248072630 con 0x7f624806ef70 2024-05-09T04:33:48.475 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.475+0000 7f624d000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f6248072f60 con 0x7f6248148c20 2024-05-09T04:33:48.475 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.475+0000 7f6245e00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f6248145880 0x7f624806e860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:40456/0 (socket says 172.21.15.145:40456) 2024-05-09T04:33:48.475 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.475+0000 7f6245e00640 1 -- 172.21.15.145:0/231562105 learned_addr learned my addr 172.21.15.145:0/231562105 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:33:48.476 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.475+0000 7f6245400640 1 -- 172.21.15.145:0/231562105 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 270217029 0 0) 0x7f6248072350 con 0x7f6248145880 2024-05-09T04:33:48.476 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.475+0000 7f6245400640 1 -- 172.21.15.145:0/231562105 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6228003520 con 0x7f6248145880 2024-05-09T04:33:48.477 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.476+0000 7f6246800640 1 --2- 172.21.15.145:0/231562105 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6248148c20 0x7f62480affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:48.477 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.476+0000 7f6245400640 1 -- 172.21.15.145:0/231562105 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 174320836 0 0) 0x7f6228003520 con 0x7f6248145880 2024-05-09T04:33:48.477 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.476+0000 7f6245400640 1 -- 172.21.15.145:0/231562105 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6248148c20 msgr2=0x7f62480affc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:48.477 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.476+0000 7f6245400640 1 --2- 172.21.15.145:0/231562105 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6248148c20 0x7f62480affc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:48.477 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.476+0000 7f6245400640 1 -- 172.21.15.145:0/231562105 >> v1:172.21.15.192:6789/0 conn(0x7f624806ef70 legacy=0x7f62480b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:48.477 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.476+0000 7f6245400640 1 -- 172.21.15.145:0/231562105 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6248072910 con 0x7f6248145880 2024-05-09T04:33:48.477 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.476+0000 7f6245400640 1 -- 172.21.15.145:0/231562105 <== mon.0 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3474413261 0 0) 0x7f62380016f0 con 0x7f6248145880 2024-05-09T04:33:48.477 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.476+0000 7f6245400640 1 -- 172.21.15.145:0/231562105 >> v1:172.21.15.145:6789/0 conn(0x7f6248145880 legacy=0x7f624806e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:48.477 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.476+0000 7f6245400640 1 --2- 172.21.15.145:0/231562105 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6228003a30 0x7f6228003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:48.477 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.476+0000 7f6245400640 1 --2- 172.21.15.145:0/231562105 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6228004800 0x7f6228004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:48.477 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.476+0000 7f6245400640 1 --2- 172.21.15.145:0/231562105 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6248148c20 0x7f622800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:48.477 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.476+0000 7f6245400640 1 -- 172.21.15.145:0/231562105 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f62280054f0 con 0x7f6228003a30 2024-05-09T04:33:48.477 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.476+0000 7f6245400640 1 -- 172.21.15.145:0/231562105 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f622800bf00 con 0x7f6228004800 2024-05-09T04:33:48.477 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.476+0000 7f6245400640 1 -- 172.21.15.145:0/231562105 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f622800c080 con 0x7f6248148c20 2024-05-09T04:33:48.477 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.476+0000 7f6245e00640 1 --2- 172.21.15.145:0/231562105 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6228003a30 0x7f6228003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:48.478 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.477+0000 7f6245e00640 1 -- 172.21.15.145:0/231562105 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6248148c20 msgr2=0x7f622800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:48.478 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.477+0000 7f6245e00640 1 --2- 172.21.15.145:0/231562105 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6248148c20 0x7f622800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:48.478 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.477+0000 7f6245e00640 1 -- 172.21.15.145:0/231562105 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6228004800 msgr2=0x7f6228004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:48.478 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.477+0000 7f6245e00640 1 --2- 172.21.15.145:0/231562105 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6228004800 0x7f6228004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:48.478 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.477+0000 7f6245e00640 1 -- 172.21.15.145:0/231562105 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6228003520 con 0x7f6228003a30 2024-05-09T04:33:48.479 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.478+0000 7f6245e00640 1 --2- 172.21.15.145:0/231562105 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6228003a30 0x7f6228003e30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f623800c190 tx=0x7f6228005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e28db235ff44c6a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:48.479 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.478+0000 7f6245400640 1 -- 172.21.15.145:0/231562105 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6238007cc0 con 0x7f6228003a30 2024-05-09T04:33:48.479 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.478+0000 7f6245400640 1 -- 172.21.15.145:0/231562105 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6238005ad0 con 0x7f6228003a30 2024-05-09T04:33:48.479 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.478+0000 7f6245400640 1 -- 172.21.15.145:0/231562105 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6238007070 con 0x7f6228003a30 2024-05-09T04:33:48.479 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.478+0000 7f624d000640 1 -- 172.21.15.145:0/231562105 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6228003a30 msgr2=0x7f6228003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:48.479 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.478+0000 7f624d000640 1 --2- 172.21.15.145:0/231562105 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6228003a30 0x7f6228003e30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f623800c190 tx=0x7f6228005830 comp rx=0 tx=0).stop 2024-05-09T04:33:48.479 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.479+0000 7f6246800640 1 -- 172.21.15.145:0/231562105 reap_dead start 2024-05-09T04:33:48.479 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.479+0000 7f624d000640 1 -- 172.21.15.145:0/231562105 shutdown_connections 2024-05-09T04:33:48.480 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.479+0000 7f624d000640 1 -- 172.21.15.145:0/231562105 >> 172.21.15.145:0/231562105 conn(0x7f624806c870 msgr2=0x7f624806cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:48.480 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.479+0000 7f624d000640 1 -- 172.21.15.145:0/231562105 shutdown_connections 2024-05-09T04:33:48.480 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.479+0000 7f624d000640 1 -- 172.21.15.145:0/231562105 wait complete. 2024-05-09T04:33:48.480 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.480+0000 7f624d000640 1 Processor -- start 2024-05-09T04:33:48.480 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.481+0000 7f624d000640 1 -- start start 2024-05-09T04:33:48.481 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.481+0000 7f624d000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6228003a30 0x7f624813e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:48.481 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.481+0000 7f624d000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f624813d8e0 0x7f624815ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:48.481 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.481+0000 7f624d000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62480b5170 0x7f62480b55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:48.482 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.481+0000 7f624d000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f62480affc0 con 0x7f6228003a30 2024-05-09T04:33:48.482 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.482+0000 7f624d000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6248145c60 con 0x7f62480b5170 2024-05-09T04:33:48.482 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.482+0000 7f624d000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f62480b0140 con 0x7f624813d8e0 2024-05-09T04:33:48.482 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.482+0000 7f6245e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6228003a30 0x7f624813e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:48.482 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.482+0000 7f6245e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6228003a30 0x7f624813e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:49266/0 (socket says 172.21.15.145:49266) 2024-05-09T04:33:48.482 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.482+0000 7f6245e00640 1 -- 172.21.15.145:0/2523919333 learned_addr learned my addr 172.21.15.145:0/2523919333 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:48.483 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.482+0000 7f6245e00640 1 -- 172.21.15.145:0/2523919333 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f624813d8e0 msgr2=0x7f624815ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:48.483 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.482+0000 7f6245e00640 1 --2- 172.21.15.145:0/2523919333 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f624813d8e0 0x7f624815ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:48.483 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.482+0000 7f6245e00640 1 -- 172.21.15.145:0/2523919333 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62480b5170 msgr2=0x7f62480b55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:48.483 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.482+0000 7f6245e00640 1 --2- 172.21.15.145:0/2523919333 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62480b5170 0x7f62480b55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:48.483 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.482+0000 7f6245e00640 1 -- 172.21.15.145:0/2523919333 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62481470d0 con 0x7f6228003a30 2024-05-09T04:33:48.483 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.482+0000 7f6245e00640 1 --2- 172.21.15.145:0/2523919333 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6228003a30 0x7f624813e070 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f623800c1f0 tx=0x7f6248072ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:48.483 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.483+0000 7f623e000640 1 -- 172.21.15.145:0/2523919333 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62380016b0 con 0x7f6228003a30 2024-05-09T04:33:48.484 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.483+0000 7f623e000640 1 -- 172.21.15.145:0/2523919333 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6238042070 con 0x7f6228003a30 2024-05-09T04:33:48.484 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.483+0000 7f623e000640 1 -- 172.21.15.145:0/2523919333 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6238007290 con 0x7f6228003a30 2024-05-09T04:33:48.484 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.483+0000 7f624d000640 1 -- 172.21.15.145:0/2523919333 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62481473b0 con 0x7f6228003a30 2024-05-09T04:33:48.484 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.483+0000 7f624d000640 1 -- 172.21.15.145:0/2523919333 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f624813e810 con 0x7f6228003a30 2024-05-09T04:33:48.484 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.484+0000 7f624d000640 1 -- 172.21.15.145:0/2523919333 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f624806a310 con 0x7f6228003a30 2024-05-09T04:33:48.485 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.485+0000 7f623e000640 1 -- 172.21.15.145:0/2523919333 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6238007430 con 0x7f6228003a30 2024-05-09T04:33:48.487 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.487+0000 7f623e000640 1 --2- 172.21.15.145:0/2523919333 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f62240762d0 0x7f6224078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:48.487 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.487+0000 7f623e000640 1 -- 172.21.15.145:0/2523919333 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f62380b7300 con 0x7f6228003a30 2024-05-09T04:33:48.487 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.487+0000 7f623fe00640 1 --2- 172.21.15.145:0/2523919333 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f62240762d0 0x7f6224078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:48.487 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.488+0000 7f623fe00640 1 --2- 172.21.15.145:0/2523919333 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f62240762d0 0x7f6224078790 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f62481434d0 tx=0x7f62300079b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:48.492 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.492+0000 7f623e000640 1 -- 172.21.15.145:0/2523919333 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6238005aa0 con 0x7f6228003a30 2024-05-09T04:33:48.744 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.744+0000 7f624d000640 1 -- 172.21.15.145:0/2523919333 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6248143220 con 0x7f6228003a30 2024-05-09T04:33:48.745 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.745+0000 7f623e000640 1 -- 172.21.15.145:0/2523919333 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3876 (secure 0 0 0) 0x7f6238080fa0 con 0x7f6228003a30 2024-05-09T04:33:48.745 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:33:48.745 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/1284540932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":1284540932},{"type":"v1","addr":"172.21.15.145:6835","nonce":1284540932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/1308656248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1308656248},{"type":"v1","addr":"172.21.15.145:6837","nonce":1308656248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3323136057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3323136057},{"type":"v1","addr":"172.21.15.192:6833","nonce":3323136057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4270,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1113351558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1113351558},{"type":"v1","addr":"172.21.15.192:6835","nonce":1113351558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-09T04:33:48.746 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 3 2024-05-09T04:33:48.748 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.748+0000 7f624d000640 1 -- 172.21.15.145:0/2523919333 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f62240762d0 msgr2=0x7f6224078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:48.748 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.748+0000 7f624d000640 1 --2- 172.21.15.145:0/2523919333 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f62240762d0 0x7f6224078790 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f62481434d0 tx=0x7f62300079b0 comp rx=0 tx=0).stop 2024-05-09T04:33:48.748 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.748+0000 7f624d000640 1 -- 172.21.15.145:0/2523919333 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6228003a30 msgr2=0x7f624813e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:48.748 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.748+0000 7f624d000640 1 --2- 172.21.15.145:0/2523919333 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6228003a30 0x7f624813e070 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f623800c1f0 tx=0x7f6248072ed0 comp rx=0 tx=0).stop 2024-05-09T04:33:48.748 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.748+0000 7f624d000640 1 -- 172.21.15.145:0/2523919333 shutdown_connections 2024-05-09T04:33:48.748 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.748+0000 7f624d000640 1 --2- 172.21.15.145:0/2523919333 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f62240762d0 0x7f6224078790 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:48.748 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.748+0000 7f624d000640 1 --2- 172.21.15.145:0/2523919333 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62480b5170 0x7f62480b55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:48.748 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.748+0000 7f624d000640 1 --2- 172.21.15.145:0/2523919333 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f624813d8e0 0x7f624815ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:48.749 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.748+0000 7f624d000640 1 --2- 172.21.15.145:0/2523919333 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6228003a30 0x7f624813e070 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:48.749 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.748+0000 7f624d000640 1 -- 172.21.15.145:0/2523919333 >> 172.21.15.145:0/2523919333 conn(0x7f624806c870 msgr2=0x7f62480b1f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:48.749 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.748+0000 7f624d000640 1 -- 172.21.15.145:0/2523919333 shutdown_connections 2024-05-09T04:33:48.749 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.748+0000 7f624d000640 1 -- 172.21.15.145:0/2523919333 wait complete. 2024-05-09T04:33:48.766 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-09T04:33:48.965 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.965+0000 7f797e400640 1 Processor -- start 2024-05-09T04:33:48.965 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.965+0000 7f797e400640 1 -- start start 2024-05-09T04:33:48.967 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.967+0000 7f797e400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f79780b4700 0x7f7978073c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:48.967 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.967+0000 7f797e400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f79780af610 0x7f79780afa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:48.968 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.967+0000 7f797e400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f79780b4ae0 0x7f79780b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:48.968 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.967+0000 7f797e400640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f79780aea70 con 0x7f79780af610 2024-05-09T04:33:48.968 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.967+0000 7f797e400640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f79780737f0 con 0x7f79780b4ae0 2024-05-09T04:33:48.968 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.967+0000 7f797e400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f7978072c00 con 0x7f79780b4700 2024-05-09T04:33:48.968 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.967+0000 7f7977e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f79780af610 0x7f79780afa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:48.968 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.967+0000 7f7977e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f79780af610 0x7f79780afa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:49274/0 (socket says 172.21.15.145:49274) 2024-05-09T04:33:48.968 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.967+0000 7f7977e00640 1 -- 172.21.15.145:0/2005270748 learned_addr learned my addr 172.21.15.145:0/2005270748 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:48.968 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.967+0000 7f797da00640 1 --2- 172.21.15.145:0/2005270748 >> v2:172.21.15.192:3300/0 conn(0x7f79780b4ae0 0x7f79780b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:48.968 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.968+0000 7f797d000640 1 --2- 172.21.15.145:0/2005270748 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f79780b4700 0x7f7978073c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:48.968 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.968+0000 7f7977e00640 1 -- 172.21.15.145:0/2005270748 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f79780b4700 msgr2=0x7f7978073c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:48.968 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.968+0000 7f7977e00640 1 --2- 172.21.15.145:0/2005270748 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f79780b4700 0x7f7978073c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:48.968 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.968+0000 7f7977e00640 1 -- 172.21.15.145:0/2005270748 >> v2:172.21.15.192:3300/0 conn(0x7f79780b4ae0 msgr2=0x7f79780b4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:48.968 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.968+0000 7f7977e00640 1 --2- 172.21.15.145:0/2005270748 >> v2:172.21.15.192:3300/0 conn(0x7f79780b4ae0 0x7f79780b4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:48.968 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.968+0000 7f7977e00640 1 -- 172.21.15.145:0/2005270748 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7978072ec0 con 0x7f79780af610 2024-05-09T04:33:48.969 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.969+0000 7f7977e00640 1 --2- 172.21.15.145:0/2005270748 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f79780af610 0x7f79780afa10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7978070600 tx=0x7f796402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66ad12503e6abee server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:48.969 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.969+0000 7f7977400640 1 -- 172.21.15.145:0/2005270748 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7964037070 con 0x7f79780af610 2024-05-09T04:33:48.969 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.969+0000 7f7977400640 1 -- 172.21.15.145:0/2005270748 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7964032440 con 0x7f79780af610 2024-05-09T04:33:48.969 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.969+0000 7f7977400640 1 -- 172.21.15.145:0/2005270748 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7964032810 con 0x7f79780af610 2024-05-09T04:33:48.970 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.970+0000 7f797e400640 1 -- 172.21.15.145:0/2005270748 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f79780af610 msgr2=0x7f79780afa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:48.970 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.970+0000 7f797e400640 1 --2- 172.21.15.145:0/2005270748 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f79780af610 0x7f79780afa10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7978070600 tx=0x7f796402a1e0 comp rx=0 tx=0).stop 2024-05-09T04:33:48.970 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.970+0000 7f797e400640 1 -- 172.21.15.145:0/2005270748 shutdown_connections 2024-05-09T04:33:48.970 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.970+0000 7f797e400640 1 --2- 172.21.15.145:0/2005270748 >> v2:172.21.15.192:3300/0 conn(0x7f79780b4ae0 0x7f79780b4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:48.971 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.970+0000 7f797e400640 1 --2- 172.21.15.145:0/2005270748 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f79780af610 0x7f79780afa10 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:48.971 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.970+0000 7f797e400640 1 --2- 172.21.15.145:0/2005270748 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f79780b4700 0x7f7978073c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:48.971 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.970+0000 7f797e400640 1 -- 172.21.15.145:0/2005270748 >> 172.21.15.145:0/2005270748 conn(0x7f797806c8a0 msgr2=0x7f797806ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:48.971 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.970+0000 7f797e400640 1 -- 172.21.15.145:0/2005270748 shutdown_connections 2024-05-09T04:33:48.971 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.970+0000 7f797e400640 1 -- 172.21.15.145:0/2005270748 wait complete. 2024-05-09T04:33:48.971 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.972+0000 7f797e400640 1 Processor -- start 2024-05-09T04:33:48.971 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.972+0000 7f797e400640 1 -- start start 2024-05-09T04:33:48.973 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.973+0000 7f797e400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f79780af610 0x7f797814dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:48.973 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.973+0000 7f797e400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f79780b4700 0x7f797809cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:48.973 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.973+0000 7f797e400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79780b4ae0 0x7f797809d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:48.973 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.973+0000 7f797e400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f79780741c0 con 0x7f79780af610 2024-05-09T04:33:48.973 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.973+0000 7f797e400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7978073480 con 0x7f79780b4ae0 2024-05-09T04:33:48.973 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.973+0000 7f797e400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f79780b5470 con 0x7f79780b4700 2024-05-09T04:33:48.973 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.973+0000 7f797d000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f79780af610 0x7f797814dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:48.973 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.973+0000 7f797d000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f79780af610 0x7f797814dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:49290/0 (socket says 172.21.15.145:49290) 2024-05-09T04:33:48.973 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.973+0000 7f797d000640 1 -- 172.21.15.145:0/208699657 learned_addr learned my addr 172.21.15.145:0/208699657 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:48.975 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.974+0000 7f7977e00640 1 --2- 172.21.15.145:0/208699657 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f79780b4700 0x7f797809cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:48.975 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.974+0000 7f797d000640 1 -- 172.21.15.145:0/208699657 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f79780b4700 msgr2=0x7f797809cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:48.975 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.974+0000 7f797d000640 1 --2- 172.21.15.145:0/208699657 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f79780b4700 0x7f797809cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:48.975 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.974+0000 7f797d000640 1 -- 172.21.15.145:0/208699657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79780b4ae0 msgr2=0x7f797809d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:48.975 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.974+0000 7f797d000640 1 --2- 172.21.15.145:0/208699657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79780b4ae0 0x7f797809d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:48.975 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.974+0000 7f797d000640 1 -- 172.21.15.145:0/208699657 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f797814e7f0 con 0x7f79780af610 2024-05-09T04:33:48.975 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.974+0000 7f797d000640 1 --2- 172.21.15.145:0/208699657 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f79780af610 0x7f797814dce0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f796c00d800 tx=0x7f796c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:48.975 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.974+0000 7f7974c00640 1 -- 172.21.15.145:0/208699657 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796c00b6c0 con 0x7f79780af610 2024-05-09T04:33:48.975 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.974+0000 7f7974c00640 1 -- 172.21.15.145:0/208699657 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f796c019070 con 0x7f79780af610 2024-05-09T04:33:48.975 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.974+0000 7f7974c00640 1 -- 172.21.15.145:0/208699657 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796c00ba80 con 0x7f79780af610 2024-05-09T04:33:48.975 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.974+0000 7f797e400640 1 -- 172.21.15.145:0/208699657 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f797814ead0 con 0x7f79780af610 2024-05-09T04:33:48.975 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.974+0000 7f797e400640 1 -- 172.21.15.145:0/208699657 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f797814e220 con 0x7f79780af610 2024-05-09T04:33:48.976 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.975+0000 7f797e400640 1 -- 172.21.15.145:0/208699657 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f797806a310 con 0x7f79780af610 2024-05-09T04:33:48.976 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.977+0000 7f7974c00640 1 -- 172.21.15.145:0/208699657 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f796c00bc20 con 0x7f79780af610 2024-05-09T04:33:48.977 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.977+0000 7f7974c00640 1 --2- 172.21.15.145:0/208699657 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f79580762d0 0x7f7958078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:48.977 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.978+0000 7f7974c00640 1 -- 172.21.15.145:0/208699657 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f796c0978e0 con 0x7f79780af610 2024-05-09T04:33:48.977 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.978+0000 7f7977e00640 1 --2- 172.21.15.145:0/208699657 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f79580762d0 0x7f7958078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:48.978 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.978+0000 7f7977e00640 1 --2- 172.21.15.145:0/208699657 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f79580762d0 0x7f7958078790 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7964034000 tx=0x7f796402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:48.982 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:48.982+0000 7f7974c00640 1 -- 172.21.15.145:0/208699657 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f796c061460 con 0x7f79780af610 2024-05-09T04:33:49.190 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.191+0000 7f797e400640 1 -- 172.21.15.145:0/208699657 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f79780b5a00 con 0x7f79780af610 2024-05-09T04:33:49.191 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.192+0000 7f7974c00640 1 -- 172.21.15.145:0/208699657 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7f796c060d60 con 0x7f79780af610 2024-05-09T04:33:49.191 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:33:49.192 INFO:teuthology.orchestra.run.smithi145.stdout:[ 2024-05-09T04:33:49.192 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-05-09T04:33:49.192 INFO:teuthology.orchestra.run.smithi145.stdout: "poolnum": 1, 2024-05-09T04:33:49.192 INFO:teuthology.orchestra.run.smithi145.stdout: "poolname": ".mgr" 2024-05-09T04:33:49.192 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-05-09T04:33:49.192 INFO:teuthology.orchestra.run.smithi145.stdout:] 2024-05-09T04:33:49.194 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.194+0000 7f797e400640 1 -- 172.21.15.145:0/208699657 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f79580762d0 msgr2=0x7f7958078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:49.194 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.194+0000 7f797e400640 1 --2- 172.21.15.145:0/208699657 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f79580762d0 0x7f7958078790 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7964034000 tx=0x7f796402e000 comp rx=0 tx=0).stop 2024-05-09T04:33:49.194 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.194+0000 7f797e400640 1 -- 172.21.15.145:0/208699657 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f79780af610 msgr2=0x7f797814dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:49.194 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.194+0000 7f797e400640 1 --2- 172.21.15.145:0/208699657 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f79780af610 0x7f797814dce0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f796c00d800 tx=0x7f796c00dcc0 comp rx=0 tx=0).stop 2024-05-09T04:33:49.194 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.194+0000 7f797e400640 1 -- 172.21.15.145:0/208699657 shutdown_connections 2024-05-09T04:33:49.194 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.194+0000 7f797e400640 1 --2- 172.21.15.145:0/208699657 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f79580762d0 0x7f7958078790 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:49.195 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.194+0000 7f797e400640 1 --2- 172.21.15.145:0/208699657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79780b4ae0 0x7f797809d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:49.195 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.194+0000 7f797e400640 1 --2- 172.21.15.145:0/208699657 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f79780b4700 0x7f797809cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:49.195 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.194+0000 7f797e400640 1 --2- 172.21.15.145:0/208699657 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f79780af610 0x7f797814dce0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:49.195 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.194+0000 7f797e400640 1 -- 172.21.15.145:0/208699657 >> 172.21.15.145:0/208699657 conn(0x7f797806c8a0 msgr2=0x7f79780b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:49.195 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.194+0000 7f797e400640 1 -- 172.21.15.145:0/208699657 shutdown_connections 2024-05-09T04:33:49.195 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.194+0000 7f797e400640 1 -- 172.21.15.145:0/208699657 wait complete. 2024-05-09T04:33:49.212 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-09T04:33:49.212 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-09T04:33:49.411 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.411+0000 7f2aac200640 1 Processor -- start 2024-05-09T04:33:49.411 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.411+0000 7f2aac200640 1 -- start start 2024-05-09T04:33:49.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.413+0000 7f2aac200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2aa4146a90 0x7f2aa40729b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:49.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.413+0000 7f2aac200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f2aa4072ef0 0x7f2aa40cdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:49.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.413+0000 7f2aac200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2aa4146e70 con 0x7f2aa40ce2b0 2024-05-09T04:33:49.413 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.413+0000 7f2aac200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f2aa4160d30 con 0x7f2aa4072ef0 2024-05-09T04:33:49.413 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.413+0000 7f2aac200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f2aa4160f80 con 0x7f2aa4146a90 2024-05-09T04:33:49.414 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.413+0000 7f2aa8e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f2aa4072ef0 0x7f2aa40cdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:49.414 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.413+0000 7f2aa8e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f2aa4072ef0 0x7f2aa40cdd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:49304/0 (socket says 172.21.15.145:49304) 2024-05-09T04:33:49.414 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.413+0000 7f2aa8e00640 1 -- 172.21.15.145:0/1982226001 learned_addr learned my addr 172.21.15.145:0/1982226001 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:49.414 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.413+0000 7f2aaa200640 1 --1- 172.21.15.145:0/1982226001 >> v1:172.21.15.145:6789/0 conn(0x7f2aa40ce2b0 0x7f2aa40ced50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:40464/0 (socket says 172.21.15.145:40464) 2024-05-09T04:33:49.414 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.413+0000 7f2aa9800640 1 --2- 172.21.15.145:0/1982226001 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2aa4146a90 0x7f2aa40729b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:49.414 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.414+0000 7f2aa8e00640 1 -- 172.21.15.145:0/1982226001 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2aa4146a90 msgr2=0x7f2aa40729b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:49.414 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.414+0000 7f2aa8e00640 1 --2- 172.21.15.145:0/1982226001 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2aa4146a90 0x7f2aa40729b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:49.414 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.414+0000 7f2aa8e00640 1 -- 172.21.15.145:0/1982226001 >> v1:172.21.15.145:6789/0 conn(0x7f2aa40ce2b0 legacy=0x7f2aa40ced50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:49.414 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.414+0000 7f2aa8e00640 1 -- 172.21.15.145:0/1982226001 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aa4161260 con 0x7f2aa4072ef0 2024-05-09T04:33:49.415 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.414+0000 7f2aa3e00640 1 -- 172.21.15.145:0/1982226001 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 6581913 0 0) 0x7f2aa4146e70 con 0x7f2aa40ce2b0 2024-05-09T04:33:49.416 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.415+0000 7f2aa8e00640 1 --2- 172.21.15.145:0/1982226001 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2aa4072ef0 0x7f2aa40cdd70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2a90009870 tx=0x7f2a9002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45ec1bb2086d65c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:49.416 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.415+0000 7f2aa3e00640 1 -- 172.21.15.145:0/1982226001 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a90037070 con 0x7f2aa4072ef0 2024-05-09T04:33:49.416 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.415+0000 7f2aa3e00640 1 -- 172.21.15.145:0/1982226001 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a90032440 con 0x7f2aa4072ef0 2024-05-09T04:33:49.416 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.415+0000 7f2aa3e00640 1 -- 172.21.15.145:0/1982226001 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a90032810 con 0x7f2aa4072ef0 2024-05-09T04:33:49.416 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.416+0000 7f2aac200640 1 -- 172.21.15.145:0/1982226001 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2aa4072ef0 msgr2=0x7f2aa40cdd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:49.416 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.416+0000 7f2aac200640 1 --2- 172.21.15.145:0/1982226001 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2aa4072ef0 0x7f2aa40cdd70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2a90009870 tx=0x7f2a9002a250 comp rx=0 tx=0).stop 2024-05-09T04:33:49.416 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.416+0000 7f2aac200640 1 -- 172.21.15.145:0/1982226001 shutdown_connections 2024-05-09T04:33:49.416 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.416+0000 7f2aac200640 1 --2- 172.21.15.145:0/1982226001 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2aa4072ef0 0x7f2aa40cdd70 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:49.416 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.416+0000 7f2aac200640 1 --2- 172.21.15.145:0/1982226001 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2aa4146a90 0x7f2aa40729b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:49.417 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.416+0000 7f2aac200640 1 -- 172.21.15.145:0/1982226001 >> 172.21.15.145:0/1982226001 conn(0x7f2aa406c8d0 msgr2=0x7f2aa406cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:49.417 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.416+0000 7f2aac200640 1 -- 172.21.15.145:0/1982226001 shutdown_connections 2024-05-09T04:33:49.417 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.416+0000 7f2aac200640 1 -- 172.21.15.145:0/1982226001 wait complete. 2024-05-09T04:33:49.417 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.418+0000 7f2aac200640 1 Processor -- start 2024-05-09T04:33:49.417 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.418+0000 7f2aac200640 1 -- start start 2024-05-09T04:33:49.419 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.419+0000 7f2aac200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2aa40ce2b0 0x7f2aa415c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:49.420 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.419+0000 7f2aac200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2aa4146a90 0x7f2aa413f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:49.420 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.419+0000 7f2aac200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2aa413f9a0 0x7f2aa4141e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:49.420 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.419+0000 7f2aac200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa4161b00 con 0x7f2aa4146a90 2024-05-09T04:33:49.420 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.419+0000 7f2aac200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa4161820 con 0x7f2aa40ce2b0 2024-05-09T04:33:49.420 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.419+0000 7f2aac200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f2aa4072610 con 0x7f2aa413f9a0 2024-05-09T04:33:49.420 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.419+0000 7f2aa8e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2aa4146a90 0x7f2aa413f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:49.420 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.419+0000 7f2aa8e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2aa4146a90 0x7f2aa413f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:49306/0 (socket says 172.21.15.145:49306) 2024-05-09T04:33:49.420 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.419+0000 7f2aa8e00640 1 -- 172.21.15.145:0/1798128243 learned_addr learned my addr 172.21.15.145:0/1798128243 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:49.420 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.419+0000 7f2aa9800640 1 --2- 172.21.15.145:0/1798128243 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2aa40ce2b0 0x7f2aa415c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:49.420 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.420+0000 7f2aaa200640 1 --2- 172.21.15.145:0/1798128243 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2aa413f9a0 0x7f2aa4141e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:49.420 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.420+0000 7f2aa8e00640 1 -- 172.21.15.145:0/1798128243 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2aa413f9a0 msgr2=0x7f2aa4141e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:49.420 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.420+0000 7f2aa8e00640 1 --2- 172.21.15.145:0/1798128243 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2aa413f9a0 0x7f2aa4141e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:49.420 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.420+0000 7f2aa8e00640 1 -- 172.21.15.145:0/1798128243 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2aa40ce2b0 msgr2=0x7f2aa415c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:49.420 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.420+0000 7f2aa8e00640 1 --2- 172.21.15.145:0/1798128243 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2aa40ce2b0 0x7f2aa415c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:49.421 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.420+0000 7f2aa8e00640 1 -- 172.21.15.145:0/1798128243 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aa4142c00 con 0x7f2aa4146a90 2024-05-09T04:33:49.421 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.420+0000 7f2aaa200640 1 --2- 172.21.15.145:0/1798128243 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2aa413f9a0 0x7f2aa4141e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:33:49.421 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.420+0000 7f2aa8e00640 1 --2- 172.21.15.145:0/1798128243 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2aa4146a90 0x7f2aa413f460 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2a9002a290 tx=0x7f2a900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:49.421 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.420+0000 7f2aa1600640 1 -- 172.21.15.145:0/1798128243 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a90041020 con 0x7f2aa4146a90 2024-05-09T04:33:49.422 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.420+0000 7f2aa1600640 1 -- 172.21.15.145:0/1798128243 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a900306c0 con 0x7f2aa4146a90 2024-05-09T04:33:49.422 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.420+0000 7f2aa1600640 1 -- 172.21.15.145:0/1798128243 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a90037040 con 0x7f2aa4146a90 2024-05-09T04:33:49.422 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.420+0000 7f2aac200640 1 -- 172.21.15.145:0/1798128243 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2aa4142ee0 con 0x7f2aa4146a90 2024-05-09T04:33:49.422 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.420+0000 7f2aac200640 1 -- 172.21.15.145:0/1798128243 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2aa41434a0 con 0x7f2aa4146a90 2024-05-09T04:33:49.422 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.422+0000 7f2aac200640 1 -- 172.21.15.145:0/1798128243 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2aa406a310 con 0x7f2aa4146a90 2024-05-09T04:33:49.422 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.423+0000 7f2aa1600640 1 -- 172.21.15.145:0/1798128243 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2a90044020 con 0x7f2aa4146a90 2024-05-09T04:33:49.423 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.423+0000 7f2aa1600640 1 --2- 172.21.15.145:0/1798128243 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f2a84076300 0x7f2a840787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:49.423 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.424+0000 7f2aa1600640 1 -- 172.21.15.145:0/1798128243 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f2a900b7ac0 con 0x7f2aa4146a90 2024-05-09T04:33:49.423 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.424+0000 7f2aa9800640 1 --2- 172.21.15.145:0/1798128243 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f2a84076300 0x7f2a840787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:49.424 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.424+0000 7f2aa9800640 1 --2- 172.21.15.145:0/1798128243 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f2a84076300 0x7f2a840787c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2a98006fd0 tx=0x7f2a98008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:49.427 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.428+0000 7f2aa1600640 1 -- 172.21.15.145:0/1798128243 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2a9004b030 con 0x7f2aa4146a90 2024-05-09T04:33:49.635 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:49.635+0000 7f2aac200640 1 -- 172.21.15.145:0/1798128243 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f2aa4072b50 con 0x7f2aa4146a90 2024-05-09T04:33:50.129 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.128+0000 7f2aa1600640 1 -- 172.21.15.145:0/1798128243 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7f2a900816d0 con 0x7f2aa4146a90 2024-05-09T04:33:50.129 INFO:teuthology.orchestra.run.smithi145.stderr:pool 'cephfs_metadata' created 2024-05-09T04:33:50.133 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.133+0000 7f2aac200640 1 -- 172.21.15.145:0/1798128243 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f2a84076300 msgr2=0x7f2a840787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:50.133 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.133+0000 7f2aac200640 1 --2- 172.21.15.145:0/1798128243 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f2a84076300 0x7f2a840787c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2a98006fd0 tx=0x7f2a98008040 comp rx=0 tx=0).stop 2024-05-09T04:33:50.133 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.133+0000 7f2aac200640 1 -- 172.21.15.145:0/1798128243 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2aa4146a90 msgr2=0x7f2aa413f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:50.133 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.133+0000 7f2aac200640 1 --2- 172.21.15.145:0/1798128243 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2aa4146a90 0x7f2aa413f460 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2a9002a290 tx=0x7f2a900046c0 comp rx=0 tx=0).stop 2024-05-09T04:33:50.133 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.133+0000 7f2aac200640 1 -- 172.21.15.145:0/1798128243 shutdown_connections 2024-05-09T04:33:50.133 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.133+0000 7f2aac200640 1 --2- 172.21.15.145:0/1798128243 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f2a84076300 0x7f2a840787c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:50.134 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.133+0000 7f2aac200640 1 --2- 172.21.15.145:0/1798128243 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2aa413f9a0 0x7f2aa4141e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:50.134 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.133+0000 7f2aac200640 1 --2- 172.21.15.145:0/1798128243 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2aa4146a90 0x7f2aa413f460 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:50.134 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.133+0000 7f2aac200640 1 --2- 172.21.15.145:0/1798128243 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2aa40ce2b0 0x7f2aa415c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:50.134 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.133+0000 7f2aac200640 1 -- 172.21.15.145:0/1798128243 >> 172.21.15.145:0/1798128243 conn(0x7f2aa406c8d0 msgr2=0x7f2aa41443b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:50.134 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.133+0000 7f2aac200640 1 -- 172.21.15.145:0/1798128243 shutdown_connections 2024-05-09T04:33:50.134 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.134+0000 7f2aac200640 1 -- 172.21.15.145:0/1798128243 wait complete. 2024-05-09T04:33:50.163 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-09T04:33:50.355 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.355+0000 7fac8ec00640 1 Processor -- start 2024-05-09T04:33:50.355 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.355+0000 7fac8ec00640 1 -- start start 2024-05-09T04:33:50.357 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.356+0000 7fac8ec00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fac8815f010 0x7fac88073070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:50.357 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.356+0000 7fac8ec00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fac880b4400 0x7fac8806e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:50.357 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.356+0000 7fac8ec00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac881601a0 con 0x7fac8806df10 2024-05-09T04:33:50.357 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.356+0000 7fac8ec00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fac8806e8b0 con 0x7fac880b4400 2024-05-09T04:33:50.357 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.356+0000 7fac8ec00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fac88160480 con 0x7fac8815f010 2024-05-09T04:33:50.357 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.357+0000 7fac8ce00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7fac8806df10 0x7fac880b3cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:40472/0 (socket says 172.21.15.145:40472) 2024-05-09T04:33:50.357 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.357+0000 7fac8ce00640 1 -- 172.21.15.145:0/3083546950 learned_addr learned my addr 172.21.15.145:0/3083546950 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:33:50.357 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.357+0000 7fac8d800640 1 --2- 172.21.15.145:0/3083546950 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fac8815f010 0x7fac88073070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:50.357 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.357+0000 7fac8e200640 1 --2- 172.21.15.145:0/3083546950 >> v2:172.21.15.192:3300/0 conn(0x7fac880b4400 0x7fac8806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:50.359 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.357+0000 7fac87e00640 1 -- 172.21.15.145:0/3083546950 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2594831434 0 0) 0x7fac881601a0 con 0x7fac8806df10 2024-05-09T04:33:50.359 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.358+0000 7fac87e00640 1 -- 172.21.15.145:0/3083546950 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fac70003520 con 0x7fac8806df10 2024-05-09T04:33:50.359 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.358+0000 7fac87e00640 1 -- 172.21.15.145:0/3083546950 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3619744953 0 0) 0x7fac70003520 con 0x7fac8806df10 2024-05-09T04:33:50.360 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.358+0000 7fac87e00640 1 -- 172.21.15.145:0/3083546950 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fac8815f010 msgr2=0x7fac88073070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:50.360 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.358+0000 7fac87e00640 1 --2- 172.21.15.145:0/3083546950 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fac8815f010 0x7fac88073070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:50.360 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.358+0000 7fac87e00640 1 -- 172.21.15.145:0/3083546950 >> v2:172.21.15.192:3300/0 conn(0x7fac880b4400 msgr2=0x7fac8806e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:50.360 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.358+0000 7fac87e00640 1 --2- 172.21.15.145:0/3083546950 >> v2:172.21.15.192:3300/0 conn(0x7fac880b4400 0x7fac8806e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:50.360 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.358+0000 7fac87e00640 1 -- 172.21.15.145:0/3083546950 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac88072870 con 0x7fac8806df10 2024-05-09T04:33:50.360 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.358+0000 7fac87e00640 1 -- 172.21.15.145:0/3083546950 <== mon.0 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3474413261 0 0) 0x7fac740016f0 con 0x7fac8806df10 2024-05-09T04:33:50.360 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.358+0000 7fac87e00640 1 -- 172.21.15.145:0/3083546950 >> v1:172.21.15.145:6789/0 conn(0x7fac8806df10 legacy=0x7fac880b3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:50.360 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.358+0000 7fac87e00640 1 --2- 172.21.15.145:0/3083546950 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fac70003a00 0x7fac70003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:50.360 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.358+0000 7fac87e00640 1 --2- 172.21.15.145:0/3083546950 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fac70004800 0x7fac70004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:50.360 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.358+0000 7fac87e00640 1 --2- 172.21.15.145:0/3083546950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac700051c0 0x7fac7000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:50.360 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.358+0000 7fac87e00640 1 -- 172.21.15.145:0/3083546950 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fac88160480 con 0x7fac70004800 2024-05-09T04:33:50.360 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.358+0000 7fac87e00640 1 -- 172.21.15.145:0/3083546950 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fac7000bf00 con 0x7fac700051c0 2024-05-09T04:33:50.360 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.358+0000 7fac87e00640 1 -- 172.21.15.145:0/3083546950 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fac7000c0d0 con 0x7fac70003a00 2024-05-09T04:33:50.360 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.358+0000 7fac8d800640 1 --2- 172.21.15.145:0/3083546950 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fac8815f010 0x7fac88073070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:33:50.361 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.358+0000 7fac8ce00640 1 --2- 172.21.15.145:0/3083546950 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fac70004800 0x7fac70004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:50.361 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.359+0000 7fac8ce00640 1 -- 172.21.15.145:0/3083546950 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fac70003a00 msgr2=0x7fac70003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:50.361 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.359+0000 7fac8ce00640 1 --2- 172.21.15.145:0/3083546950 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fac70003a00 0x7fac70003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:50.361 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.359+0000 7fac8ce00640 1 -- 172.21.15.145:0/3083546950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac700051c0 msgr2=0x7fac7000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:50.361 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.359+0000 7fac8ce00640 1 --2- 172.21.15.145:0/3083546950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac700051c0 0x7fac7000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:50.361 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.359+0000 7fac8ce00640 1 -- 172.21.15.145:0/3083546950 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac70003520 con 0x7fac70004800 2024-05-09T04:33:50.361 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.360+0000 7fac8ce00640 1 --2- 172.21.15.145:0/3083546950 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fac70004800 0x7fac70004c80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fac7400c2c0 tx=0x7fac70005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdd6d91371331dc4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:50.361 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.360+0000 7fac87e00640 1 -- 172.21.15.145:0/3083546950 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac74007d50 con 0x7fac70004800 2024-05-09T04:33:50.361 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.360+0000 7fac87e00640 1 -- 172.21.15.145:0/3083546950 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac74005ad0 con 0x7fac70004800 2024-05-09T04:33:50.361 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.360+0000 7fac87e00640 1 -- 172.21.15.145:0/3083546950 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac74007070 con 0x7fac70004800 2024-05-09T04:33:50.362 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.360+0000 7fac8ec00640 1 -- 172.21.15.145:0/3083546950 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fac70004800 msgr2=0x7fac70004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:50.362 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.360+0000 7fac8ec00640 1 --2- 172.21.15.145:0/3083546950 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fac70004800 0x7fac70004c80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fac7400c2c0 tx=0x7fac70005830 comp rx=0 tx=0).stop 2024-05-09T04:33:50.362 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.360+0000 7fac8e200640 1 -- 172.21.15.145:0/3083546950 reap_dead start 2024-05-09T04:33:50.363 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.360+0000 7fac8ec00640 1 -- 172.21.15.145:0/3083546950 shutdown_connections 2024-05-09T04:33:50.363 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.360+0000 7fac8ec00640 1 -- 172.21.15.145:0/3083546950 >> 172.21.15.145:0/3083546950 conn(0x7fac880b29f0 msgr2=0x7fac880b2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:50.363 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.360+0000 7fac8ec00640 1 -- 172.21.15.145:0/3083546950 shutdown_connections 2024-05-09T04:33:50.363 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.361+0000 7fac8ec00640 1 -- 172.21.15.145:0/3083546950 wait complete. 2024-05-09T04:33:50.363 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.362+0000 7fac8ec00640 1 Processor -- start 2024-05-09T04:33:50.363 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.362+0000 7fac8ec00640 1 -- start start 2024-05-09T04:33:50.365 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.363+0000 7fac8ec00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fac70004800 0x7fac8814c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:50.365 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.363+0000 7fac8ec00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac8809ba10 0x7fac880cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:50.365 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.363+0000 7fac8ec00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fac88159c90 0x7fac880cc800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:50.365 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.363+0000 7fac8ec00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fac88073070 con 0x7fac88159c90 2024-05-09T04:33:50.365 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.363+0000 7fac8ec00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fac880702e0 con 0x7fac8809ba10 2024-05-09T04:33:50.365 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.363+0000 7fac8ec00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fac880731f0 con 0x7fac70004800 2024-05-09T04:33:50.365 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.363+0000 7fac8e200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fac88159c90 0x7fac880cc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:50.365 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.364+0000 7fac8e200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fac88159c90 0x7fac880cc800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:49322/0 (socket says 172.21.15.145:49322) 2024-05-09T04:33:50.366 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.364+0000 7fac8e200640 1 -- 172.21.15.145:0/2415124719 learned_addr learned my addr 172.21.15.145:0/2415124719 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:50.366 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.364+0000 7fac8d800640 1 --2- 172.21.15.145:0/2415124719 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fac70004800 0x7fac8814c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:50.366 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.364+0000 7fac8ce00640 1 --2- 172.21.15.145:0/2415124719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac8809ba10 0x7fac880cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:50.366 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.364+0000 7fac8e200640 1 -- 172.21.15.145:0/2415124719 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fac70004800 msgr2=0x7fac8814c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:50.366 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.364+0000 7fac8e200640 1 --2- 172.21.15.145:0/2415124719 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fac70004800 0x7fac8814c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:50.366 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.364+0000 7fac8e200640 1 -- 172.21.15.145:0/2415124719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac8809ba10 msgr2=0x7fac880cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:50.366 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.364+0000 7fac8e200640 1 --2- 172.21.15.145:0/2415124719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac8809ba10 0x7fac880cc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:50.366 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.364+0000 7fac8e200640 1 -- 172.21.15.145:0/2415124719 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac8809bf10 con 0x7fac88159c90 2024-05-09T04:33:50.366 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.364+0000 7fac8e200640 1 --2- 172.21.15.145:0/2415124719 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fac88159c90 0x7fac880cc800 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fac880b47e0 tx=0x7fac8000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:50.366 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.364+0000 7fac85600640 1 -- 172.21.15.145:0/2415124719 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac8000ba80 con 0x7fac88159c90 2024-05-09T04:33:50.366 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.365+0000 7fac85600640 1 -- 172.21.15.145:0/2415124719 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac8001a070 con 0x7fac88159c90 2024-05-09T04:33:50.366 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.365+0000 7fac85600640 1 -- 172.21.15.145:0/2415124719 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac80014400 con 0x7fac88159c90 2024-05-09T04:33:50.366 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.365+0000 7fac8ec00640 1 -- 172.21.15.145:0/2415124719 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac8809c1f0 con 0x7fac88159c90 2024-05-09T04:33:50.366 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.365+0000 7fac8ec00640 1 -- 172.21.15.145:0/2415124719 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac8809c7b0 con 0x7fac88159c90 2024-05-09T04:33:50.367 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.366+0000 7fac8ec00640 1 -- 172.21.15.145:0/2415124719 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac880cd460 con 0x7fac88159c90 2024-05-09T04:33:50.367 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.367+0000 7fac85600640 1 -- 172.21.15.145:0/2415124719 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fac80005d10 con 0x7fac88159c90 2024-05-09T04:33:50.367 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.368+0000 7fac85600640 1 --2- 172.21.15.145:0/2415124719 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fac68076300 0x7fac680787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:50.368 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.368+0000 7fac85600640 1 -- 172.21.15.145:0/2415124719 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7fac800975d0 con 0x7fac88159c90 2024-05-09T04:33:50.368 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.368+0000 7fac8d800640 1 --2- 172.21.15.145:0/2415124719 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fac68076300 0x7fac680787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:50.368 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.369+0000 7fac8d800640 1 --2- 172.21.15.145:0/2415124719 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fac68076300 0x7fac680787c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fac7c0047b0 tx=0x7fac7c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:50.373 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.373+0000 7fac85600640 1 -- 172.21.15.145:0/2415124719 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fac880cd460 con 0x7fac88159c90 2024-05-09T04:33:50.579 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:50.579+0000 7fac8ec00640 1 -- 172.21.15.145:0/2415124719 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fac8809c9f0 con 0x7fac88159c90 2024-05-09T04:33:51.136 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.136+0000 7fac85600640 1 -- 172.21.15.145:0/2415124719 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) ==== 170+0+0 (secure 0 0 0) 0x7fac80024020 con 0x7fac88159c90 2024-05-09T04:33:51.136 INFO:teuthology.orchestra.run.smithi145.stderr:pool 'cephfs_data' created 2024-05-09T04:33:51.140 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.140+0000 7fac8ec00640 1 -- 172.21.15.145:0/2415124719 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fac68076300 msgr2=0x7fac680787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:51.140 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.140+0000 7fac8ec00640 1 --2- 172.21.15.145:0/2415124719 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fac68076300 0x7fac680787c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fac7c0047b0 tx=0x7fac7c008040 comp rx=0 tx=0).stop 2024-05-09T04:33:51.140 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.141+0000 7fac8ec00640 1 -- 172.21.15.145:0/2415124719 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fac88159c90 msgr2=0x7fac880cc800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:51.140 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.141+0000 7fac8ec00640 1 --2- 172.21.15.145:0/2415124719 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fac88159c90 0x7fac880cc800 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fac880b47e0 tx=0x7fac8000b480 comp rx=0 tx=0).stop 2024-05-09T04:33:51.143 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.142+0000 7fac8ec00640 1 -- 172.21.15.145:0/2415124719 shutdown_connections 2024-05-09T04:33:51.143 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.142+0000 7fac8ec00640 1 --2- 172.21.15.145:0/2415124719 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fac68076300 0x7fac680787c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:51.143 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.142+0000 7fac8ec00640 1 --2- 172.21.15.145:0/2415124719 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fac88159c90 0x7fac880cc800 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:51.143 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.142+0000 7fac8ec00640 1 --2- 172.21.15.145:0/2415124719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac8809ba10 0x7fac880cc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:51.143 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.142+0000 7fac8ec00640 1 --2- 172.21.15.145:0/2415124719 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fac70004800 0x7fac8814c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:51.143 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.142+0000 7fac8ec00640 1 -- 172.21.15.145:0/2415124719 >> 172.21.15.145:0/2415124719 conn(0x7fac880b29f0 msgr2=0x7fac8806ecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:51.144 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.145+0000 7fac8ec00640 1 -- 172.21.15.145:0/2415124719 shutdown_connections 2024-05-09T04:33:51.146 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.145+0000 7fac8ec00640 1 -- 172.21.15.145:0/2415124719 wait complete. 2024-05-09T04:33:51.185 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-09T04:33:51.358 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.358+0000 7f17af400640 1 Processor -- start 2024-05-09T04:33:51.358 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.359+0000 7f17af400640 1 -- start start 2024-05-09T04:33:51.360 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.360+0000 7f17af400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f17a8143450 0x7f17a8142820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:51.360 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.360+0000 7f17af400640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17a806e560 con 0x7f17a8142d60 2024-05-09T04:33:51.361 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.360+0000 7f17af400640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17a8140490 con 0x7f17a81400b0 2024-05-09T04:33:51.361 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.360+0000 7f17af400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f17a80d0290 con 0x7f17a8143450 2024-05-09T04:33:51.361 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.360+0000 7f17ad400640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f17a8142d60 0x7f17a806ea10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:33856/0 (socket says 172.21.15.145:33856) 2024-05-09T04:33:51.361 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.360+0000 7f17ad400640 1 -- 172.21.15.145:0/212108240 learned_addr learned my addr 172.21.15.145:0/212108240 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:33:51.361 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.361+0000 7f17a7e00640 1 --2- 172.21.15.145:0/212108240 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f17a8143450 0x7f17a8142820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:51.361 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.361+0000 7f17a7400640 1 -- 172.21.15.145:0/212108240 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3829769719 0 0) 0x7f17a806e560 con 0x7f17a8142d60 2024-05-09T04:33:51.361 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.361+0000 7f17a7400640 1 -- 172.21.15.145:0/212108240 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f178c003520 con 0x7f17a8142d60 2024-05-09T04:33:51.363 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.361+0000 7f17a7400640 1 -- 172.21.15.145:0/212108240 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1493632086 0 0) 0x7f178c003520 con 0x7f17a8142d60 2024-05-09T04:33:51.363 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.361+0000 7f17a7400640 1 -- 172.21.15.145:0/212108240 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f17a8143450 msgr2=0x7f17a8142820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:51.363 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.361+0000 7f17a7400640 1 --2- 172.21.15.145:0/212108240 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f17a8143450 0x7f17a8142820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:51.363 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.361+0000 7f17a7400640 1 -- 172.21.15.145:0/212108240 >> v1:172.21.15.192:6789/0 conn(0x7f17a81400b0 legacy=0x7f17a80cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:51.363 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.361+0000 7f17a7400640 1 -- 172.21.15.145:0/212108240 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17a80d0570 con 0x7f17a8142d60 2024-05-09T04:33:51.363 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.361+0000 7f17a7400640 1 -- 172.21.15.145:0/212108240 <== mon.0 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3474413261 0 0) 0x7f17a00016f0 con 0x7f17a8142d60 2024-05-09T04:33:51.364 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.361+0000 7f17a7e00640 1 --2- 172.21.15.145:0/212108240 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f17a8143450 0x7f17a8142820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:33:51.364 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.361+0000 7f17a7400640 1 -- 172.21.15.145:0/212108240 >> v1:172.21.15.145:6789/0 conn(0x7f17a8142d60 legacy=0x7f17a806ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:51.364 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.361+0000 7f17a7400640 1 --2- 172.21.15.145:0/212108240 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f178c003a30 0x7f178c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:51.364 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.361+0000 7f17a7400640 1 --2- 172.21.15.145:0/212108240 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f17a8143450 0x7f178c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:51.364 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.362+0000 7f17a7400640 1 --2- 172.21.15.145:0/212108240 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f178c004d40 0x7f178c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:51.364 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.362+0000 7f17a7400640 1 -- 172.21.15.145:0/212108240 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f17a80d0290 con 0x7f178c004d40 2024-05-09T04:33:51.364 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.362+0000 7f17a7400640 1 -- 172.21.15.145:0/212108240 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f178c00bbf0 con 0x7f178c003a30 2024-05-09T04:33:51.364 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.362+0000 7f17a7400640 1 -- 172.21.15.145:0/212108240 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f178c00bdf0 con 0x7f17a8143450 2024-05-09T04:33:51.364 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.362+0000 7f17ad400640 1 --2- 172.21.15.145:0/212108240 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f178c004d40 0x7f178c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:51.364 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.362+0000 7f17a7e00640 1 --2- 172.21.15.145:0/212108240 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f17a8143450 0x7f178c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:51.364 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.362+0000 7f17aca00640 1 --2- 172.21.15.145:0/212108240 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f178c003a30 0x7f178c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:51.364 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.362+0000 7f17ad400640 1 -- 172.21.15.145:0/212108240 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f17a8143450 msgr2=0x7f178c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:51.364 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.362+0000 7f17ad400640 1 --2- 172.21.15.145:0/212108240 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f17a8143450 0x7f178c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:51.365 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.362+0000 7f17ad400640 1 -- 172.21.15.145:0/212108240 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f178c003a30 msgr2=0x7f178c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:51.365 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.362+0000 7f17ad400640 1 --2- 172.21.15.145:0/212108240 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f178c003a30 0x7f178c003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:51.365 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.362+0000 7f17ad400640 1 -- 172.21.15.145:0/212108240 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f178c003520 con 0x7f178c004d40 2024-05-09T04:33:51.365 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.363+0000 7f17ad400640 1 --2- 172.21.15.145:0/212108240 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f178c004d40 0x7f178c0051c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f17a000c240 tx=0x7f178c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=921c19a01d3ea9aa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:51.365 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.363+0000 7f17a7400640 1 -- 172.21.15.145:0/212108240 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a0007d70 con 0x7f178c004d40 2024-05-09T04:33:51.365 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.363+0000 7f17a7400640 1 -- 172.21.15.145:0/212108240 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17a0005af0 con 0x7f178c004d40 2024-05-09T04:33:51.365 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.363+0000 7f17a7400640 1 -- 172.21.15.145:0/212108240 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a0007070 con 0x7f178c004d40 2024-05-09T04:33:51.366 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.364+0000 7f17af400640 1 -- 172.21.15.145:0/212108240 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f178c004d40 msgr2=0x7f178c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:51.366 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.364+0000 7f17af400640 1 --2- 172.21.15.145:0/212108240 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f178c004d40 0x7f178c0051c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f17a000c240 tx=0x7f178c005830 comp rx=0 tx=0).stop 2024-05-09T04:33:51.366 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.364+0000 7f17ad400640 1 -- 172.21.15.145:0/212108240 reap_dead start 2024-05-09T04:33:51.366 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.364+0000 7f17af400640 1 -- 172.21.15.145:0/212108240 shutdown_connections 2024-05-09T04:33:51.367 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.364+0000 7f17af400640 1 -- 172.21.15.145:0/212108240 >> 172.21.15.145:0/212108240 conn(0x7f17a806cab0 msgr2=0x7f17a80d0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:51.367 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.364+0000 7f17af400640 1 -- 172.21.15.145:0/212108240 shutdown_connections 2024-05-09T04:33:51.367 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.364+0000 7f17af400640 1 -- 172.21.15.145:0/212108240 wait complete. 2024-05-09T04:33:51.367 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.366+0000 7f17af400640 1 Processor -- start 2024-05-09T04:33:51.367 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.366+0000 7f17af400640 1 -- start start 2024-05-09T04:33:51.369 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.367+0000 7f17af400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f178c004d40 0x7f17a8154f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:51.369 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.367+0000 7f17af400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f17a813cb50 0x7f17a80ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:51.369 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.367+0000 7f17af400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f17a80b0750 0x7f17a80b0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:51.369 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.367+0000 7f17af400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f17a80cfee0 con 0x7f17a80b0750 2024-05-09T04:33:51.369 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.367+0000 7f17af400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f17a80d0b30 con 0x7f178c004d40 2024-05-09T04:33:51.369 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.367+0000 7f17af400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f17a80afe40 con 0x7f17a813cb50 2024-05-09T04:33:51.369 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.367+0000 7f17ad400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f17a80b0750 0x7f17a80b0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:51.369 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.367+0000 7f17ad400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f17a80b0750 0x7f17a80b0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:44834/0 (socket says 172.21.15.145:44834) 2024-05-09T04:33:51.369 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.367+0000 7f17ad400640 1 -- 172.21.15.145:0/2582461285 learned_addr learned my addr 172.21.15.145:0/2582461285 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:51.369 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.368+0000 7f17a7e00640 1 --2- 172.21.15.145:0/2582461285 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f17a813cb50 0x7f17a80ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:51.369 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.368+0000 7f17aca00640 1 --2- 172.21.15.145:0/2582461285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f178c004d40 0x7f17a8154f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:51.369 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.368+0000 7f17ad400640 1 -- 172.21.15.145:0/2582461285 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f17a813cb50 msgr2=0x7f17a80ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:51.369 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.368+0000 7f17ad400640 1 --2- 172.21.15.145:0/2582461285 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f17a813cb50 0x7f17a80ae880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:51.369 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.368+0000 7f17ad400640 1 -- 172.21.15.145:0/2582461285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f178c004d40 msgr2=0x7f17a8154f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:51.370 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.368+0000 7f17ad400640 1 --2- 172.21.15.145:0/2582461285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f178c004d40 0x7f17a8154f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:51.370 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.368+0000 7f17ad400640 1 -- 172.21.15.145:0/2582461285 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17a80aef80 con 0x7f17a80b0750 2024-05-09T04:33:51.370 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.368+0000 7f17a7e00640 1 --2- 172.21.15.145:0/2582461285 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f17a813cb50 0x7f17a80ae880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:33:51.370 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.368+0000 7f17ad400640 1 --2- 172.21.15.145:0/2582461285 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f17a80b0750 0x7f17a80b0bd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f17a002adf0 tx=0x7f17a80d0460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:51.370 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.369+0000 7f17a4c00640 1 -- 172.21.15.145:0/2582461285 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a0001cb0 con 0x7f17a80b0750 2024-05-09T04:33:51.370 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.369+0000 7f17a4c00640 1 -- 172.21.15.145:0/2582461285 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17a0042070 con 0x7f17a80b0750 2024-05-09T04:33:51.370 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.369+0000 7f17a4c00640 1 -- 172.21.15.145:0/2582461285 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a00074b0 con 0x7f17a80b0750 2024-05-09T04:33:51.370 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.369+0000 7f17af400640 1 -- 172.21.15.145:0/2582461285 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17a80af260 con 0x7f17a80b0750 2024-05-09T04:33:51.370 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.369+0000 7f17af400640 1 -- 172.21.15.145:0/2582461285 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17a80af820 con 0x7f17a80b0750 2024-05-09T04:33:51.371 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.370+0000 7f17af400640 1 -- 172.21.15.145:0/2582461285 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17a806a310 con 0x7f17a80b0750 2024-05-09T04:33:51.371 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.371+0000 7f17a4c00640 1 -- 172.21.15.145:0/2582461285 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f17a0007650 con 0x7f17a80b0750 2024-05-09T04:33:51.372 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.372+0000 7f17a4c00640 1 --2- 172.21.15.145:0/2582461285 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f1784076300 0x7f17840787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:51.372 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.372+0000 7f17a4c00640 1 -- 172.21.15.145:0/2582461285 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7f17a00b7990 con 0x7f17a80b0750 2024-05-09T04:33:51.372 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.372+0000 7f17aca00640 1 --2- 172.21.15.145:0/2582461285 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f1784076300 0x7f17840787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:51.372 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.373+0000 7f17aca00640 1 --2- 172.21.15.145:0/2582461285 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f1784076300 0x7f17840787c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1798000f80 tx=0x7f1798009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:51.377 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.377+0000 7f17a4c00640 1 -- 172.21.15.145:0/2582461285 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f17a004c020 con 0x7f17a80b0750 2024-05-09T04:33:51.626 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:51.627+0000 7f17af400640 1 -- 172.21.15.145:0/2582461285 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f17a80b0020 con 0x7f17a80b0750 2024-05-09T04:33:52.161 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.160+0000 7f17a4c00640 1 -- 172.21.15.145:0/2582461285 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f17a00812b0 con 0x7f17a80b0750 2024-05-09T04:33:52.161 INFO:teuthology.orchestra.run.smithi145.stderr:new fs with metadata pool 2 and data pool 3 2024-05-09T04:33:52.165 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.164+0000 7f17af400640 1 -- 172.21.15.145:0/2582461285 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f1784076300 msgr2=0x7f17840787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:52.165 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.164+0000 7f17af400640 1 --2- 172.21.15.145:0/2582461285 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f1784076300 0x7f17840787c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1798000f80 tx=0x7f1798009040 comp rx=0 tx=0).stop 2024-05-09T04:33:52.165 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.164+0000 7f17af400640 1 -- 172.21.15.145:0/2582461285 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f17a80b0750 msgr2=0x7f17a80b0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:52.165 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.164+0000 7f17af400640 1 --2- 172.21.15.145:0/2582461285 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f17a80b0750 0x7f17a80b0bd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f17a002adf0 tx=0x7f17a80d0460 comp rx=0 tx=0).stop 2024-05-09T04:33:52.165 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.165+0000 7f17af400640 1 -- 172.21.15.145:0/2582461285 shutdown_connections 2024-05-09T04:33:52.165 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.165+0000 7f17af400640 1 --2- 172.21.15.145:0/2582461285 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f1784076300 0x7f17840787c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:52.165 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.165+0000 7f17af400640 1 --2- 172.21.15.145:0/2582461285 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f17a80b0750 0x7f17a80b0bd0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:52.165 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.165+0000 7f17af400640 1 --2- 172.21.15.145:0/2582461285 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f17a813cb50 0x7f17a80ae880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:52.165 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.165+0000 7f17af400640 1 --2- 172.21.15.145:0/2582461285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f178c004d40 0x7f17a8154f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:52.165 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.165+0000 7f17af400640 1 -- 172.21.15.145:0/2582461285 >> 172.21.15.145:0/2582461285 conn(0x7f17a806cab0 msgr2=0x7f17a806e910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:52.165 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.165+0000 7f17af400640 1 -- 172.21.15.145:0/2582461285 shutdown_connections 2024-05-09T04:33:52.165 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.165+0000 7f17af400640 1 -- 172.21.15.145:0/2582461285 wait complete. 2024-05-09T04:33:52.202 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T04:33:52.375 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.375+0000 7f23ca200640 1 Processor -- start 2024-05-09T04:33:52.375 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.375+0000 7f23ca200640 1 -- start start 2024-05-09T04:33:52.377 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.377+0000 7f23ca200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f23c406e860 0x7f23c406ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:52.377 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.377+0000 7f23ca200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23c415d780 0x7f23c4160c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:52.377 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.377+0000 7f23ca200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f23c40ce620 0x7f23c40ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:52.377 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.377+0000 7f23ca200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f23c406f180 con 0x7f23c40ce620 2024-05-09T04:33:52.377 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.377+0000 7f23ca200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f23c41615c0 con 0x7f23c406e860 2024-05-09T04:33:52.377 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.377+0000 7f23ca200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f23c41618a0 con 0x7f23c415d780 2024-05-09T04:33:52.377 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.377+0000 7f23c3e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f23c40ce620 0x7f23c40ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:52.377 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.377+0000 7f23c3e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f23c40ce620 0x7f23c40ceab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:44850/0 (socket says 172.21.15.145:44850) 2024-05-09T04:33:52.377 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.377+0000 7f23c3e00640 1 -- 172.21.15.145:0/3657167148 learned_addr learned my addr 172.21.15.145:0/3657167148 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:52.378 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.378+0000 7f23c3e00640 1 -- 172.21.15.145:0/3657167148 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23c415d780 msgr2=0x7f23c4160c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:52.378 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.378+0000 7f23c3e00640 1 --2- 172.21.15.145:0/3657167148 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23c415d780 0x7f23c4160c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:52.378 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.378+0000 7f23c3e00640 1 -- 172.21.15.145:0/3657167148 >> v2:172.21.15.192:3300/0 conn(0x7f23c406e860 msgr2=0x7f23c406ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:52.378 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.378+0000 7f23c3400640 1 --2- 172.21.15.145:0/3657167148 >> v2:172.21.15.192:3300/0 conn(0x7f23c406e860 0x7f23c406ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:52.378 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.378+0000 7f23c3e00640 1 --2- 172.21.15.145:0/3657167148 >> v2:172.21.15.192:3300/0 conn(0x7f23c406e860 0x7f23c406ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:52.378 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.378+0000 7f23c3e00640 1 -- 172.21.15.145:0/3657167148 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23c40b4110 con 0x7f23c40ce620 2024-05-09T04:33:52.379 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.379+0000 7f23c3e00640 1 --2- 172.21.15.145:0/3657167148 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f23c40ce620 0x7f23c40ceab0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f23c406f360 tx=0x7f23b402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f940c0022f07e29 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:52.380 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.379+0000 7f23c2000640 1 -- 172.21.15.145:0/3657167148 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b4037070 con 0x7f23c40ce620 2024-05-09T04:33:52.380 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.379+0000 7f23c2000640 1 -- 172.21.15.145:0/3657167148 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23b4032440 con 0x7f23c40ce620 2024-05-09T04:33:52.380 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.379+0000 7f23c2000640 1 -- 172.21.15.145:0/3657167148 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b4032810 con 0x7f23c40ce620 2024-05-09T04:33:52.380 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.379+0000 7f23ca200640 1 -- 172.21.15.145:0/3657167148 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f23c40ce620 msgr2=0x7f23c40ceab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:52.380 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.379+0000 7f23ca200640 1 --2- 172.21.15.145:0/3657167148 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f23c40ce620 0x7f23c40ceab0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f23c406f360 tx=0x7f23b402a1e0 comp rx=0 tx=0).stop 2024-05-09T04:33:52.380 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.380+0000 7f23ca200640 1 -- 172.21.15.145:0/3657167148 shutdown_connections 2024-05-09T04:33:52.380 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.380+0000 7f23ca200640 1 --2- 172.21.15.145:0/3657167148 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f23c40ce620 0x7f23c40ceab0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:52.380 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.380+0000 7f23ca200640 1 --2- 172.21.15.145:0/3657167148 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23c415d780 0x7f23c4160c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:52.380 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.380+0000 7f23ca200640 1 --2- 172.21.15.145:0/3657167148 >> v2:172.21.15.192:3300/0 conn(0x7f23c406e860 0x7f23c406ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:52.380 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.380+0000 7f23ca200640 1 -- 172.21.15.145:0/3657167148 >> 172.21.15.145:0/3657167148 conn(0x7f23c406c870 msgr2=0x7f23c406cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:52.380 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.380+0000 7f23ca200640 1 -- 172.21.15.145:0/3657167148 shutdown_connections 2024-05-09T04:33:52.380 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.380+0000 7f23ca200640 1 -- 172.21.15.145:0/3657167148 wait complete. 2024-05-09T04:33:52.381 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.381+0000 7f23ca200640 1 Processor -- start 2024-05-09T04:33:52.381 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.382+0000 7f23ca200640 1 -- start start 2024-05-09T04:33:52.382 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.382+0000 7f23ca200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f23c406e860 0x7f23c4141f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:52.382 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.382+0000 7f23ca200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23c40ce620 0x7f23c413c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:52.382 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.382+0000 7f23ca200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23c415d780 0x7f23c413cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:52.382 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.382+0000 7f23ca200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f23c4146d80 con 0x7f23c406e860 2024-05-09T04:33:52.383 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.382+0000 7f23ca200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f23c4146c00 con 0x7f23c415d780 2024-05-09T04:33:52.383 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.382+0000 7f23ca200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f23c4161190 con 0x7f23c40ce620 2024-05-09T04:33:52.383 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.383+0000 7f23c3400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f23c406e860 0x7f23c4141f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:52.383 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.383+0000 7f23c3400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f23c406e860 0x7f23c4141f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:44864/0 (socket says 172.21.15.145:44864) 2024-05-09T04:33:52.383 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.383+0000 7f23c3400640 1 -- 172.21.15.145:0/2353525290 learned_addr learned my addr 172.21.15.145:0/2353525290 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:52.384 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.383+0000 7f23c2a00640 1 --2- 172.21.15.145:0/2353525290 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23c40ce620 0x7f23c413c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:52.384 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.383+0000 7f23c3e00640 1 --2- 172.21.15.145:0/2353525290 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23c415d780 0x7f23c413cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:52.384 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.383+0000 7f23c3400640 1 -- 172.21.15.145:0/2353525290 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23c40ce620 msgr2=0x7f23c413c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:52.384 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.383+0000 7f23c3400640 1 --2- 172.21.15.145:0/2353525290 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23c40ce620 0x7f23c413c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:52.384 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.383+0000 7f23c3400640 1 -- 172.21.15.145:0/2353525290 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23c415d780 msgr2=0x7f23c413cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:52.385 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.383+0000 7f23c3400640 1 --2- 172.21.15.145:0/2353525290 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23c415d780 0x7f23c413cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:52.385 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.383+0000 7f23c3400640 1 -- 172.21.15.145:0/2353525290 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23c414e350 con 0x7f23c406e860 2024-05-09T04:33:52.385 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.384+0000 7f23c3400640 1 --2- 172.21.15.145:0/2353525290 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f23c406e860 0x7f23c4141f30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f23bc00cd00 tx=0x7f23bc00a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:52.385 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.384+0000 7f23bb400640 1 -- 172.21.15.145:0/2353525290 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23bc003990 con 0x7f23c406e860 2024-05-09T04:33:52.385 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.384+0000 7f23bb400640 1 -- 172.21.15.145:0/2353525290 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23bc003b30 con 0x7f23c406e860 2024-05-09T04:33:52.385 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.384+0000 7f23bb400640 1 -- 172.21.15.145:0/2353525290 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23bc004020 con 0x7f23c406e860 2024-05-09T04:33:52.385 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.384+0000 7f23ca200640 1 -- 172.21.15.145:0/2353525290 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23c414e630 con 0x7f23c406e860 2024-05-09T04:33:52.386 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.384+0000 7f23ca200640 1 -- 172.21.15.145:0/2353525290 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23c414ebf0 con 0x7f23c406e860 2024-05-09T04:33:52.386 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.385+0000 7f23ca200640 1 -- 172.21.15.145:0/2353525290 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23c406a310 con 0x7f23c406e860 2024-05-09T04:33:52.386 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.386+0000 7f23bb400640 1 -- 172.21.15.145:0/2353525290 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f23bc0041c0 con 0x7f23c406e860 2024-05-09T04:33:52.388 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.388+0000 7f23bb400640 1 --2- 172.21.15.145:0/2353525290 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f23a00762d0 0x7f23a0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:52.388 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.388+0000 7f23bb400640 1 -- 172.21.15.145:0/2353525290 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f23bc097ac0 con 0x7f23c406e860 2024-05-09T04:33:52.388 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.388+0000 7f23c2a00640 1 --2- 172.21.15.145:0/2353525290 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f23a00762d0 0x7f23a0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:52.388 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.389+0000 7f23c2a00640 1 --2- 172.21.15.145:0/2353525290 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f23a00762d0 0x7f23a0078790 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f23c40b5310 tx=0x7f23ac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:52.392 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.393+0000 7f23bb400640 1 -- 172.21.15.145:0/2353525290 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f23bc014030 con 0x7f23c406e860 2024-05-09T04:33:52.599 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.599+0000 7f23ca200640 1 -- 172.21.15.145:0/2353525290 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f23c415cf40 con 0x7f23c406e860 2024-05-09T04:33:52.601 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.602+0000 7f23bb400640 1 -- 172.21.15.145:0/2353525290 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16666 (secure 0 0 0) 0x7f23bc061410 con 0x7f23c406e860 2024-05-09T04:33:52.602 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:33:52.602 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":19,"fsid":"957ee6c1-073b-44cf-9b93-9491b3fdfc1d","created":"2024-05-09T04:33:26.544367+0000","modified":"2024-05-09T04:33:51.628969+0000","last_up_change":"2024-05-09T04:33:39.039318+0000","last_in_change":"2024-05-09T04:33:29.902361+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T04:33:39.795344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T04:33:49.651743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T04:33:50.596065+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22e550ad-847b-4952-80e6-63d859ed1a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6801","nonce":3182572912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6803","nonce":3182572912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6807","nonce":3182572912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6805","nonce":3182572912}]},"public_addr":"172.21.15.145:6801/3182572912","cluster_addr":"172.21.15.145:6803/3182572912","heartbeat_back_addr":"172.21.15.145:6807/3182572912","heartbeat_front_addr":"172.21.15.145:6805/3182572912","state":["exists","up"]},{"osd":1,"uuid":"0f6b9fca-0c41-4145-8d74-598ddb42f2a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6825","nonce":1845225555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6827","nonce":1845225555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6831","nonce":1845225555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6829","nonce":1845225555}]},"public_addr":"172.21.15.145:6825/1845225555","cluster_addr":"172.21.15.145:6827/1845225555","heartbeat_back_addr":"172.21.15.145:6831/1845225555","heartbeat_front_addr":"172.21.15.145:6829/1845225555","state":["exists","up"]},{"osd":2,"uuid":"b3b1f67e-5790-483f-8610-e05a5bce4c30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6817","nonce":2168958993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6819","nonce":2168958993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6823","nonce":2168958993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6821","nonce":2168958993}]},"public_addr":"172.21.15.145:6817/2168958993","cluster_addr":"172.21.15.145:6819/2168958993","heartbeat_back_addr":"172.21.15.145:6823/2168958993","heartbeat_front_addr":"172.21.15.145:6821/2168958993","state":["exists","up"]},{"osd":3,"uuid":"884787a5-e037-4d31-9aae-743030eba4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6809","nonce":337009826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6811","nonce":337009826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6815","nonce":337009826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6813","nonce":337009826}]},"public_addr":"172.21.15.145:6809/337009826","cluster_addr":"172.21.15.145:6811/337009826","heartbeat_back_addr":"172.21.15.145:6815/337009826","heartbeat_front_addr":"172.21.15.145:6813/337009826","state":["exists","up"]},{"osd":4,"uuid":"0759fc1e-c5fc-4a2f-86bb-e8eccd83f833","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6818","nonce":3117700995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6821","nonce":3117700995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6827","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6829","nonce":3117700995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6823","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6825","nonce":3117700995}]},"public_addr":"172.21.15.192:6818/3117700995","cluster_addr":"172.21.15.192:6821/3117700995","heartbeat_back_addr":"172.21.15.192:6829/3117700995","heartbeat_front_addr":"172.21.15.192:6825/3117700995","state":["exists","up"]},{"osd":5,"uuid":"9443cc22-9379-46e3-9173-98b592bf2c16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6801","nonce":1328373268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6803","nonce":1328373268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6807","nonce":1328373268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6805","nonce":1328373268}]},"public_addr":"172.21.15.192:6801/1328373268","cluster_addr":"172.21.15.192:6803/1328373268","heartbeat_back_addr":"172.21.15.192:6807/1328373268","heartbeat_front_addr":"172.21.15.192:6805/1328373268","state":["exists","up"]},{"osd":6,"uuid":"d05a3311-505a-45b5-85b9-aa3113e96987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6817","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6820","nonce":1868011057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6824","nonce":1868011057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6831","nonce":1868011057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6828","nonce":1868011057}]},"public_addr":"172.21.15.192:6820/1868011057","cluster_addr":"172.21.15.192:6824/1868011057","heartbeat_back_addr":"172.21.15.192:6831/1868011057","heartbeat_front_addr":"172.21.15.192:6828/1868011057","state":["exists","up"]},{"osd":7,"uuid":"c7c23dbc-2029-43ec-9f53-25f3d58019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6809","nonce":3368512157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6811","nonce":3368512157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6815","nonce":3368512157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6813","nonce":3368512157}]},"public_addr":"172.21.15.192:6809/3368512157","cluster_addr":"172.21.15.192:6811/3368512157","heartbeat_back_addr":"172.21.15.192:6815/3368512157","heartbeat_front_addr":"172.21.15.192:6813/3368512157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.091188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.032567+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:34.797468+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.228839+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.622400+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.473103+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.566092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.349395+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T04:33:52.605 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.605+0000 7f23ca200640 1 -- 172.21.15.145:0/2353525290 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f23a00762d0 msgr2=0x7f23a0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:52.605 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.605+0000 7f23ca200640 1 --2- 172.21.15.145:0/2353525290 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f23a00762d0 0x7f23a0078790 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f23c40b5310 tx=0x7f23ac008040 comp rx=0 tx=0).stop 2024-05-09T04:33:52.605 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.605+0000 7f23ca200640 1 -- 172.21.15.145:0/2353525290 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f23c406e860 msgr2=0x7f23c4141f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:52.605 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.605+0000 7f23ca200640 1 --2- 172.21.15.145:0/2353525290 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f23c406e860 0x7f23c4141f30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f23bc00cd00 tx=0x7f23bc00a5c0 comp rx=0 tx=0).stop 2024-05-09T04:33:52.605 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.605+0000 7f23ca200640 1 -- 172.21.15.145:0/2353525290 shutdown_connections 2024-05-09T04:33:52.605 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.605+0000 7f23ca200640 1 --2- 172.21.15.145:0/2353525290 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f23a00762d0 0x7f23a0078790 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:52.605 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.605+0000 7f23ca200640 1 --2- 172.21.15.145:0/2353525290 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23c415d780 0x7f23c413cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:52.605 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.605+0000 7f23ca200640 1 --2- 172.21.15.145:0/2353525290 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23c40ce620 0x7f23c413c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:52.605 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.605+0000 7f23ca200640 1 --2- 172.21.15.145:0/2353525290 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f23c406e860 0x7f23c4141f30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:52.606 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.605+0000 7f23ca200640 1 -- 172.21.15.145:0/2353525290 >> 172.21.15.145:0/2353525290 conn(0x7f23c406c870 msgr2=0x7f23c415ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:52.606 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.605+0000 7f23ca200640 1 -- 172.21.15.145:0/2353525290 shutdown_connections 2024-05-09T04:33:52.606 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.605+0000 7f23ca200640 1 -- 172.21.15.145:0/2353525290 wait complete. 2024-05-09T04:33:52.623 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T04:33:52.819 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.819+0000 7f8ae3200640 1 Processor -- start 2024-05-09T04:33:52.819 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.819+0000 7f8ae3200640 1 -- start start 2024-05-09T04:33:52.821 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.821+0000 7f8ae3200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f8ae40b4700 0x7f8ae4073c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:52.822 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.821+0000 7f8ae3200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8ae40af610 0x7f8ae40afa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:52.822 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.821+0000 7f8ae3200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ae4072c00 con 0x7f8ae40b4ae0 2024-05-09T04:33:52.822 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.821+0000 7f8ae3200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f8ae40aea70 con 0x7f8ae40b4700 2024-05-09T04:33:52.822 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.821+0000 7f8ae3200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8ae40737f0 con 0x7f8ae40af610 2024-05-09T04:33:52.822 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.821+0000 7f8ae1e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f8ae40b4700 0x7f8ae4073c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:52.822 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.821+0000 7f8ae1e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f8ae40b4700 0x7f8ae4073c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:44872/0 (socket says 172.21.15.145:44872) 2024-05-09T04:33:52.822 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.821+0000 7f8ae1e00640 1 -- 172.21.15.145:0/1076478440 learned_addr learned my addr 172.21.15.145:0/1076478440 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:52.822 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.822+0000 7f8ae1e00640 1 -- 172.21.15.145:0/1076478440 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8ae40af610 msgr2=0x7f8ae40afa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:52.822 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.822+0000 7f8adbe00640 1 --2- 172.21.15.145:0/1076478440 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8ae40af610 0x7f8ae40afa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:52.822 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.822+0000 7f8ae1e00640 1 --2- 172.21.15.145:0/1076478440 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8ae40af610 0x7f8ae40afa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:52.822 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.822+0000 7f8ae1e00640 1 -- 172.21.15.145:0/1076478440 >> v1:172.21.15.192:6789/0 conn(0x7f8ae40b4ae0 legacy=0x7f8ae40b4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:52.822 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.822+0000 7f8ae1e00640 1 -- 172.21.15.145:0/1076478440 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ae4072ec0 con 0x7f8ae40b4700 2024-05-09T04:33:52.822 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.822+0000 7f8adbe00640 1 --2- 172.21.15.145:0/1076478440 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8ae40af610 0x7f8ae40afa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:33:52.823 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.823+0000 7f8ae1e00640 1 --2- 172.21.15.145:0/1076478440 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ae40b4700 0x7f8ae4073c80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4009870 tx=0x7f8ad402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2d05d7ed015c852 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:52.824 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.823+0000 7f8ae1400640 1 -- 172.21.15.145:0/1076478440 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad4037070 con 0x7f8ae40b4700 2024-05-09T04:33:52.824 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.823+0000 7f8ae1400640 1 -- 172.21.15.145:0/1076478440 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ad4032440 con 0x7f8ae40b4700 2024-05-09T04:33:52.824 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.823+0000 7f8ae1400640 1 -- 172.21.15.145:0/1076478440 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad4032810 con 0x7f8ae40b4700 2024-05-09T04:33:52.824 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.824+0000 7f8ae3200640 1 -- 172.21.15.145:0/1076478440 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ae40b4700 msgr2=0x7f8ae4073c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:52.824 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.824+0000 7f8ae3200640 1 --2- 172.21.15.145:0/1076478440 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ae40b4700 0x7f8ae4073c80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4009870 tx=0x7f8ad402a1c0 comp rx=0 tx=0).stop 2024-05-09T04:33:52.824 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.824+0000 7f8ae3200640 1 -- 172.21.15.145:0/1076478440 shutdown_connections 2024-05-09T04:33:52.824 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.824+0000 7f8ae3200640 1 --2- 172.21.15.145:0/1076478440 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8ae40af610 0x7f8ae40afa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:52.824 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.824+0000 7f8ae3200640 1 --2- 172.21.15.145:0/1076478440 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ae40b4700 0x7f8ae4073c80 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:52.824 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.824+0000 7f8ae3200640 1 -- 172.21.15.145:0/1076478440 >> 172.21.15.145:0/1076478440 conn(0x7f8ae406c8a0 msgr2=0x7f8ae406ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:52.824 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.824+0000 7f8ae3200640 1 -- 172.21.15.145:0/1076478440 shutdown_connections 2024-05-09T04:33:52.824 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.824+0000 7f8ae3200640 1 -- 172.21.15.145:0/1076478440 wait complete. 2024-05-09T04:33:52.825 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.825+0000 7f8ae3200640 1 Processor -- start 2024-05-09T04:33:52.825 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.826+0000 7f8ae3200640 1 -- start start 2024-05-09T04:33:52.828 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.826+0000 7f8ae3200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8ae40af610 0x7f8ae4159350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:52.828 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.826+0000 7f8ae3200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ae40b4700 0x7f8ae41f8f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:52.828 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.827+0000 7f8ae3200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ae40b4ae0 0x7f8ae415d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:52.828 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.827+0000 7f8ae3200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae40741c0 con 0x7f8ae40b4ae0 2024-05-09T04:33:52.828 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.827+0000 7f8ae3200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae4073480 con 0x7f8ae40b4700 2024-05-09T04:33:52.828 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.827+0000 7f8ae3200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8ae406e320 con 0x7f8ae40af610 2024-05-09T04:33:52.828 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.827+0000 7f8ae2800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ae40b4ae0 0x7f8ae415d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:52.828 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.827+0000 7f8ae2800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ae40b4ae0 0x7f8ae415d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:44884/0 (socket says 172.21.15.145:44884) 2024-05-09T04:33:52.828 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.827+0000 7f8ae2800640 1 -- 172.21.15.145:0/1437391457 learned_addr learned my addr 172.21.15.145:0/1437391457 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:52.828 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.827+0000 7f8adbe00640 1 --2- 172.21.15.145:0/1437391457 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ae40b4700 0x7f8ae41f8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:52.828 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.827+0000 7f8ae1e00640 1 --2- 172.21.15.145:0/1437391457 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8ae40af610 0x7f8ae4159350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:52.828 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.827+0000 7f8ae2800640 1 -- 172.21.15.145:0/1437391457 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8ae40af610 msgr2=0x7f8ae4159350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:52.828 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.827+0000 7f8ae2800640 1 --2- 172.21.15.145:0/1437391457 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8ae40af610 0x7f8ae4159350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:52.828 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.827+0000 7f8ae2800640 1 -- 172.21.15.145:0/1437391457 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ae40b4700 msgr2=0x7f8ae41f8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:52.828 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.827+0000 7f8ae2800640 1 --2- 172.21.15.145:0/1437391457 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ae40b4700 0x7f8ae41f8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:52.829 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.827+0000 7f8ae2800640 1 -- 172.21.15.145:0/1437391457 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ae4159a50 con 0x7f8ae40b4ae0 2024-05-09T04:33:52.829 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.827+0000 7f8adbe00640 1 --2- 172.21.15.145:0/1437391457 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ae40b4700 0x7f8ae41f8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:33:52.829 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.827+0000 7f8ae2800640 1 --2- 172.21.15.145:0/1437391457 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ae40b4ae0 0x7f8ae415d820 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8adc00b970 tx=0x7f8adc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:52.829 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.828+0000 7f8ada000640 1 -- 172.21.15.145:0/1437391457 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8adc019070 con 0x7f8ae40b4ae0 2024-05-09T04:33:52.829 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.828+0000 7f8ada000640 1 -- 172.21.15.145:0/1437391457 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8adc00cb40 con 0x7f8ae40b4ae0 2024-05-09T04:33:52.829 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.828+0000 7f8ada000640 1 -- 172.21.15.145:0/1437391457 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8adc014440 con 0x7f8ae40b4ae0 2024-05-09T04:33:52.829 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.828+0000 7f8ae3200640 1 -- 172.21.15.145:0/1437391457 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ae4159d30 con 0x7f8ae40b4ae0 2024-05-09T04:33:52.829 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.828+0000 7f8ae3200640 1 -- 172.21.15.145:0/1437391457 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ae415a2f0 con 0x7f8ae40b4ae0 2024-05-09T04:33:52.829 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.829+0000 7f8ae3200640 1 -- 172.21.15.145:0/1437391457 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ae406a310 con 0x7f8ae40b4ae0 2024-05-09T04:33:52.830 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.830+0000 7f8ada000640 1 -- 172.21.15.145:0/1437391457 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8adc01c070 con 0x7f8ae40b4ae0 2024-05-09T04:33:52.831 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.831+0000 7f8ada000640 1 --2- 172.21.15.145:0/1437391457 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f8ac00762d0 0x7f8ac0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:52.831 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.831+0000 7f8ada000640 1 -- 172.21.15.145:0/1437391457 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f8adc097290 con 0x7f8ae40b4ae0 2024-05-09T04:33:52.831 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.831+0000 7f8ae1e00640 1 --2- 172.21.15.145:0/1437391457 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f8ac00762d0 0x7f8ac0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:52.831 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.832+0000 7f8ae1e00640 1 --2- 172.21.15.145:0/1437391457 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f8ac00762d0 0x7f8ac0078790 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4034000 tx=0x7f8ad402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:52.835 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:52.836+0000 7f8ada000640 1 -- 172.21.15.145:0/1437391457 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8adc060b50 con 0x7f8ae40b4ae0 2024-05-09T04:33:53.026 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.026+0000 7f8ae3200640 1 -- 172.21.15.145:0/1437391457 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8ae40b19d0 con 0x7f8ae40b4ae0 2024-05-09T04:33:53.029 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.029+0000 7f8ada000640 1 -- 172.21.15.145:0/1437391457 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16666 (secure 0 0 0) 0x7f8adc060450 con 0x7f8ae40b4ae0 2024-05-09T04:33:53.029 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:33:53.030 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":19,"fsid":"957ee6c1-073b-44cf-9b93-9491b3fdfc1d","created":"2024-05-09T04:33:26.544367+0000","modified":"2024-05-09T04:33:51.628969+0000","last_up_change":"2024-05-09T04:33:39.039318+0000","last_in_change":"2024-05-09T04:33:29.902361+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T04:33:39.795344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T04:33:49.651743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T04:33:50.596065+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22e550ad-847b-4952-80e6-63d859ed1a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6801","nonce":3182572912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6803","nonce":3182572912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6807","nonce":3182572912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6805","nonce":3182572912}]},"public_addr":"172.21.15.145:6801/3182572912","cluster_addr":"172.21.15.145:6803/3182572912","heartbeat_back_addr":"172.21.15.145:6807/3182572912","heartbeat_front_addr":"172.21.15.145:6805/3182572912","state":["exists","up"]},{"osd":1,"uuid":"0f6b9fca-0c41-4145-8d74-598ddb42f2a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6825","nonce":1845225555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6827","nonce":1845225555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6831","nonce":1845225555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6829","nonce":1845225555}]},"public_addr":"172.21.15.145:6825/1845225555","cluster_addr":"172.21.15.145:6827/1845225555","heartbeat_back_addr":"172.21.15.145:6831/1845225555","heartbeat_front_addr":"172.21.15.145:6829/1845225555","state":["exists","up"]},{"osd":2,"uuid":"b3b1f67e-5790-483f-8610-e05a5bce4c30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6817","nonce":2168958993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6819","nonce":2168958993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6823","nonce":2168958993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6821","nonce":2168958993}]},"public_addr":"172.21.15.145:6817/2168958993","cluster_addr":"172.21.15.145:6819/2168958993","heartbeat_back_addr":"172.21.15.145:6823/2168958993","heartbeat_front_addr":"172.21.15.145:6821/2168958993","state":["exists","up"]},{"osd":3,"uuid":"884787a5-e037-4d31-9aae-743030eba4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6809","nonce":337009826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6811","nonce":337009826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6815","nonce":337009826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6813","nonce":337009826}]},"public_addr":"172.21.15.145:6809/337009826","cluster_addr":"172.21.15.145:6811/337009826","heartbeat_back_addr":"172.21.15.145:6815/337009826","heartbeat_front_addr":"172.21.15.145:6813/337009826","state":["exists","up"]},{"osd":4,"uuid":"0759fc1e-c5fc-4a2f-86bb-e8eccd83f833","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6818","nonce":3117700995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6821","nonce":3117700995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6827","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6829","nonce":3117700995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6823","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6825","nonce":3117700995}]},"public_addr":"172.21.15.192:6818/3117700995","cluster_addr":"172.21.15.192:6821/3117700995","heartbeat_back_addr":"172.21.15.192:6829/3117700995","heartbeat_front_addr":"172.21.15.192:6825/3117700995","state":["exists","up"]},{"osd":5,"uuid":"9443cc22-9379-46e3-9173-98b592bf2c16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6801","nonce":1328373268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6803","nonce":1328373268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6807","nonce":1328373268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6805","nonce":1328373268}]},"public_addr":"172.21.15.192:6801/1328373268","cluster_addr":"172.21.15.192:6803/1328373268","heartbeat_back_addr":"172.21.15.192:6807/1328373268","heartbeat_front_addr":"172.21.15.192:6805/1328373268","state":["exists","up"]},{"osd":6,"uuid":"d05a3311-505a-45b5-85b9-aa3113e96987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6817","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6820","nonce":1868011057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6824","nonce":1868011057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6831","nonce":1868011057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6828","nonce":1868011057}]},"public_addr":"172.21.15.192:6820/1868011057","cluster_addr":"172.21.15.192:6824/1868011057","heartbeat_back_addr":"172.21.15.192:6831/1868011057","heartbeat_front_addr":"172.21.15.192:6828/1868011057","state":["exists","up"]},{"osd":7,"uuid":"c7c23dbc-2029-43ec-9f53-25f3d58019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6809","nonce":3368512157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6811","nonce":3368512157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6815","nonce":3368512157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6813","nonce":3368512157}]},"public_addr":"172.21.15.192:6809/3368512157","cluster_addr":"172.21.15.192:6811/3368512157","heartbeat_back_addr":"172.21.15.192:6815/3368512157","heartbeat_front_addr":"172.21.15.192:6813/3368512157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.091188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.032567+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:34.797468+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.228839+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.622400+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.473103+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.566092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.349395+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T04:33:53.032 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.032+0000 7f8ae3200640 1 -- 172.21.15.145:0/1437391457 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f8ac00762d0 msgr2=0x7f8ac0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:53.032 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.032+0000 7f8ae3200640 1 --2- 172.21.15.145:0/1437391457 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f8ac00762d0 0x7f8ac0078790 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4034000 tx=0x7f8ad402e000 comp rx=0 tx=0).stop 2024-05-09T04:33:53.032 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.032+0000 7f8ae3200640 1 -- 172.21.15.145:0/1437391457 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ae40b4ae0 msgr2=0x7f8ae415d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:53.032 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.032+0000 7f8ae3200640 1 --2- 172.21.15.145:0/1437391457 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ae40b4ae0 0x7f8ae415d820 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8adc00b970 tx=0x7f8adc00be30 comp rx=0 tx=0).stop 2024-05-09T04:33:53.032 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.032+0000 7f8ae3200640 1 -- 172.21.15.145:0/1437391457 shutdown_connections 2024-05-09T04:33:53.032 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.032+0000 7f8ae3200640 1 --2- 172.21.15.145:0/1437391457 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f8ac00762d0 0x7f8ac0078790 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:53.032 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.032+0000 7f8ae3200640 1 --2- 172.21.15.145:0/1437391457 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ae40b4ae0 0x7f8ae415d820 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:53.032 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.032+0000 7f8ae3200640 1 --2- 172.21.15.145:0/1437391457 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ae40b4700 0x7f8ae41f8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:53.032 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.032+0000 7f8ae3200640 1 --2- 172.21.15.145:0/1437391457 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8ae40af610 0x7f8ae4159350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:53.033 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.032+0000 7f8ae3200640 1 -- 172.21.15.145:0/1437391457 >> 172.21.15.145:0/1437391457 conn(0x7f8ae406c8a0 msgr2=0x7f8ae40af9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:53.033 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.032+0000 7f8ae3200640 1 -- 172.21.15.145:0/1437391457 shutdown_connections 2024-05-09T04:33:53.033 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.032+0000 7f8ae3200640 1 -- 172.21.15.145:0/1437391457 wait complete. 2024-05-09T04:33:53.049 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-09T04:33:53.246 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.246+0000 7fae9c400640 1 Processor -- start 2024-05-09T04:33:53.246 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.246+0000 7fae9c400640 1 -- start start 2024-05-09T04:33:53.247 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.247+0000 7fae9c400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fae94143450 0x7fae94142820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:53.247 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.247+0000 7fae9c400640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae9406e560 con 0x7fae941400b0 2024-05-09T04:33:53.247 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.247+0000 7fae9c400640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae94140490 con 0x7fae94142d60 2024-05-09T04:33:53.247 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.247+0000 7fae9c400640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fae940d0290 con 0x7fae94143450 2024-05-09T04:33:53.247 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.248+0000 7fae99a00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7fae941400b0 0x7fae940cf7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:33864/0 (socket says 172.21.15.145:33864) 2024-05-09T04:33:53.247 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.248+0000 7fae99a00640 1 -- 172.21.15.145:0/4073148773 learned_addr learned my addr 172.21.15.145:0/4073148773 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:33:53.249 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.248+0000 7fae93e00640 1 -- 172.21.15.145:0/4073148773 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2379887931 0 0) 0x7fae9406e560 con 0x7fae941400b0 2024-05-09T04:33:53.249 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.248+0000 7fae93e00640 1 -- 172.21.15.145:0/4073148773 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fae7c003520 con 0x7fae941400b0 2024-05-09T04:33:53.249 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.248+0000 7fae99000640 1 --2- 172.21.15.145:0/4073148773 >> v2:172.21.15.192:3300/0 conn(0x7fae94143450 0x7fae94142820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:53.249 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.248+0000 7fae93e00640 1 -- 172.21.15.145:0/4073148773 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3830365147 0 0) 0x7fae7c003520 con 0x7fae941400b0 2024-05-09T04:33:53.249 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.248+0000 7fae93e00640 1 -- 172.21.15.145:0/4073148773 >> v2:172.21.15.192:3300/0 conn(0x7fae94143450 msgr2=0x7fae94142820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:53.250 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.248+0000 7fae93e00640 1 --2- 172.21.15.145:0/4073148773 >> v2:172.21.15.192:3300/0 conn(0x7fae94143450 0x7fae94142820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:53.250 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.248+0000 7fae93e00640 1 -- 172.21.15.145:0/4073148773 >> v1:172.21.15.192:6789/0 conn(0x7fae94142d60 legacy=0x7fae9406ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:53.250 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.248+0000 7fae93e00640 1 -- 172.21.15.145:0/4073148773 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae940d0570 con 0x7fae941400b0 2024-05-09T04:33:53.250 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.248+0000 7fae93e00640 1 -- 172.21.15.145:0/4073148773 <== mon.0 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3474413261 0 0) 0x7fae88001a70 con 0x7fae941400b0 2024-05-09T04:33:53.250 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.248+0000 7fae93e00640 1 -- 172.21.15.145:0/4073148773 >> v1:172.21.15.145:6789/0 conn(0x7fae941400b0 legacy=0x7fae940cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:53.250 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.248+0000 7fae93e00640 1 --2- 172.21.15.145:0/4073148773 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae7c003a50 0x7fae7c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:53.250 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.248+0000 7fae93e00640 1 --2- 172.21.15.145:0/4073148773 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fae7c004800 0x7fae7c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:53.250 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.249+0000 7fae93e00640 1 --2- 172.21.15.145:0/4073148773 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae7c0053c0 0x7fae7c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:53.250 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.249+0000 7fae93e00640 1 -- 172.21.15.145:0/4073148773 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fae940d0290 con 0x7fae7c003a50 2024-05-09T04:33:53.250 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.249+0000 7fae93e00640 1 -- 172.21.15.145:0/4073148773 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fae7c00c250 con 0x7fae7c0053c0 2024-05-09T04:33:53.250 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.249+0000 7fae93e00640 1 -- 172.21.15.145:0/4073148773 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fae7c00c450 con 0x7fae7c004800 2024-05-09T04:33:53.250 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.249+0000 7fae99a00640 1 --2- 172.21.15.145:0/4073148773 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae7c003a50 0x7fae7c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:53.250 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.249+0000 7fae99a00640 1 -- 172.21.15.145:0/4073148773 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fae7c004800 msgr2=0x7fae7c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:53.250 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.249+0000 7fae99a00640 1 --2- 172.21.15.145:0/4073148773 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fae7c004800 0x7fae7c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:53.251 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.249+0000 7fae99a00640 1 -- 172.21.15.145:0/4073148773 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae7c0053c0 msgr2=0x7fae7c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:53.251 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.249+0000 7fae99a00640 1 --2- 172.21.15.145:0/4073148773 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae7c0053c0 0x7fae7c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:53.251 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.249+0000 7fae9a400640 1 -- 172.21.15.145:0/4073148773 reap_dead start 2024-05-09T04:33:53.251 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.249+0000 7fae99a00640 1 -- 172.21.15.145:0/4073148773 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae7c003520 con 0x7fae7c003a50 2024-05-09T04:33:53.252 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.250+0000 7fae99a00640 1 --2- 172.21.15.145:0/4073148773 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae7c003a50 0x7fae7c003e70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fae8800a920 tx=0x7fae7c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2a65d6425e4e587 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:53.252 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.250+0000 7fae93e00640 1 -- 172.21.15.145:0/4073148773 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8800a4c0 con 0x7fae7c003a50 2024-05-09T04:33:53.252 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.250+0000 7fae93e00640 1 -- 172.21.15.145:0/4073148773 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae8802f070 con 0x7fae7c003a50 2024-05-09T04:33:53.252 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.250+0000 7fae93e00640 1 -- 172.21.15.145:0/4073148773 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8802bc50 con 0x7fae7c003a50 2024-05-09T04:33:53.252 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.251+0000 7fae9c400640 1 -- 172.21.15.145:0/4073148773 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae7c003a50 msgr2=0x7fae7c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:53.252 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.251+0000 7fae9c400640 1 --2- 172.21.15.145:0/4073148773 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae7c003a50 0x7fae7c003e70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fae8800a920 tx=0x7fae7c005830 comp rx=0 tx=0).stop 2024-05-09T04:33:53.252 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.251+0000 7fae9c400640 1 -- 172.21.15.145:0/4073148773 shutdown_connections 2024-05-09T04:33:53.252 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.251+0000 7fae9c400640 1 --2- 172.21.15.145:0/4073148773 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae7c003a50 0x7fae7c003e70 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:53.252 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.251+0000 7fae9c400640 1 -- 172.21.15.145:0/4073148773 >> 172.21.15.145:0/4073148773 conn(0x7fae9406cab0 msgr2=0x7fae940d0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:53.252 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.251+0000 7fae9c400640 1 -- 172.21.15.145:0/4073148773 shutdown_connections 2024-05-09T04:33:53.252 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.251+0000 7fae9c400640 1 -- 172.21.15.145:0/4073148773 wait complete. 2024-05-09T04:33:53.253 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.253+0000 7fae9c400640 1 Processor -- start 2024-05-09T04:33:53.253 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.253+0000 7fae9c400640 1 -- start start 2024-05-09T04:33:53.255 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.254+0000 7fae9c400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fae7c003a50 0x7fae9413c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:53.255 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.254+0000 7fae9c400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae940afc10 0x7fae940b4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:53.255 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.254+0000 7fae9c400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae9413e6f0 0x7fae9413eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:53.255 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.254+0000 7fae9c400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fae94142820 con 0x7fae940afc10 2024-05-09T04:33:53.256 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.254+0000 7fae9c400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fae940d0b30 con 0x7fae9413e6f0 2024-05-09T04:33:53.256 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.254+0000 7fae9c400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fae941429a0 con 0x7fae7c003a50 2024-05-09T04:33:53.256 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.254+0000 7fae99000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae940afc10 0x7fae940b4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:53.256 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.254+0000 7fae99000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae940afc10 0x7fae940b4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:44896/0 (socket says 172.21.15.145:44896) 2024-05-09T04:33:53.256 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.254+0000 7fae99000640 1 -- 172.21.15.145:0/3501699926 learned_addr learned my addr 172.21.15.145:0/3501699926 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:53.256 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.254+0000 7fae99a00640 1 --2- 172.21.15.145:0/3501699926 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fae7c003a50 0x7fae9413c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:53.256 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.255+0000 7fae9a400640 1 --2- 172.21.15.145:0/3501699926 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae9413e6f0 0x7fae9413eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:53.256 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.255+0000 7fae99000640 1 -- 172.21.15.145:0/3501699926 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fae7c003a50 msgr2=0x7fae9413c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:53.256 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.255+0000 7fae99000640 1 --2- 172.21.15.145:0/3501699926 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fae7c003a50 0x7fae9413c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:53.256 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.255+0000 7fae99000640 1 -- 172.21.15.145:0/3501699926 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae9413e6f0 msgr2=0x7fae9413eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:53.256 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.255+0000 7fae99000640 1 --2- 172.21.15.145:0/3501699926 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae9413e6f0 0x7fae9413eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:53.256 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.255+0000 7fae99000640 1 -- 172.21.15.145:0/3501699926 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae940b4810 con 0x7fae940afc10 2024-05-09T04:33:53.256 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.255+0000 7fae99a00640 1 --2- 172.21.15.145:0/3501699926 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fae7c003a50 0x7fae9413c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:33:53.256 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.255+0000 7fae99000640 1 --2- 172.21.15.145:0/3501699926 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae940afc10 0x7fae940b4090 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fae8000d5b0 tx=0x7fae8000da70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:53.256 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.255+0000 7fae91600640 1 -- 172.21.15.145:0/3501699926 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae800125f0 con 0x7fae940afc10 2024-05-09T04:33:53.257 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.255+0000 7fae91600640 1 -- 172.21.15.145:0/3501699926 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae80019070 con 0x7fae940afc10 2024-05-09T04:33:53.257 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.255+0000 7fae91600640 1 -- 172.21.15.145:0/3501699926 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae800129d0 con 0x7fae940afc10 2024-05-09T04:33:53.257 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.255+0000 7fae9c400640 1 -- 172.21.15.145:0/3501699926 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae940b4af0 con 0x7fae940afc10 2024-05-09T04:33:53.257 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.255+0000 7fae9c400640 1 -- 172.21.15.145:0/3501699926 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae940b50b0 con 0x7fae940afc10 2024-05-09T04:33:53.257 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.257+0000 7fae9c400640 1 -- 172.21.15.145:0/3501699926 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae9406a310 con 0x7fae940afc10 2024-05-09T04:33:53.257 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.258+0000 7fae91600640 1 -- 172.21.15.145:0/3501699926 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fae80012b70 con 0x7fae940afc10 2024-05-09T04:33:53.258 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.258+0000 7fae91600640 1 --2- 172.21.15.145:0/3501699926 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fae74076300 0x7fae740787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:53.258 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.259+0000 7fae91600640 1 -- 172.21.15.145:0/3501699926 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fae80097870 con 0x7fae940afc10 2024-05-09T04:33:53.258 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.259+0000 7fae99a00640 1 --2- 172.21.15.145:0/3501699926 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fae74076300 0x7fae740787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:53.259 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.259+0000 7fae99a00640 1 --2- 172.21.15.145:0/3501699926 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fae74076300 0x7fae740787c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fae88033000 tx=0x7fae88038040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:53.262 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.263+0000 7fae91600640 1 -- 172.21.15.145:0/3501699926 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fae800611c0 con 0x7fae940afc10 2024-05-09T04:33:53.508 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:53.508+0000 7fae9c400640 1 -- 172.21.15.145:0/3501699926 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fae940d1980 con 0x7fae940afc10 2024-05-09T04:33:54.198 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.198+0000 7fae91600640 1 -- 172.21.15.145:0/3501699926 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7fae80060ac0 con 0x7fae940afc10 2024-05-09T04:33:54.203 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.203+0000 7fae9c400640 1 -- 172.21.15.145:0/3501699926 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fae74076300 msgr2=0x7fae740787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:54.203 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.203+0000 7fae9c400640 1 --2- 172.21.15.145:0/3501699926 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fae74076300 0x7fae740787c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fae88033000 tx=0x7fae88038040 comp rx=0 tx=0).stop 2024-05-09T04:33:54.203 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.203+0000 7fae9c400640 1 -- 172.21.15.145:0/3501699926 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae940afc10 msgr2=0x7fae940b4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:54.203 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.203+0000 7fae9c400640 1 --2- 172.21.15.145:0/3501699926 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae940afc10 0x7fae940b4090 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fae8000d5b0 tx=0x7fae8000da70 comp rx=0 tx=0).stop 2024-05-09T04:33:54.203 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.203+0000 7fae9c400640 1 -- 172.21.15.145:0/3501699926 shutdown_connections 2024-05-09T04:33:54.203 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.203+0000 7fae9c400640 1 --2- 172.21.15.145:0/3501699926 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fae74076300 0x7fae740787c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:54.203 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.203+0000 7fae9c400640 1 --2- 172.21.15.145:0/3501699926 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae9413e6f0 0x7fae9413eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:54.203 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.203+0000 7fae9c400640 1 --2- 172.21.15.145:0/3501699926 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fae940afc10 0x7fae940b4090 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:54.203 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.203+0000 7fae9c400640 1 --2- 172.21.15.145:0/3501699926 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fae7c003a50 0x7fae9413c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:54.203 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.203+0000 7fae9c400640 1 -- 172.21.15.145:0/3501699926 >> 172.21.15.145:0/3501699926 conn(0x7fae9406cab0 msgr2=0x7fae94161450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:54.204 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.203+0000 7fae9c400640 1 -- 172.21.15.145:0/3501699926 shutdown_connections 2024-05-09T04:33:54.204 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.203+0000 7fae9c400640 1 -- 172.21.15.145:0/3501699926 wait complete. 2024-05-09T04:33:54.229 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-09T04:33:54.229 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-09T04:33:54.436 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.436+0000 7f06d6200640 1 Processor -- start 2024-05-09T04:33:54.437 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.436+0000 7f06d6200640 1 -- start start 2024-05-09T04:33:54.439 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.438+0000 7f06d6200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f06d014d2c0 0x7f06d00b4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:54.439 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.438+0000 7f06d6200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f06d0072be0 0x7f06d006e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:54.439 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.438+0000 7f06d6200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06d00b4fe0 con 0x7f06d00b4a60 2024-05-09T04:33:54.440 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.438+0000 7f06d6200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f06d006e4c0 con 0x7f06d0072be0 2024-05-09T04:33:54.440 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.438+0000 7f06d6200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f06d00b52c0 con 0x7f06d014d2c0 2024-05-09T04:33:54.440 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.438+0000 7f06cfe00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f06d0072be0 0x7f06d006e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:54.440 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.439+0000 7f06cfe00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f06d0072be0 0x7f06d006e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:44910/0 (socket says 172.21.15.145:44910) 2024-05-09T04:33:54.440 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.439+0000 7f06cfe00640 1 -- 172.21.15.145:0/2122001540 learned_addr learned my addr 172.21.15.145:0/2122001540 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:54.440 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.439+0000 7f06cf400640 1 --2- 172.21.15.145:0/2122001540 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f06d014d2c0 0x7f06d00b4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:54.440 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.439+0000 7f06cfe00640 1 -- 172.21.15.145:0/2122001540 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f06d014d2c0 msgr2=0x7f06d00b4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:54.440 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.439+0000 7f06cfe00640 1 --2- 172.21.15.145:0/2122001540 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f06d014d2c0 0x7f06d00b4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:54.441 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.439+0000 7f06cfe00640 1 -- 172.21.15.145:0/2122001540 >> v1:172.21.15.192:6789/0 conn(0x7f06d00b4a60 legacy=0x7f06d00724d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:54.441 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.439+0000 7f06cfe00640 1 -- 172.21.15.145:0/2122001540 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06d00b55a0 con 0x7f06d0072be0 2024-05-09T04:33:54.443 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.440+0000 7f06cfe00640 1 --2- 172.21.15.145:0/2122001540 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f06d0072be0 0x7f06d006e950 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f06c8009e50 tx=0x7f06c802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a6c76f8a43e38ec server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:54.443 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.441+0000 7f06ce000640 1 -- 172.21.15.145:0/2122001540 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c8036070 con 0x7f06d0072be0 2024-05-09T04:33:54.443 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.441+0000 7f06ce000640 1 -- 172.21.15.145:0/2122001540 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06c8031600 con 0x7f06d0072be0 2024-05-09T04:33:54.443 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.441+0000 7f06ce000640 1 -- 172.21.15.145:0/2122001540 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c80319d0 con 0x7f06d0072be0 2024-05-09T04:33:54.443 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.441+0000 7f06d6200640 1 -- 172.21.15.145:0/2122001540 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f06d0072be0 msgr2=0x7f06d006e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:54.444 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.441+0000 7f06d6200640 1 --2- 172.21.15.145:0/2122001540 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f06d0072be0 0x7f06d006e950 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f06c8009e50 tx=0x7f06c802a680 comp rx=0 tx=0).stop 2024-05-09T04:33:54.444 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.441+0000 7f06d6200640 1 -- 172.21.15.145:0/2122001540 shutdown_connections 2024-05-09T04:33:54.444 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.441+0000 7f06d6200640 1 --2- 172.21.15.145:0/2122001540 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f06d0072be0 0x7f06d006e950 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:54.444 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.441+0000 7f06d6200640 1 --2- 172.21.15.145:0/2122001540 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f06d014d2c0 0x7f06d00b4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:54.444 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.441+0000 7f06d6200640 1 -- 172.21.15.145:0/2122001540 >> 172.21.15.145:0/2122001540 conn(0x7f06d006cab0 msgr2=0x7f06d00d0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:54.444 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.441+0000 7f06d6200640 1 -- 172.21.15.145:0/2122001540 shutdown_connections 2024-05-09T04:33:54.444 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.441+0000 7f06d6200640 1 -- 172.21.15.145:0/2122001540 wait complete. 2024-05-09T04:33:54.444 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.443+0000 7f06d6200640 1 Processor -- start 2024-05-09T04:33:54.445 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.443+0000 7f06d6200640 1 -- start start 2024-05-09T04:33:54.448 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.444+0000 7f06d6200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f06d0072be0 0x7f06d00cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:54.448 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.444+0000 7f06d6200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f06d00b4a60 0x7f06d0162f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:54.448 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.444+0000 7f06d6200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f06d014d2c0 0x7f06d0163440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:54.448 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.444+0000 7f06d6200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f06d006ee90 con 0x7f06d00b4a60 2024-05-09T04:33:54.448 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.444+0000 7f06d6200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f06d0167590 con 0x7f06d014d2c0 2024-05-09T04:33:54.448 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.444+0000 7f06d6200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f06d006f010 con 0x7f06d0072be0 2024-05-09T04:33:54.448 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.444+0000 7f06cea00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f06d00b4a60 0x7f06d0162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:54.448 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.444+0000 7f06cea00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f06d00b4a60 0x7f06d0162f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:44916/0 (socket says 172.21.15.145:44916) 2024-05-09T04:33:54.448 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.444+0000 7f06cea00640 1 -- 172.21.15.145:0/1758458500 learned_addr learned my addr 172.21.15.145:0/1758458500 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:54.449 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.445+0000 7f06cf400640 1 --2- 172.21.15.145:0/1758458500 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f06d0072be0 0x7f06d00cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:54.449 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.445+0000 7f06cfe00640 1 --2- 172.21.15.145:0/1758458500 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f06d014d2c0 0x7f06d0163440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:54.449 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.445+0000 7f06cea00640 1 -- 172.21.15.145:0/1758458500 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f06d0072be0 msgr2=0x7f06d00cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:54.449 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.445+0000 7f06cea00640 1 --2- 172.21.15.145:0/1758458500 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f06d0072be0 0x7f06d00cfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:54.449 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.445+0000 7f06cea00640 1 -- 172.21.15.145:0/1758458500 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f06d014d2c0 msgr2=0x7f06d0163440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:54.449 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.445+0000 7f06cea00640 1 --2- 172.21.15.145:0/1758458500 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f06d014d2c0 0x7f06d0163440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:54.449 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.445+0000 7f06cea00640 1 -- 172.21.15.145:0/1758458500 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06d00ccfd0 con 0x7f06d00b4a60 2024-05-09T04:33:54.449 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.445+0000 7f06cea00640 1 --2- 172.21.15.145:0/1758458500 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f06d00b4a60 0x7f06d0162f00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f06b800c450 tx=0x7f06b800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:54.449 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.445+0000 7f06c7400640 1 -- 172.21.15.145:0/1758458500 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b80052d0 con 0x7f06d00b4a60 2024-05-09T04:33:54.449 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.446+0000 7f06c7400640 1 -- 172.21.15.145:0/1758458500 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06b8005470 con 0x7f06d00b4a60 2024-05-09T04:33:54.450 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.446+0000 7f06c7400640 1 -- 172.21.15.145:0/1758458500 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b8012870 con 0x7f06d00b4a60 2024-05-09T04:33:54.450 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.446+0000 7f06d6200640 1 -- 172.21.15.145:0/1758458500 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06d00c7c10 con 0x7f06d00b4a60 2024-05-09T04:33:54.450 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.446+0000 7f06d6200640 1 -- 172.21.15.145:0/1758458500 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06d01489e0 con 0x7f06d00b4a60 2024-05-09T04:33:54.450 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.447+0000 7f06d6200640 1 -- 172.21.15.145:0/1758458500 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06d006a310 con 0x7f06d00b4a60 2024-05-09T04:33:54.450 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.448+0000 7f06c7400640 1 -- 172.21.15.145:0/1758458500 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f06b8012a10 con 0x7f06d00b4a60 2024-05-09T04:33:54.451 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.449+0000 7f06c7400640 1 --2- 172.21.15.145:0/1758458500 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f06ac0762d0 0x7f06ac078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:54.451 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.449+0000 7f06c7400640 1 -- 172.21.15.145:0/1758458500 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f06b80972b0 con 0x7f06d00b4a60 2024-05-09T04:33:54.451 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.449+0000 7f06cf400640 1 --2- 172.21.15.145:0/1758458500 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f06ac0762d0 0x7f06ac078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:54.451 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.450+0000 7f06cf400640 1 --2- 172.21.15.145:0/1758458500 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f06ac0762d0 0x7f06ac078790 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f06c0006fd0 tx=0x7f06c0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:54.453 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.453+0000 7f06c7400640 1 -- 172.21.15.145:0/1758458500 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f06b8060b70 con 0x7f06d00b4a60 2024-05-09T04:33:54.698 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:54.698+0000 7f06d6200640 1 -- 172.21.15.145:0/1758458500 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f06d00b40d0 con 0x7f06d00b4a60 2024-05-09T04:33:55.202 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.202+0000 7f06c7400640 1 -- 172.21.15.145:0/1758458500 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7f06b8060470 con 0x7f06d00b4a60 2024-05-09T04:33:55.206 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.205+0000 7f06d6200640 1 -- 172.21.15.145:0/1758458500 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f06ac0762d0 msgr2=0x7f06ac078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:55.206 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.205+0000 7f06d6200640 1 --2- 172.21.15.145:0/1758458500 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f06ac0762d0 0x7f06ac078790 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f06c0006fd0 tx=0x7f06c0008040 comp rx=0 tx=0).stop 2024-05-09T04:33:55.206 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.206+0000 7f06d6200640 1 -- 172.21.15.145:0/1758458500 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f06d00b4a60 msgr2=0x7f06d0162f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:55.206 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.206+0000 7f06d6200640 1 --2- 172.21.15.145:0/1758458500 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f06d00b4a60 0x7f06d0162f00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f06b800c450 tx=0x7f06b800c910 comp rx=0 tx=0).stop 2024-05-09T04:33:55.206 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.206+0000 7f06d6200640 1 -- 172.21.15.145:0/1758458500 shutdown_connections 2024-05-09T04:33:55.206 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.206+0000 7f06d6200640 1 --2- 172.21.15.145:0/1758458500 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f06ac0762d0 0x7f06ac078790 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:55.206 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.206+0000 7f06d6200640 1 --2- 172.21.15.145:0/1758458500 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f06d014d2c0 0x7f06d0163440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:55.206 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.206+0000 7f06d6200640 1 --2- 172.21.15.145:0/1758458500 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f06d00b4a60 0x7f06d0162f00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:55.206 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.206+0000 7f06d6200640 1 --2- 172.21.15.145:0/1758458500 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f06d0072be0 0x7f06d00cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:55.206 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.206+0000 7f06d6200640 1 -- 172.21.15.145:0/1758458500 >> 172.21.15.145:0/1758458500 conn(0x7f06d006cab0 msgr2=0x7f06d006f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:55.206 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.206+0000 7f06d6200640 1 -- 172.21.15.145:0/1758458500 shutdown_connections 2024-05-09T04:33:55.207 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.206+0000 7f06d6200640 1 -- 172.21.15.145:0/1758458500 wait complete. 2024-05-09T04:33:55.227 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-09T04:33:55.227 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T04:33:55.422 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.422+0000 7f1bc7400640 1 Processor -- start 2024-05-09T04:33:55.422 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.422+0000 7f1bc7400640 1 -- start start 2024-05-09T04:33:55.423 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.424+0000 7f1bc7400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f1bc00b5920 0x7f1bc0167db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:55.424 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.424+0000 7f1bc7400640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bc01615c0 con 0x7f1bc006e860 2024-05-09T04:33:55.424 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.424+0000 7f1bc7400640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bc01618a0 con 0x7f1bc015d780 2024-05-09T04:33:55.424 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.424+0000 7f1bc7400640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f1bc0161360 con 0x7f1bc00b5920 2024-05-09T04:33:55.424 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.424+0000 7f1bc4a00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f1bc006e860 0x7f1bc006ec40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:33876/0 (socket says 172.21.15.145:33876) 2024-05-09T04:33:55.424 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.424+0000 7f1bc4a00640 1 -- 172.21.15.145:0/3553787298 learned_addr learned my addr 172.21.15.145:0/3553787298 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:33:55.426 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.424+0000 7f1bbf400640 1 -- 172.21.15.145:0/3553787298 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1736755327 0 0) 0x7f1bc01615c0 con 0x7f1bc006e860 2024-05-09T04:33:55.426 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.424+0000 7f1bbf400640 1 -- 172.21.15.145:0/3553787298 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ba8003520 con 0x7f1bc006e860 2024-05-09T04:33:55.426 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.424+0000 7f1bc5400640 1 --2- 172.21.15.145:0/3553787298 >> v2:172.21.15.192:3300/0 conn(0x7f1bc00b5920 0x7f1bc0167db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:55.426 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.425+0000 7f1bbf400640 1 -- 172.21.15.145:0/3553787298 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3688225667 0 0) 0x7f1ba8003520 con 0x7f1bc006e860 2024-05-09T04:33:55.426 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.425+0000 7f1bbf400640 1 -- 172.21.15.145:0/3553787298 >> v2:172.21.15.192:3300/0 conn(0x7f1bc00b5920 msgr2=0x7f1bc0167db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:55.426 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.425+0000 7f1bbf400640 1 --2- 172.21.15.145:0/3553787298 >> v2:172.21.15.192:3300/0 conn(0x7f1bc00b5920 0x7f1bc0167db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:55.426 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.425+0000 7f1bbf400640 1 -- 172.21.15.145:0/3553787298 >> v1:172.21.15.192:6789/0 conn(0x7f1bc015d780 legacy=0x7f1bc0160c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:55.426 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.425+0000 7f1bbf400640 1 -- 172.21.15.145:0/3553787298 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bc00b4110 con 0x7f1bc006e860 2024-05-09T04:33:55.426 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.425+0000 7f1bbf400640 1 -- 172.21.15.145:0/3553787298 <== mon.0 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3474413261 0 0) 0x7f1bb40016f0 con 0x7f1bc006e860 2024-05-09T04:33:55.426 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.425+0000 7f1bbf400640 1 -- 172.21.15.145:0/3553787298 >> v1:172.21.15.145:6789/0 conn(0x7f1bc006e860 legacy=0x7f1bc006ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:55.426 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.425+0000 7f1bbf400640 1 --2- 172.21.15.145:0/3553787298 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ba8003a50 0x7f1ba8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:55.426 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.425+0000 7f1bbf400640 1 --2- 172.21.15.145:0/3553787298 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1ba8004800 0x7f1ba8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:55.426 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.425+0000 7f1bbf400640 1 --2- 172.21.15.145:0/3553787298 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ba80052e0 0x7f1ba800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:55.427 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.425+0000 7f1bbf400640 1 -- 172.21.15.145:0/3553787298 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc0161360 con 0x7f1ba8004800 2024-05-09T04:33:55.427 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.425+0000 7f1bbf400640 1 -- 172.21.15.145:0/3553787298 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba800c1b0 con 0x7f1ba80052e0 2024-05-09T04:33:55.427 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.425+0000 7f1bbf400640 1 -- 172.21.15.145:0/3553787298 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f1ba800c3b0 con 0x7f1ba8003a50 2024-05-09T04:33:55.427 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.425+0000 7f1bbfe00640 1 --2- 172.21.15.145:0/3553787298 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1ba8004800 0x7f1ba8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:55.427 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.425+0000 7f1bc4a00640 1 --2- 172.21.15.145:0/3553787298 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ba8003a50 0x7f1ba8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:55.427 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.426+0000 7f1bbfe00640 1 -- 172.21.15.145:0/3553787298 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ba8003a50 msgr2=0x7f1ba8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:55.427 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.426+0000 7f1bbfe00640 1 --2- 172.21.15.145:0/3553787298 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ba8003a50 0x7f1ba8003e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:55.427 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.426+0000 7f1bbfe00640 1 -- 172.21.15.145:0/3553787298 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ba80052e0 msgr2=0x7f1ba800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:55.427 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.426+0000 7f1bbfe00640 1 --2- 172.21.15.145:0/3553787298 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ba80052e0 0x7f1ba800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:55.427 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.426+0000 7f1bbfe00640 1 -- 172.21.15.145:0/3553787298 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ba8003520 con 0x7f1ba8004800 2024-05-09T04:33:55.427 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.426+0000 7f1bc5400640 1 -- 172.21.15.145:0/3553787298 reap_dead start 2024-05-09T04:33:55.429 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.427+0000 7f1bbfe00640 1 --2- 172.21.15.145:0/3553787298 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1ba8004800 0x7f1ba8004c80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1bac001760 tx=0x7f1ba8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7538e3e1ff1045d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:55.429 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.427+0000 7f1bbf400640 1 -- 172.21.15.145:0/3553787298 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bac0013b0 con 0x7f1ba8004800 2024-05-09T04:33:55.429 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.427+0000 7f1bbf400640 1 -- 172.21.15.145:0/3553787298 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bac0052d0 con 0x7f1ba8004800 2024-05-09T04:33:55.429 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.427+0000 7f1bbf400640 1 -- 172.21.15.145:0/3553787298 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bac02aaa0 con 0x7f1ba8004800 2024-05-09T04:33:55.429 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.427+0000 7f1bc7400640 1 -- 172.21.15.145:0/3553787298 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1ba8004800 msgr2=0x7f1ba8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:55.429 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.427+0000 7f1bc7400640 1 --2- 172.21.15.145:0/3553787298 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1ba8004800 0x7f1ba8004c80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1bac001760 tx=0x7f1ba8005830 comp rx=0 tx=0).stop 2024-05-09T04:33:55.429 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.427+0000 7f1bc7400640 1 -- 172.21.15.145:0/3553787298 shutdown_connections 2024-05-09T04:33:55.429 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.427+0000 7f1bc7400640 1 --2- 172.21.15.145:0/3553787298 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1ba8004800 0x7f1ba8004c80 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:55.429 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.427+0000 7f1bc7400640 1 -- 172.21.15.145:0/3553787298 >> 172.21.15.145:0/3553787298 conn(0x7f1bc006c870 msgr2=0x7f1bc006cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:55.429 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.428+0000 7f1bc7400640 1 -- 172.21.15.145:0/3553787298 shutdown_connections 2024-05-09T04:33:55.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.428+0000 7f1bc7400640 1 -- 172.21.15.145:0/3553787298 wait complete. 2024-05-09T04:33:55.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.429+0000 7f1bc7400640 1 Processor -- start 2024-05-09T04:33:55.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.429+0000 7f1bc7400640 1 -- start start 2024-05-09T04:33:55.431 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.430+0000 7f1bc7400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1ba8004800 0x7f1bc014f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:55.431 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.430+0000 7f1bc7400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1bc01398c0 0x7f1bc0072c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:55.431 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.430+0000 7f1bc7400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1bc0139d20 0x7f1bc0141b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:55.432 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.430+0000 7f1bc7400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc0146d80 con 0x7f1ba8004800 2024-05-09T04:33:55.432 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.430+0000 7f1bc7400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1bc0146c00 con 0x7f1bc01398c0 2024-05-09T04:33:55.432 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.430+0000 7f1bc7400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f1bc006ec40 con 0x7f1bc0139d20 2024-05-09T04:33:55.432 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.431+0000 7f1bc4a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1ba8004800 0x7f1bc014f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:55.432 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.431+0000 7f1bc4a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1ba8004800 0x7f1bc014f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:44920/0 (socket says 172.21.15.145:44920) 2024-05-09T04:33:55.432 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.431+0000 7f1bc4a00640 1 -- 172.21.15.145:0/2112751817 learned_addr learned my addr 172.21.15.145:0/2112751817 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:55.432 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.431+0000 7f1bc5400640 1 --2- 172.21.15.145:0/2112751817 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1bc0139d20 0x7f1bc0141b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:55.432 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.431+0000 7f1bbfe00640 1 --2- 172.21.15.145:0/2112751817 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1bc01398c0 0x7f1bc0072c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:55.432 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.431+0000 7f1bc4a00640 1 -- 172.21.15.145:0/2112751817 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1bc0139d20 msgr2=0x7f1bc0141b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:55.432 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.431+0000 7f1bc4a00640 1 --2- 172.21.15.145:0/2112751817 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1bc0139d20 0x7f1bc0141b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:55.432 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.431+0000 7f1bc4a00640 1 -- 172.21.15.145:0/2112751817 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1bc01398c0 msgr2=0x7f1bc0072c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:55.432 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.431+0000 7f1bc4a00640 1 --2- 172.21.15.145:0/2112751817 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1bc01398c0 0x7f1bc0072c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:55.432 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.431+0000 7f1bc4a00640 1 -- 172.21.15.145:0/2112751817 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bc0073680 con 0x7f1ba8004800 2024-05-09T04:33:55.432 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.431+0000 7f1bbfe00640 1 --2- 172.21.15.145:0/2112751817 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1bc01398c0 0x7f1bc0072c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:33:55.432 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.431+0000 7f1bc4a00640 1 --2- 172.21.15.145:0/2112751817 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1ba8004800 0x7f1bc014f110 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1bb400d7a0 tx=0x7f1bb400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:55.433 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.432+0000 7f1bbcc00640 1 -- 172.21.15.145:0/2112751817 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb40072d0 con 0x7f1ba8004800 2024-05-09T04:33:55.433 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.432+0000 7f1bbcc00640 1 -- 172.21.15.145:0/2112751817 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bb4007470 con 0x7f1ba8004800 2024-05-09T04:33:55.433 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.432+0000 7f1bbcc00640 1 -- 172.21.15.145:0/2112751817 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb4004a20 con 0x7f1ba8004800 2024-05-09T04:33:55.433 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.432+0000 7f1bc7400640 1 -- 172.21.15.145:0/2112751817 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bc013bfb0 con 0x7f1ba8004800 2024-05-09T04:33:55.433 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.432+0000 7f1bc7400640 1 -- 172.21.15.145:0/2112751817 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bc013c570 con 0x7f1ba8004800 2024-05-09T04:33:55.433 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.433+0000 7f1bc7400640 1 -- 172.21.15.145:0/2112751817 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bc006a310 con 0x7f1ba8004800 2024-05-09T04:33:55.433 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.434+0000 7f1bbcc00640 1 -- 172.21.15.145:0/2112751817 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1bb401b020 con 0x7f1ba8004800 2024-05-09T04:33:55.435 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.435+0000 7f1bbcc00640 1 --2- 172.21.15.145:0/2112751817 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f1ba00762d0 0x7f1ba0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:55.435 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.435+0000 7f1bbcc00640 1 -- 172.21.15.145:0/2112751817 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1bb40973d0 con 0x7f1ba8004800 2024-05-09T04:33:55.435 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.435+0000 7f1bbfe00640 1 --2- 172.21.15.145:0/2112751817 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f1ba00762d0 0x7f1ba0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:55.435 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.436+0000 7f1bbfe00640 1 --2- 172.21.15.145:0/2112751817 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f1ba00762d0 0x7f1ba0078790 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1bac032000 tx=0x7f1bac02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:55.439 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.440+0000 7f1bbcc00640 1 -- 172.21.15.145:0/2112751817 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f1bb4060ca0 con 0x7f1ba8004800 2024-05-09T04:33:55.691 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.692+0000 7f1bc7400640 1 -- 172.21.15.145:0/2112751817 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1bc01437e0 con 0x7f1ba8004800 2024-05-09T04:33:55.693 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.693+0000 7f1bbcc00640 1 -- 172.21.15.145:0/2112751817 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5187 (secure 0 0 0) 0x7f1bb40605a0 con 0x7f1ba8004800 2024-05-09T04:33:55.693 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:33:55.693 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/1284540932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":1284540932},{"type":"v1","addr":"172.21.15.145:6835","nonce":1284540932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/1308656248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1308656248},{"type":"v1","addr":"172.21.15.145:6837","nonce":1308656248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3323136057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3323136057},{"type":"v1","addr":"172.21.15.192:6833","nonce":3323136057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T04:33:51.628921+0000","modified":"2024-05-09T04:33:54.699907+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.192:6835/1113351558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1113351558},{"type":"v1","addr":"172.21.15.192:6835","nonce":1113351558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4270,"qdb_cluster":[4270]},"id":1}]} 2024-05-09T04:33:55.693 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 8 2024-05-09T04:33:55.696 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.695+0000 7f1bc7400640 1 -- 172.21.15.145:0/2112751817 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f1ba00762d0 msgr2=0x7f1ba0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:55.696 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.696+0000 7f1bc7400640 1 --2- 172.21.15.145:0/2112751817 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f1ba00762d0 0x7f1ba0078790 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1bac032000 tx=0x7f1bac02e000 comp rx=0 tx=0).stop 2024-05-09T04:33:55.696 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.696+0000 7f1bc7400640 1 -- 172.21.15.145:0/2112751817 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1ba8004800 msgr2=0x7f1bc014f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:55.696 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.696+0000 7f1bc7400640 1 --2- 172.21.15.145:0/2112751817 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1ba8004800 0x7f1bc014f110 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1bb400d7a0 tx=0x7f1bb400dc60 comp rx=0 tx=0).stop 2024-05-09T04:33:55.696 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.696+0000 7f1bc7400640 1 -- 172.21.15.145:0/2112751817 shutdown_connections 2024-05-09T04:33:55.696 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.696+0000 7f1bc7400640 1 --2- 172.21.15.145:0/2112751817 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f1ba00762d0 0x7f1ba0078790 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:55.696 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.696+0000 7f1bc7400640 1 --2- 172.21.15.145:0/2112751817 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1bc0139d20 0x7f1bc0141b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:55.696 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.696+0000 7f1bc7400640 1 --2- 172.21.15.145:0/2112751817 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1bc01398c0 0x7f1bc0072c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:55.696 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.696+0000 7f1bc7400640 1 --2- 172.21.15.145:0/2112751817 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1ba8004800 0x7f1bc014f110 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:55.696 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.696+0000 7f1bc7400640 1 -- 172.21.15.145:0/2112751817 >> 172.21.15.145:0/2112751817 conn(0x7f1bc006c870 msgr2=0x7f1bc015fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:55.697 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.696+0000 7f1bc7400640 1 -- 172.21.15.145:0/2112751817 shutdown_connections 2024-05-09T04:33:55.697 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.696+0000 7f1bc7400640 1 -- 172.21.15.145:0/2112751817 wait complete. 2024-05-09T04:33:55.716 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T04:33:55.915 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.915+0000 7fda0a800640 1 Processor -- start 2024-05-09T04:33:55.916 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.915+0000 7fda0a800640 1 -- start start 2024-05-09T04:33:55.917 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.917+0000 7fda0a800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fda040b51f0 0x7fda040afa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:55.917 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.917+0000 7fda0a800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda04072c00 con 0x7fda040af610 2024-05-09T04:33:55.917 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.917+0000 7fda0a800640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda04072ec0 con 0x7fda040b4700 2024-05-09T04:33:55.917 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.917+0000 7fda0a800640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fda040aea70 con 0x7fda040b51f0 2024-05-09T04:33:55.917 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.917+0000 7fda09e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fda040b51f0 0x7fda040afa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:55.918 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.917+0000 7fda09e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fda040b51f0 0x7fda040afa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:44932/0 (socket says 172.21.15.145:44932) 2024-05-09T04:33:55.918 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.917+0000 7fda09e00640 1 -- 172.21.15.145:0/3048680442 learned_addr learned my addr 172.21.15.145:0/3048680442 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:55.918 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.917+0000 7fda08a00640 1 --1- 172.21.15.145:0/3048680442 >> v1:172.21.15.145:6789/0 conn(0x7fda040af610 0x7fda040b4ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:33886/0 (socket says 172.21.15.145:33886) 2024-05-09T04:33:55.918 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.918+0000 7fda09e00640 1 -- 172.21.15.145:0/3048680442 >> v1:172.21.15.192:6789/0 conn(0x7fda040b4700 legacy=0x7fda04073c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:55.918 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.918+0000 7fda09e00640 1 -- 172.21.15.145:0/3048680442 >> v1:172.21.15.145:6789/0 conn(0x7fda040af610 legacy=0x7fda040b4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:55.918 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.918+0000 7fda09e00640 1 -- 172.21.15.145:0/3048680442 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda040731a0 con 0x7fda040b51f0 2024-05-09T04:33:55.919 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.919+0000 7fda09e00640 1 --2- 172.21.15.145:0/3048680442 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda040b51f0 0x7fda040afa40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc009e50 tx=0x7fd9fc02a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bc92bd69997264f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:55.919 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.919+0000 7fda03e00640 1 -- 172.21.15.145:0/3048680442 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc036070 con 0x7fda040b51f0 2024-05-09T04:33:55.919 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.919+0000 7fda03e00640 1 -- 172.21.15.145:0/3048680442 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9fc031600 con 0x7fda040b51f0 2024-05-09T04:33:55.919 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.919+0000 7fda03e00640 1 -- 172.21.15.145:0/3048680442 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc0319f0 con 0x7fda040b51f0 2024-05-09T04:33:55.920 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.920+0000 7fda0a800640 1 -- 172.21.15.145:0/3048680442 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda040b51f0 msgr2=0x7fda040afa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:55.920 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.920+0000 7fda0a800640 1 --2- 172.21.15.145:0/3048680442 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda040b51f0 0x7fda040afa40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc009e50 tx=0x7fd9fc02a7c0 comp rx=0 tx=0).stop 2024-05-09T04:33:55.920 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.920+0000 7fda0a800640 1 -- 172.21.15.145:0/3048680442 shutdown_connections 2024-05-09T04:33:55.920 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.920+0000 7fda0a800640 1 --2- 172.21.15.145:0/3048680442 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda040b51f0 0x7fda040afa40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:55.920 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.920+0000 7fda0a800640 1 -- 172.21.15.145:0/3048680442 >> 172.21.15.145:0/3048680442 conn(0x7fda0406c8a0 msgr2=0x7fda0406ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:55.920 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.920+0000 7fda0a800640 1 -- 172.21.15.145:0/3048680442 shutdown_connections 2024-05-09T04:33:55.920 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.920+0000 7fda0a800640 1 -- 172.21.15.145:0/3048680442 wait complete. 2024-05-09T04:33:55.921 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.922+0000 7fda0a800640 1 Processor -- start 2024-05-09T04:33:55.922 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.922+0000 7fda0a800640 1 -- start start 2024-05-09T04:33:55.923 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.923+0000 7fda0a800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda040af610 0x7fda0414a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:55.923 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.923+0000 7fda0a800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fda040b4700 0x7fda0414add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:55.923 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.923+0000 7fda0a800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda040b51f0 0x7fda0415fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:55.923 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.923+0000 7fda0a800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fda0406e320 con 0x7fda040b51f0 2024-05-09T04:33:55.924 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.923+0000 7fda0a800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fda040737f0 con 0x7fda040af610 2024-05-09T04:33:55.924 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.923+0000 7fda0a800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fda0406e4a0 con 0x7fda040b4700 2024-05-09T04:33:55.924 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.924+0000 7fda09e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda040b51f0 0x7fda0415fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:55.924 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.924+0000 7fda09e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda040b51f0 0x7fda0415fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:44940/0 (socket says 172.21.15.145:44940) 2024-05-09T04:33:55.924 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.924+0000 7fda09e00640 1 -- 172.21.15.145:0/3344536468 learned_addr learned my addr 172.21.15.145:0/3344536468 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:55.924 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.924+0000 7fda08a00640 1 --2- 172.21.15.145:0/3344536468 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fda040b4700 0x7fda0414add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:55.924 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.924+0000 7fda09400640 1 --2- 172.21.15.145:0/3344536468 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda040af610 0x7fda0414a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:55.925 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.924+0000 7fda09e00640 1 -- 172.21.15.145:0/3344536468 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fda040b4700 msgr2=0x7fda0414add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:55.925 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.924+0000 7fda09e00640 1 --2- 172.21.15.145:0/3344536468 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fda040b4700 0x7fda0414add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:55.925 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.924+0000 7fda09e00640 1 -- 172.21.15.145:0/3344536468 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda040af610 msgr2=0x7fda0414a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:55.925 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.924+0000 7fda09e00640 1 --2- 172.21.15.145:0/3344536468 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda040af610 0x7fda0414a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:55.925 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.924+0000 7fda09e00640 1 -- 172.21.15.145:0/3344536468 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda0415edf0 con 0x7fda040b51f0 2024-05-09T04:33:55.925 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.924+0000 7fda09e00640 1 --2- 172.21.15.145:0/3344536468 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda040b51f0 0x7fda0415fe50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc034040 tx=0x7fd9fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:55.925 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.925+0000 7fda01600640 1 -- 172.21.15.145:0/3344536468 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc036050 con 0x7fda040b51f0 2024-05-09T04:33:55.926 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.925+0000 7fda01600640 1 -- 172.21.15.145:0/3344536468 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9fc042070 con 0x7fda040b51f0 2024-05-09T04:33:55.926 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.925+0000 7fda01600640 1 -- 172.21.15.145:0/3344536468 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc009510 con 0x7fda040b51f0 2024-05-09T04:33:55.926 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.925+0000 7fda0a800640 1 -- 172.21.15.145:0/3344536468 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda0415f090 con 0x7fda040b51f0 2024-05-09T04:33:55.926 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.925+0000 7fda0a800640 1 -- 172.21.15.145:0/3344536468 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda0415f650 con 0x7fda040b51f0 2024-05-09T04:33:55.926 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.926+0000 7fda0a800640 1 -- 172.21.15.145:0/3344536468 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda0406a310 con 0x7fda040b51f0 2024-05-09T04:33:55.927 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.927+0000 7fda01600640 1 -- 172.21.15.145:0/3344536468 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd9fc004470 con 0x7fda040b51f0 2024-05-09T04:33:55.929 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.929+0000 7fda01600640 1 --2- 172.21.15.145:0/3344536468 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fd9e40762d0 0x7fd9e4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:55.929 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.929+0000 7fda01600640 1 -- 172.21.15.145:0/3344536468 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd9fc0b85a0 con 0x7fda040b51f0 2024-05-09T04:33:55.929 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.929+0000 7fda09400640 1 --2- 172.21.15.145:0/3344536468 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fd9e40762d0 0x7fd9e4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:55.929 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.930+0000 7fda09400640 1 --2- 172.21.15.145:0/3344536468 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fd9e40762d0 0x7fd9e4078790 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8001680 tx=0x7fd9f8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:55.933 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:55.933+0000 7fda01600640 1 -- 172.21.15.145:0/3344536468 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd9fc081ef0 con 0x7fda040b51f0 2024-05-09T04:33:56.138 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.138+0000 7fda0a800640 1 -- 172.21.15.145:0/3344536468 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fda041621e0 con 0x7fda040b51f0 2024-05-09T04:33:56.140 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.141+0000 7fda01600640 1 -- 172.21.15.145:0/3344536468 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16653 (secure 0 0 0) 0x7fd9fc0817f0 con 0x7fda040b51f0 2024-05-09T04:33:56.141 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:33:56.141 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":20,"fsid":"957ee6c1-073b-44cf-9b93-9491b3fdfc1d","created":"2024-05-09T04:33:26.544367+0000","modified":"2024-05-09T04:33:53.178020+0000","last_up_change":"2024-05-09T04:33:39.039318+0000","last_in_change":"2024-05-09T04:33:29.902361+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T04:33:39.795344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T04:33:49.651743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T04:33:50.596065+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22e550ad-847b-4952-80e6-63d859ed1a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6801","nonce":3182572912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6803","nonce":3182572912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6807","nonce":3182572912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6805","nonce":3182572912}]},"public_addr":"172.21.15.145:6801/3182572912","cluster_addr":"172.21.15.145:6803/3182572912","heartbeat_back_addr":"172.21.15.145:6807/3182572912","heartbeat_front_addr":"172.21.15.145:6805/3182572912","state":["exists","up"]},{"osd":1,"uuid":"0f6b9fca-0c41-4145-8d74-598ddb42f2a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6825","nonce":1845225555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6827","nonce":1845225555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6831","nonce":1845225555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6829","nonce":1845225555}]},"public_addr":"172.21.15.145:6825/1845225555","cluster_addr":"172.21.15.145:6827/1845225555","heartbeat_back_addr":"172.21.15.145:6831/1845225555","heartbeat_front_addr":"172.21.15.145:6829/1845225555","state":["exists","up"]},{"osd":2,"uuid":"b3b1f67e-5790-483f-8610-e05a5bce4c30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6817","nonce":2168958993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6819","nonce":2168958993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6823","nonce":2168958993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6821","nonce":2168958993}]},"public_addr":"172.21.15.145:6817/2168958993","cluster_addr":"172.21.15.145:6819/2168958993","heartbeat_back_addr":"172.21.15.145:6823/2168958993","heartbeat_front_addr":"172.21.15.145:6821/2168958993","state":["exists","up"]},{"osd":3,"uuid":"884787a5-e037-4d31-9aae-743030eba4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6809","nonce":337009826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6811","nonce":337009826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6815","nonce":337009826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6813","nonce":337009826}]},"public_addr":"172.21.15.145:6809/337009826","cluster_addr":"172.21.15.145:6811/337009826","heartbeat_back_addr":"172.21.15.145:6815/337009826","heartbeat_front_addr":"172.21.15.145:6813/337009826","state":["exists","up"]},{"osd":4,"uuid":"0759fc1e-c5fc-4a2f-86bb-e8eccd83f833","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6818","nonce":3117700995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6821","nonce":3117700995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6827","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6829","nonce":3117700995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6823","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6825","nonce":3117700995}]},"public_addr":"172.21.15.192:6818/3117700995","cluster_addr":"172.21.15.192:6821/3117700995","heartbeat_back_addr":"172.21.15.192:6829/3117700995","heartbeat_front_addr":"172.21.15.192:6825/3117700995","state":["exists","up"]},{"osd":5,"uuid":"9443cc22-9379-46e3-9173-98b592bf2c16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6801","nonce":1328373268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6803","nonce":1328373268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6807","nonce":1328373268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6805","nonce":1328373268}]},"public_addr":"172.21.15.192:6801/1328373268","cluster_addr":"172.21.15.192:6803/1328373268","heartbeat_back_addr":"172.21.15.192:6807/1328373268","heartbeat_front_addr":"172.21.15.192:6805/1328373268","state":["exists","up"]},{"osd":6,"uuid":"d05a3311-505a-45b5-85b9-aa3113e96987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6817","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6820","nonce":1868011057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6824","nonce":1868011057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6831","nonce":1868011057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6828","nonce":1868011057}]},"public_addr":"172.21.15.192:6820/1868011057","cluster_addr":"172.21.15.192:6824/1868011057","heartbeat_back_addr":"172.21.15.192:6831/1868011057","heartbeat_front_addr":"172.21.15.192:6828/1868011057","state":["exists","up"]},{"osd":7,"uuid":"c7c23dbc-2029-43ec-9f53-25f3d58019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6809","nonce":3368512157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6811","nonce":3368512157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6815","nonce":3368512157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6813","nonce":3368512157}]},"public_addr":"172.21.15.192:6809/3368512157","cluster_addr":"172.21.15.192:6811/3368512157","heartbeat_back_addr":"172.21.15.192:6815/3368512157","heartbeat_front_addr":"172.21.15.192:6813/3368512157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.091188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.032567+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:34.797468+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.228839+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.622400+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.473103+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.566092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.349395+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T04:33:56.144 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.143+0000 7fda0a800640 1 -- 172.21.15.145:0/3344536468 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fd9e40762d0 msgr2=0x7fd9e4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:56.144 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.143+0000 7fda0a800640 1 --2- 172.21.15.145:0/3344536468 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fd9e40762d0 0x7fd9e4078790 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8001680 tx=0x7fd9f8009040 comp rx=0 tx=0).stop 2024-05-09T04:33:56.144 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.144+0000 7fda0a800640 1 -- 172.21.15.145:0/3344536468 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda040b51f0 msgr2=0x7fda0415fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:56.144 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.144+0000 7fda0a800640 1 --2- 172.21.15.145:0/3344536468 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda040b51f0 0x7fda0415fe50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc034040 tx=0x7fd9fc004a40 comp rx=0 tx=0).stop 2024-05-09T04:33:56.144 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.144+0000 7fda0a800640 1 -- 172.21.15.145:0/3344536468 shutdown_connections 2024-05-09T04:33:56.144 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.144+0000 7fda0a800640 1 --2- 172.21.15.145:0/3344536468 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fd9e40762d0 0x7fd9e4078790 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:56.144 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.144+0000 7fda0a800640 1 --2- 172.21.15.145:0/3344536468 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fda040b51f0 0x7fda0415fe50 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:56.144 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.144+0000 7fda0a800640 1 --2- 172.21.15.145:0/3344536468 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fda040b4700 0x7fda0414add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:56.144 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.144+0000 7fda0a800640 1 --2- 172.21.15.145:0/3344536468 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda040af610 0x7fda0414a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:56.144 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.144+0000 7fda0a800640 1 -- 172.21.15.145:0/3344536468 >> 172.21.15.145:0/3344536468 conn(0x7fda0406c8a0 msgr2=0x7fda040b18a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:56.145 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.144+0000 7fda0a800640 1 -- 172.21.15.145:0/3344536468 shutdown_connections 2024-05-09T04:33:56.145 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.144+0000 7fda0a800640 1 -- 172.21.15.145:0/3344536468 wait complete. 2024-05-09T04:33:56.163 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-09T04:33:56.164 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T04:33:56.357 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.357+0000 7f42e2600640 1 Processor -- start 2024-05-09T04:33:56.357 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.357+0000 7f42e2600640 1 -- start start 2024-05-09T04:33:56.358 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.358+0000 7f42e2600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f42dc148c20 0x7f42dc0affc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:56.359 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.358+0000 7f42e2600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42dc072350 con 0x7f42dc06ef70 2024-05-09T04:33:56.359 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.358+0000 7f42e2600640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42dc072630 con 0x7f42dc145880 2024-05-09T04:33:56.359 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.358+0000 7f42e2600640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f42dc072f60 con 0x7f42dc148c20 2024-05-09T04:33:56.359 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.359+0000 7f42daa00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f42dc06ef70 0x7f42dc0b51b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:33890/0 (socket says 172.21.15.145:33890) 2024-05-09T04:33:56.359 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.359+0000 7f42daa00640 1 -- 172.21.15.145:0/1380516963 learned_addr learned my addr 172.21.15.145:0/1380516963 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:33:56.359 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.359+0000 7f42dbe00640 1 --2- 172.21.15.145:0/1380516963 >> v2:172.21.15.145:3300/0 conn(0x7f42dc148c20 0x7f42dc0affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:56.359 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.359+0000 7f42dbe00640 1 -- 172.21.15.145:0/1380516963 >> v1:172.21.15.192:6789/0 conn(0x7f42dc145880 legacy=0x7f42dc06e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:56.359 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.359+0000 7f42dbe00640 1 -- 172.21.15.145:0/1380516963 >> v1:172.21.15.145:6789/0 conn(0x7f42dc06ef70 legacy=0x7f42dc0b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:56.360 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.359+0000 7f42dbe00640 1 -- 172.21.15.145:0/1380516963 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42dc072910 con 0x7f42dc148c20 2024-05-09T04:33:56.361 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.360+0000 7f42dbe00640 1 --2- 172.21.15.145:0/1380516963 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f42dc148c20 0x7f42dc0affc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f42d4009d90 tx=0x7f42d402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3792a5ef9b4e4208 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:56.361 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.360+0000 7f42da000640 1 -- 172.21.15.145:0/1380516963 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d4031600 con 0x7f42dc148c20 2024-05-09T04:33:56.361 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.361+0000 7f42da000640 1 -- 172.21.15.145:0/1380516963 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42d40317a0 con 0x7f42dc148c20 2024-05-09T04:33:56.361 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.361+0000 7f42da000640 1 -- 172.21.15.145:0/1380516963 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d4031b70 con 0x7f42dc148c20 2024-05-09T04:33:56.361 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.361+0000 7f42e2600640 1 -- 172.21.15.145:0/1380516963 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f42dc148c20 msgr2=0x7f42dc0affc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:56.361 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.361+0000 7f42e2600640 1 --2- 172.21.15.145:0/1380516963 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f42dc148c20 0x7f42dc0affc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f42d4009d90 tx=0x7f42d402a5d0 comp rx=0 tx=0).stop 2024-05-09T04:33:56.361 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.361+0000 7f42e2600640 1 -- 172.21.15.145:0/1380516963 shutdown_connections 2024-05-09T04:33:56.361 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.361+0000 7f42e2600640 1 --2- 172.21.15.145:0/1380516963 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f42dc148c20 0x7f42dc0affc0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:56.361 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.361+0000 7f42e2600640 1 -- 172.21.15.145:0/1380516963 >> 172.21.15.145:0/1380516963 conn(0x7f42dc06c870 msgr2=0x7f42dc06cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:56.362 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.361+0000 7f42e2600640 1 -- 172.21.15.145:0/1380516963 shutdown_connections 2024-05-09T04:33:56.362 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.361+0000 7f42e2600640 1 -- 172.21.15.145:0/1380516963 wait complete. 2024-05-09T04:33:56.362 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.363+0000 7f42e2600640 1 Processor -- start 2024-05-09T04:33:56.363 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.363+0000 7f42e2600640 1 -- start start 2024-05-09T04:33:56.365 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.364+0000 7f42e2600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f42dc06ef70 0x7f42dc073880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:56.365 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.364+0000 7f42e2600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42dc145880 0x7f42dc131880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:56.365 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.364+0000 7f42e2600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f42dc148c20 0x7f42dc162300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:56.365 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.364+0000 7f42e2600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f42dc0b58c0 con 0x7f42dc148c20 2024-05-09T04:33:56.365 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.364+0000 7f42e2600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f42dc145c60 con 0x7f42dc145880 2024-05-09T04:33:56.365 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.364+0000 7f42e2600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f42dc0b5a40 con 0x7f42dc06ef70 2024-05-09T04:33:56.365 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.364+0000 7f42dbe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f42dc148c20 0x7f42dc162300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:56.365 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.364+0000 7f42dbe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f42dc148c20 0x7f42dc162300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:44952/0 (socket says 172.21.15.145:44952) 2024-05-09T04:33:56.366 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.364+0000 7f42dbe00640 1 -- 172.21.15.145:0/1541266668 learned_addr learned my addr 172.21.15.145:0/1541266668 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:56.366 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.365+0000 7f42daa00640 1 --2- 172.21.15.145:0/1541266668 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42dc145880 0x7f42dc131880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:56.366 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.365+0000 7f42dbe00640 1 -- 172.21.15.145:0/1541266668 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f42dc06ef70 msgr2=0x7f42dc073880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:56.366 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.365+0000 7f42dbe00640 1 --2- 172.21.15.145:0/1541266668 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f42dc06ef70 0x7f42dc073880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:56.366 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.365+0000 7f42dbe00640 1 -- 172.21.15.145:0/1541266668 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42dc145880 msgr2=0x7f42dc131880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:56.366 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.365+0000 7f42dbe00640 1 --2- 172.21.15.145:0/1541266668 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42dc145880 0x7f42dc131880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:56.366 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.365+0000 7f42dbe00640 1 -- 172.21.15.145:0/1541266668 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42dc137be0 con 0x7f42dc148c20 2024-05-09T04:33:56.366 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.365+0000 7f42dbe00640 1 --2- 172.21.15.145:0/1541266668 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f42dc148c20 0x7f42dc162300 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f42d4009d90 tx=0x7f42d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:56.366 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.365+0000 7f42d3400640 1 -- 172.21.15.145:0/1541266668 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d403f070 con 0x7f42dc148c20 2024-05-09T04:33:56.366 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.365+0000 7f42d3400640 1 -- 172.21.15.145:0/1541266668 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42d4030b60 con 0x7f42dc148c20 2024-05-09T04:33:56.366 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.365+0000 7f42d3400640 1 -- 172.21.15.145:0/1541266668 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d4043410 con 0x7f42dc148c20 2024-05-09T04:33:56.366 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.365+0000 7f42e2600640 1 -- 172.21.15.145:0/1541266668 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42dc1f8bf0 con 0x7f42dc148c20 2024-05-09T04:33:56.366 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.365+0000 7f42e2600640 1 -- 172.21.15.145:0/1541266668 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42dc161b00 con 0x7f42dc148c20 2024-05-09T04:33:56.367 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.367+0000 7f42e2600640 1 -- 172.21.15.145:0/1541266668 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42dc06a310 con 0x7f42dc148c20 2024-05-09T04:33:56.367 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.368+0000 7f42d3400640 1 -- 172.21.15.145:0/1541266668 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f42d403b410 con 0x7f42dc148c20 2024-05-09T04:33:56.368 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.368+0000 7f42d3400640 1 --2- 172.21.15.145:0/1541266668 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f42bc0762d0 0x7f42bc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:56.368 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.369+0000 7f42d3400640 1 -- 172.21.15.145:0/1541266668 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f42d40b8560 con 0x7f42dc148c20 2024-05-09T04:33:56.368 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.369+0000 7f42db400640 1 --2- 172.21.15.145:0/1541266668 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f42bc0762d0 0x7f42bc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:56.369 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.369+0000 7f42db400640 1 --2- 172.21.15.145:0/1541266668 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f42bc0762d0 0x7f42bc078790 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f42cc003cc0 tx=0x7f42cc0079b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:56.373 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.373+0000 7f42d3400640 1 -- 172.21.15.145:0/1541266668 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f42dc1f8bf0 con 0x7f42dc148c20 2024-05-09T04:33:56.563 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.563+0000 7f42e2600640 1 -- 172.21.15.145:0/1541266668 --> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f42dc06d460 con 0x7f42bc0762d0 2024-05-09T04:33:56.586 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.586+0000 7f42d3400640 1 -- 172.21.15.145:0/1541266668 <== mgr.4104 v2:172.21.15.145:6832/2725855396 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251924 (secure 0 0 0) 0x7f42dc06d460 con 0x7f42bc0762d0 2024-05-09T04:33:56.590 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:33:56.591 INFO:teuthology.orchestra.run.smithi145.stderr:dumped all 2024-05-09T04:33:56.593 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.591+0000 7f42e2600640 1 -- 172.21.15.145:0/1541266668 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f42bc0762d0 msgr2=0x7f42bc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:56.593 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.591+0000 7f42e2600640 1 --2- 172.21.15.145:0/1541266668 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f42bc0762d0 0x7f42bc078790 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f42cc003cc0 tx=0x7f42cc0079b0 comp rx=0 tx=0).stop 2024-05-09T04:33:56.593 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.591+0000 7f42e2600640 1 -- 172.21.15.145:0/1541266668 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f42dc148c20 msgr2=0x7f42dc162300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:56.593 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.591+0000 7f42e2600640 1 --2- 172.21.15.145:0/1541266668 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f42dc148c20 0x7f42dc162300 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f42d4009d90 tx=0x7f42d4004a40 comp rx=0 tx=0).stop 2024-05-09T04:33:56.594 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.592+0000 7f42e2600640 1 -- 172.21.15.145:0/1541266668 shutdown_connections 2024-05-09T04:33:56.594 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.592+0000 7f42e2600640 1 --2- 172.21.15.145:0/1541266668 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f42bc0762d0 0x7f42bc078790 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:56.594 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.592+0000 7f42e2600640 1 --2- 172.21.15.145:0/1541266668 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f42dc148c20 0x7f42dc162300 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:56.594 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.592+0000 7f42e2600640 1 --2- 172.21.15.145:0/1541266668 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42dc145880 0x7f42dc131880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:56.594 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.592+0000 7f42e2600640 1 --2- 172.21.15.145:0/1541266668 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f42dc06ef70 0x7f42dc073880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:56.594 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.592+0000 7f42e2600640 1 -- 172.21.15.145:0/1541266668 >> 172.21.15.145:0/1541266668 conn(0x7f42dc06c870 msgr2=0x7f42dc0af6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:56.594 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.592+0000 7f42e2600640 1 -- 172.21.15.145:0/1541266668 shutdown_connections 2024-05-09T04:33:56.594 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.592+0000 7f42e2600640 1 -- 172.21.15.145:0/1541266668 wait complete. 2024-05-09T04:33:56.609 INFO:teuthology.orchestra.run.smithi145.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-05-09T04:33:55.757551+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":132,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218976,"kb_used_data":2788,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755744,"statfs":{"total":773094113280,"available":772869881856,"internally_reserved":0,"allocated":2854912,"data_stored":1734128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.625557"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T04:33:52.204965+0000","last_change":"2024-05-09T04:33:52.200596+0000","last_active":"2024-05-09T04:33:52.204965+0000","last_peered":"2024-05-09T04:33:52.204965+0000","last_clean":"2024-05-09T04:33:52.204965+0000","last_became_active":"2024-05-09T04:33:52.199781+0000","last_became_peered":"2024-05-09T04:33:52.199781+0000","last_unstale":"2024-05-09T04:33:52.204965+0000","last_undegraded":"2024-05-09T04:33:52.204965+0000","last_fullsized":"2024-05-09T04:33:52.204965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:33:25.042543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.154485+0000","last_change":"2024-05-09T04:33:52.186325+0000","last_active":"2024-05-09T04:33:54.154485+0000","last_peered":"2024-05-09T04:33:54.154485+0000","last_clean":"2024-05-09T04:33:54.154485+0000","last_became_active":"2024-05-09T04:33:52.186047+0000","last_became_peered":"2024-05-09T04:33:52.186047+0000","last_unstale":"2024-05-09T04:33:54.154485+0000","last_undegraded":"2024-05-09T04:33:54.154485+0000","last_fullsized":"2024-05-09T04:33:54.154485+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:24:37.669022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.194578+0000","last_change":"2024-05-09T04:33:52.187398+0000","last_active":"2024-05-09T04:33:53.194578+0000","last_peered":"2024-05-09T04:33:53.194578+0000","last_clean":"2024-05-09T04:33:53.194578+0000","last_became_active":"2024-05-09T04:33:52.186979+0000","last_became_peered":"2024-05-09T04:33:52.186979+0000","last_unstale":"2024-05-09T04:33:53.194578+0000","last_undegraded":"2024-05-09T04:33:53.194578+0000","last_fullsized":"2024-05-09T04:33:53.194578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:34:29.852986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.211667+0000","last_change":"2024-05-09T04:33:52.210098+0000","last_active":"2024-05-09T04:33:54.211667+0000","last_peered":"2024-05-09T04:33:54.211667+0000","last_clean":"2024-05-09T04:33:54.211667+0000","last_became_active":"2024-05-09T04:33:52.209575+0000","last_became_peered":"2024-05-09T04:33:52.209575+0000","last_unstale":"2024-05-09T04:33:54.211667+0000","last_undegraded":"2024-05-09T04:33:54.211667+0000","last_fullsized":"2024-05-09T04:33:54.211667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:24:41.478222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T04:33:52.202014+0000","last_change":"2024-05-09T04:33:52.195969+0000","last_active":"2024-05-09T04:33:52.202014+0000","last_peered":"2024-05-09T04:33:52.202014+0000","last_clean":"2024-05-09T04:33:52.202014+0000","last_became_active":"2024-05-09T04:33:52.195578+0000","last_became_peered":"2024-05-09T04:33:52.195578+0000","last_unstale":"2024-05-09T04:33:52.202014+0000","last_undegraded":"2024-05-09T04:33:52.202014+0000","last_fullsized":"2024-05-09T04:33:52.202014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:27:41.648285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.211122+0000","last_change":"2024-05-09T04:33:52.198619+0000","last_active":"2024-05-09T04:33:54.211122+0000","last_peered":"2024-05-09T04:33:54.211122+0000","last_clean":"2024-05-09T04:33:54.211122+0000","last_became_active":"2024-05-09T04:33:52.198345+0000","last_became_peered":"2024-05-09T04:33:52.198345+0000","last_unstale":"2024-05-09T04:33:54.211122+0000","last_undegraded":"2024-05-09T04:33:54.211122+0000","last_fullsized":"2024-05-09T04:33:54.211122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:24:37.669022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T04:33:52.207328+0000","last_change":"2024-05-09T04:33:52.202397+0000","last_active":"2024-05-09T04:33:52.207328+0000","last_peered":"2024-05-09T04:33:52.207328+0000","last_clean":"2024-05-09T04:33:52.207328+0000","last_became_active":"2024-05-09T04:33:52.202176+0000","last_became_peered":"2024-05-09T04:33:52.202176+0000","last_unstale":"2024-05-09T04:33:52.207328+0000","last_undegraded":"2024-05-09T04:33:52.207328+0000","last_fullsized":"2024-05-09T04:33:52.207328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:35:43.478085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.196047+0000","last_change":"2024-05-09T04:33:52.207259+0000","last_active":"2024-05-09T04:33:53.196047+0000","last_peered":"2024-05-09T04:33:53.196047+0000","last_clean":"2024-05-09T04:33:53.196047+0000","last_became_active":"2024-05-09T04:33:52.204206+0000","last_became_peered":"2024-05-09T04:33:52.204206+0000","last_unstale":"2024-05-09T04:33:53.196047+0000","last_undegraded":"2024-05-09T04:33:53.196047+0000","last_fullsized":"2024-05-09T04:33:53.196047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:35:43.478085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T04:33:52.213659+0000","last_change":"2024-05-09T04:33:52.205582+0000","last_active":"2024-05-09T04:33:52.213659+0000","last_peered":"2024-05-09T04:33:52.213659+0000","last_clean":"2024-05-09T04:33:52.213659+0000","last_became_active":"2024-05-09T04:33:52.205237+0000","last_became_peered":"2024-05-09T04:33:52.205237+0000","last_unstale":"2024-05-09T04:33:52.213659+0000","last_undegraded":"2024-05-09T04:33:52.213659+0000","last_fullsized":"2024-05-09T04:33:52.213659+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:34:40.257159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.686517+0000","last_change":"2024-05-09T04:33:52.198058+0000","last_active":"2024-05-09T04:33:53.686517+0000","last_peered":"2024-05-09T04:33:53.686517+0000","last_clean":"2024-05-09T04:33:53.686517+0000","last_became_active":"2024-05-09T04:33:52.197701+0000","last_became_peered":"2024-05-09T04:33:52.197701+0000","last_unstale":"2024-05-09T04:33:53.686517+0000","last_undegraded":"2024-05-09T04:33:53.686517+0000","last_fullsized":"2024-05-09T04:33:53.686517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:24:37.669022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.686063+0000","last_change":"2024-05-09T04:33:52.194755+0000","last_active":"2024-05-09T04:33:53.686063+0000","last_peered":"2024-05-09T04:33:53.686063+0000","last_clean":"2024-05-09T04:33:53.686063+0000","last_became_active":"2024-05-09T04:33:52.194270+0000","last_became_peered":"2024-05-09T04:33:52.194270+0000","last_unstale":"2024-05-09T04:33:53.686063+0000","last_undegraded":"2024-05-09T04:33:53.686063+0000","last_fullsized":"2024-05-09T04:33:53.686063+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:59:28.002585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.195766+0000","last_change":"2024-05-09T04:33:52.200331+0000","last_active":"2024-05-09T04:33:53.195766+0000","last_peered":"2024-05-09T04:33:53.195766+0000","last_clean":"2024-05-09T04:33:53.195766+0000","last_became_active":"2024-05-09T04:33:52.199739+0000","last_became_peered":"2024-05-09T04:33:52.199739+0000","last_unstale":"2024-05-09T04:33:53.195766+0000","last_undegraded":"2024-05-09T04:33:53.195766+0000","last_fullsized":"2024-05-09T04:33:53.195766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:29:12.420301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.196580+0000","last_change":"2024-05-09T04:33:52.211817+0000","last_active":"2024-05-09T04:33:53.196580+0000","last_peered":"2024-05-09T04:33:53.196580+0000","last_clean":"2024-05-09T04:33:53.196580+0000","last_became_active":"2024-05-09T04:33:52.211533+0000","last_became_peered":"2024-05-09T04:33:52.211533+0000","last_unstale":"2024-05-09T04:33:53.196580+0000","last_undegraded":"2024-05-09T04:33:53.196580+0000","last_fullsized":"2024-05-09T04:33:53.196580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:34:29.852986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.193922+0000","last_change":"2024-05-09T04:33:52.177443+0000","last_active":"2024-05-09T04:33:53.193922+0000","last_peered":"2024-05-09T04:33:53.193922+0000","last_clean":"2024-05-09T04:33:53.193922+0000","last_became_active":"2024-05-09T04:33:52.177115+0000","last_became_peered":"2024-05-09T04:33:52.177115+0000","last_unstale":"2024-05-09T04:33:53.193922+0000","last_undegraded":"2024-05-09T04:33:53.193922+0000","last_fullsized":"2024-05-09T04:33:53.193922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:33:25.042543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.154598+0000","last_change":"2024-05-09T04:33:52.209143+0000","last_active":"2024-05-09T04:33:54.154598+0000","last_peered":"2024-05-09T04:33:54.154598+0000","last_clean":"2024-05-09T04:33:54.154598+0000","last_became_active":"2024-05-09T04:33:52.206547+0000","last_became_peered":"2024-05-09T04:33:52.206547+0000","last_unstale":"2024-05-09T04:33:54.154598+0000","last_undegraded":"2024-05-09T04:33:54.154598+0000","last_fullsized":"2024-05-09T04:33:54.154598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:02:17.813339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.211033+0000","last_change":"2024-05-09T04:33:52.198040+0000","last_active":"2024-05-09T04:33:54.211033+0000","last_peered":"2024-05-09T04:33:54.211033+0000","last_clean":"2024-05-09T04:33:54.211033+0000","last_became_active":"2024-05-09T04:33:52.197706+0000","last_became_peered":"2024-05-09T04:33:52.197706+0000","last_unstale":"2024-05-09T04:33:54.211033+0000","last_undegraded":"2024-05-09T04:33:54.211033+0000","last_fullsized":"2024-05-09T04:33:54.211033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:17.801441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.212667+0000","last_change":"2024-05-09T04:33:52.195546+0000","last_active":"2024-05-09T04:33:54.212667+0000","last_peered":"2024-05-09T04:33:54.212667+0000","last_clean":"2024-05-09T04:33:54.212667+0000","last_became_active":"2024-05-09T04:33:52.195138+0000","last_became_peered":"2024-05-09T04:33:52.195138+0000","last_unstale":"2024-05-09T04:33:54.212667+0000","last_undegraded":"2024-05-09T04:33:54.212667+0000","last_fullsized":"2024-05-09T04:33:54.212667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:42.767983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.732773+0000","last_change":"2024-05-09T04:33:52.192303+0000","last_active":"2024-05-09T04:33:54.732773+0000","last_peered":"2024-05-09T04:33:54.732773+0000","last_clean":"2024-05-09T04:33:54.732773+0000","last_became_active":"2024-05-09T04:33:52.191878+0000","last_became_peered":"2024-05-09T04:33:52.191878+0000","last_unstale":"2024-05-09T04:33:54.732773+0000","last_undegraded":"2024-05-09T04:33:54.732773+0000","last_fullsized":"2024-05-09T04:33:54.732773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:22:41.930610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T04:33:52.194033+0000","last_change":"2024-05-09T04:33:52.190589+0000","last_active":"2024-05-09T04:33:52.194033+0000","last_peered":"2024-05-09T04:33:52.194033+0000","last_clean":"2024-05-09T04:33:52.194033+0000","last_became_active":"2024-05-09T04:33:52.190312+0000","last_became_peered":"2024-05-09T04:33:52.190312+0000","last_unstale":"2024-05-09T04:33:52.194033+0000","last_undegraded":"2024-05-09T04:33:52.194033+0000","last_fullsized":"2024-05-09T04:33:52.194033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:00:54.458043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.686133+0000","last_change":"2024-05-09T04:33:52.195370+0000","last_active":"2024-05-09T04:33:53.686133+0000","last_peered":"2024-05-09T04:33:53.686133+0000","last_clean":"2024-05-09T04:33:53.686133+0000","last_became_active":"2024-05-09T04:33:52.194414+0000","last_became_peered":"2024-05-09T04:33:52.194414+0000","last_unstale":"2024-05-09T04:33:53.686133+0000","last_undegraded":"2024-05-09T04:33:53.686133+0000","last_fullsized":"2024-05-09T04:33:53.686133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:29:08.279802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.193862+0000","last_change":"2024-05-09T04:33:52.201922+0000","last_active":"2024-05-09T04:33:53.193862+0000","last_peered":"2024-05-09T04:33:53.193862+0000","last_clean":"2024-05-09T04:33:53.193862+0000","last_became_active":"2024-05-09T04:33:52.201635+0000","last_became_peered":"2024-05-09T04:33:52.201635+0000","last_unstale":"2024-05-09T04:33:53.193862+0000","last_undegraded":"2024-05-09T04:33:53.193862+0000","last_fullsized":"2024-05-09T04:33:53.193862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:27:41.648285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.211533+0000","last_change":"2024-05-09T04:33:52.195480+0000","last_active":"2024-05-09T04:33:54.211533+0000","last_peered":"2024-05-09T04:33:54.211533+0000","last_clean":"2024-05-09T04:33:54.211533+0000","last_became_active":"2024-05-09T04:33:52.195195+0000","last_became_peered":"2024-05-09T04:33:52.195195+0000","last_unstale":"2024-05-09T04:33:54.211533+0000","last_undegraded":"2024-05-09T04:33:54.211533+0000","last_fullsized":"2024-05-09T04:33:54.211533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:02:17.813339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.196691+0000","last_change":"2024-05-09T04:33:52.208038+0000","last_active":"2024-05-09T04:33:53.196691+0000","last_peered":"2024-05-09T04:33:53.196691+0000","last_clean":"2024-05-09T04:33:53.196691+0000","last_became_active":"2024-05-09T04:33:52.207755+0000","last_became_peered":"2024-05-09T04:33:52.207755+0000","last_unstale":"2024-05-09T04:33:53.196691+0000","last_undegraded":"2024-05-09T04:33:53.196691+0000","last_fullsized":"2024-05-09T04:33:53.196691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:59:28.002585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.211317+0000","last_change":"2024-05-09T04:33:52.209997+0000","last_active":"2024-05-09T04:33:54.211317+0000","last_peered":"2024-05-09T04:33:54.211317+0000","last_clean":"2024-05-09T04:33:54.211317+0000","last_became_active":"2024-05-09T04:33:52.209702+0000","last_became_peered":"2024-05-09T04:33:52.209702+0000","last_unstale":"2024-05-09T04:33:54.211317+0000","last_undegraded":"2024-05-09T04:33:54.211317+0000","last_fullsized":"2024-05-09T04:33:54.211317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:56.927838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.197678+0000","last_change":"2024-05-09T04:33:52.207838+0000","last_active":"2024-05-09T04:33:53.197678+0000","last_peered":"2024-05-09T04:33:53.197678+0000","last_clean":"2024-05-09T04:33:53.197678+0000","last_became_active":"2024-05-09T04:33:52.207601+0000","last_became_peered":"2024-05-09T04:33:52.207601+0000","last_unstale":"2024-05-09T04:33:53.197678+0000","last_undegraded":"2024-05-09T04:33:53.197678+0000","last_fullsized":"2024-05-09T04:33:53.197678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:56.927838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.686401+0000","last_change":"2024-05-09T04:33:52.196143+0000","last_active":"2024-05-09T04:33:53.686401+0000","last_peered":"2024-05-09T04:33:53.686401+0000","last_clean":"2024-05-09T04:33:53.686401+0000","last_became_active":"2024-05-09T04:33:52.195814+0000","last_became_peered":"2024-05-09T04:33:52.195814+0000","last_unstale":"2024-05-09T04:33:53.686401+0000","last_undegraded":"2024-05-09T04:33:53.686401+0000","last_fullsized":"2024-05-09T04:33:53.686401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:17.801441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.154429+0000","last_change":"2024-05-09T04:33:52.205497+0000","last_active":"2024-05-09T04:33:54.154429+0000","last_peered":"2024-05-09T04:33:54.154429+0000","last_clean":"2024-05-09T04:33:54.154429+0000","last_became_active":"2024-05-09T04:33:52.205219+0000","last_became_peered":"2024-05-09T04:33:52.205219+0000","last_unstale":"2024-05-09T04:33:54.154429+0000","last_undegraded":"2024-05-09T04:33:54.154429+0000","last_fullsized":"2024-05-09T04:33:54.154429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:17.801441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.197606+0000","last_change":"2024-05-09T04:33:52.213356+0000","last_active":"2024-05-09T04:33:53.197606+0000","last_peered":"2024-05-09T04:33:53.197606+0000","last_clean":"2024-05-09T04:33:53.197606+0000","last_became_active":"2024-05-09T04:33:52.212783+0000","last_became_peered":"2024-05-09T04:33:52.212783+0000","last_unstale":"2024-05-09T04:33:53.197606+0000","last_undegraded":"2024-05-09T04:33:53.197606+0000","last_fullsized":"2024-05-09T04:33:53.197606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:24:37.669022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.194008+0000","last_change":"2024-05-09T04:33:52.209567+0000","last_active":"2024-05-09T04:33:53.194008+0000","last_peered":"2024-05-09T04:33:53.194008+0000","last_clean":"2024-05-09T04:33:53.194008+0000","last_became_active":"2024-05-09T04:33:52.209260+0000","last_became_peered":"2024-05-09T04:33:52.209260+0000","last_unstale":"2024-05-09T04:33:53.194008+0000","last_undegraded":"2024-05-09T04:33:53.194008+0000","last_fullsized":"2024-05-09T04:33:53.194008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:35:43.478085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.211635+0000","last_change":"2024-05-09T04:33:52.210744+0000","last_active":"2024-05-09T04:33:54.211635+0000","last_peered":"2024-05-09T04:33:54.211635+0000","last_clean":"2024-05-09T04:33:54.211635+0000","last_became_active":"2024-05-09T04:33:52.210421+0000","last_became_peered":"2024-05-09T04:33:52.210421+0000","last_unstale":"2024-05-09T04:33:54.211635+0000","last_undegraded":"2024-05-09T04:33:54.211635+0000","last_fullsized":"2024-05-09T04:33:54.211635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:13:19.513750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.732744+0000","last_change":"2024-05-09T04:33:52.193832+0000","last_active":"2024-05-09T04:33:54.732744+0000","last_peered":"2024-05-09T04:33:54.732744+0000","last_clean":"2024-05-09T04:33:54.732744+0000","last_became_active":"2024-05-09T04:33:52.193531+0000","last_became_peered":"2024-05-09T04:33:52.193531+0000","last_unstale":"2024-05-09T04:33:54.732744+0000","last_undegraded":"2024-05-09T04:33:54.732744+0000","last_fullsized":"2024-05-09T04:33:54.732744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:19:44.181447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.685556+0000","last_change":"2024-05-09T04:33:52.198677+0000","last_active":"2024-05-09T04:33:53.685556+0000","last_peered":"2024-05-09T04:33:53.685556+0000","last_clean":"2024-05-09T04:33:53.685556+0000","last_became_active":"2024-05-09T04:33:52.198334+0000","last_became_peered":"2024-05-09T04:33:52.198334+0000","last_unstale":"2024-05-09T04:33:53.685556+0000","last_undegraded":"2024-05-09T04:33:53.685556+0000","last_fullsized":"2024-05-09T04:33:53.685556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:53:58.795910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.194924+0000","last_change":"2024-05-09T04:33:52.192680+0000","last_active":"2024-05-09T04:33:53.194924+0000","last_peered":"2024-05-09T04:33:53.194924+0000","last_clean":"2024-05-09T04:33:53.194924+0000","last_became_active":"2024-05-09T04:33:52.192454+0000","last_became_peered":"2024-05-09T04:33:52.192454+0000","last_unstale":"2024-05-09T04:33:53.194924+0000","last_undegraded":"2024-05-09T04:33:53.194924+0000","last_fullsized":"2024-05-09T04:33:53.194924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:50:52.026927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.196881+0000","last_change":"2024-05-09T04:33:51.177063+0000","last_active":"2024-05-09T04:33:53.196881+0000","last_peered":"2024-05-09T04:33:53.196881+0000","last_clean":"2024-05-09T04:33:53.196881+0000","last_became_active":"2024-05-09T04:33:51.176788+0000","last_became_peered":"2024-05-09T04:33:51.176788+0000","last_unstale":"2024-05-09T04:33:53.196881+0000","last_undegraded":"2024-05-09T04:33:53.196881+0000","last_fullsized":"2024-05-09T04:33:53.196881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:04:10.949412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.194154+0000","last_change":"2024-05-09T04:33:52.190946+0000","last_active":"2024-05-09T04:33:53.194154+0000","last_peered":"2024-05-09T04:33:53.194154+0000","last_clean":"2024-05-09T04:33:53.194154+0000","last_became_active":"2024-05-09T04:33:52.190598+0000","last_became_peered":"2024-05-09T04:33:52.190598+0000","last_unstale":"2024-05-09T04:33:53.194154+0000","last_undegraded":"2024-05-09T04:33:53.194154+0000","last_fullsized":"2024-05-09T04:33:53.194154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:53:58.795910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.685499+0000","last_change":"2024-05-09T04:33:51.177930+0000","last_active":"2024-05-09T04:33:53.685499+0000","last_peered":"2024-05-09T04:33:53.685499+0000","last_clean":"2024-05-09T04:33:53.685499+0000","last_became_active":"2024-05-09T04:33:51.174452+0000","last_became_peered":"2024-05-09T04:33:51.174452+0000","last_unstale":"2024-05-09T04:33:53.685499+0000","last_undegraded":"2024-05-09T04:33:53.685499+0000","last_fullsized":"2024-05-09T04:33:53.685499+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:26:18.474267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.196968+0000","last_change":"2024-05-09T04:33:51.169721+0000","last_active":"2024-05-09T04:33:53.196968+0000","last_peered":"2024-05-09T04:33:53.196968+0000","last_clean":"2024-05-09T04:33:53.196968+0000","last_became_active":"2024-05-09T04:33:51.169422+0000","last_became_peered":"2024-05-09T04:33:51.169422+0000","last_unstale":"2024-05-09T04:33:53.196968+0000","last_undegraded":"2024-05-09T04:33:53.196968+0000","last_fullsized":"2024-05-09T04:33:53.196968+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:36:52.136732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.196263+0000","last_change":"2024-05-09T04:33:52.208855+0000","last_active":"2024-05-09T04:33:53.196263+0000","last_peered":"2024-05-09T04:33:53.196263+0000","last_clean":"2024-05-09T04:33:53.196263+0000","last_became_active":"2024-05-09T04:33:52.208588+0000","last_became_peered":"2024-05-09T04:33:52.208588+0000","last_unstale":"2024-05-09T04:33:53.196263+0000","last_undegraded":"2024-05-09T04:33:53.196263+0000","last_fullsized":"2024-05-09T04:33:53.196263+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:53:58.795910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.211333+0000","last_change":"2024-05-09T04:33:51.178180+0000","last_active":"2024-05-09T04:33:54.211333+0000","last_peered":"2024-05-09T04:33:54.211333+0000","last_clean":"2024-05-09T04:33:54.211333+0000","last_became_active":"2024-05-09T04:33:51.175516+0000","last_became_peered":"2024-05-09T04:33:51.175516+0000","last_unstale":"2024-05-09T04:33:54.211333+0000","last_undegraded":"2024-05-09T04:33:54.211333+0000","last_fullsized":"2024-05-09T04:33:54.211333+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:26:18.474267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.154507+0000","last_change":"2024-05-09T04:33:52.196488+0000","last_active":"2024-05-09T04:33:54.154507+0000","last_peered":"2024-05-09T04:33:54.154507+0000","last_clean":"2024-05-09T04:33:54.154507+0000","last_became_active":"2024-05-09T04:33:52.196175+0000","last_became_peered":"2024-05-09T04:33:52.196175+0000","last_unstale":"2024-05-09T04:33:54.154507+0000","last_undegraded":"2024-05-09T04:33:54.154507+0000","last_fullsized":"2024-05-09T04:33:54.154507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:56.927838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.686280+0000","last_change":"2024-05-09T04:33:51.185310+0000","last_active":"2024-05-09T04:33:53.686280+0000","last_peered":"2024-05-09T04:33:53.686280+0000","last_clean":"2024-05-09T04:33:53.686280+0000","last_became_active":"2024-05-09T04:33:51.185016+0000","last_became_peered":"2024-05-09T04:33:51.185016+0000","last_unstale":"2024-05-09T04:33:53.686280+0000","last_undegraded":"2024-05-09T04:33:53.686280+0000","last_fullsized":"2024-05-09T04:33:53.686280+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:50:51.015799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.212510+0000","last_change":"2024-05-09T04:33:52.193316+0000","last_active":"2024-05-09T04:33:54.212510+0000","last_peered":"2024-05-09T04:33:54.212510+0000","last_clean":"2024-05-09T04:33:54.212510+0000","last_became_active":"2024-05-09T04:33:52.192939+0000","last_became_peered":"2024-05-09T04:33:52.192939+0000","last_unstale":"2024-05-09T04:33:54.212510+0000","last_undegraded":"2024-05-09T04:33:54.212510+0000","last_fullsized":"2024-05-09T04:33:54.212510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:46:44.064608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.212393+0000","last_change":"2024-05-09T04:33:51.183116+0000","last_active":"2024-05-09T04:33:54.212393+0000","last_peered":"2024-05-09T04:33:54.212393+0000","last_clean":"2024-05-09T04:33:54.212393+0000","last_became_active":"2024-05-09T04:33:51.182750+0000","last_became_peered":"2024-05-09T04:33:51.182750+0000","last_unstale":"2024-05-09T04:33:54.212393+0000","last_undegraded":"2024-05-09T04:33:54.212393+0000","last_fullsized":"2024-05-09T04:33:54.212393+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:04:10.949412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.195562+0000","last_change":"2024-05-09T04:33:52.201494+0000","last_active":"2024-05-09T04:33:53.195562+0000","last_peered":"2024-05-09T04:33:53.195562+0000","last_clean":"2024-05-09T04:33:53.195562+0000","last_became_active":"2024-05-09T04:33:52.197939+0000","last_became_peered":"2024-05-09T04:33:52.197939+0000","last_unstale":"2024-05-09T04:33:53.195562+0000","last_undegraded":"2024-05-09T04:33:53.195562+0000","last_fullsized":"2024-05-09T04:33:53.195562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:29:08.279802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.154034+0000","last_change":"2024-05-09T04:33:51.178564+0000","last_active":"2024-05-09T04:33:54.154034+0000","last_peered":"2024-05-09T04:33:54.154034+0000","last_clean":"2024-05-09T04:33:54.154034+0000","last_became_active":"2024-05-09T04:33:51.177732+0000","last_became_peered":"2024-05-09T04:33:51.177732+0000","last_unstale":"2024-05-09T04:33:54.154034+0000","last_undegraded":"2024-05-09T04:33:54.154034+0000","last_fullsized":"2024-05-09T04:33:54.154034+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:50:51.015799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.196170+0000","last_change":"2024-05-09T04:33:52.200019+0000","last_active":"2024-05-09T04:33:53.196170+0000","last_peered":"2024-05-09T04:33:53.196170+0000","last_clean":"2024-05-09T04:33:53.196170+0000","last_became_active":"2024-05-09T04:33:52.199279+0000","last_became_peered":"2024-05-09T04:33:52.199279+0000","last_unstale":"2024-05-09T04:33:53.196170+0000","last_undegraded":"2024-05-09T04:33:53.196170+0000","last_fullsized":"2024-05-09T04:33:53.196170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:19:44.181447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.685981+0000","last_change":"2024-05-09T04:33:51.173622+0000","last_active":"2024-05-09T04:33:53.685981+0000","last_peered":"2024-05-09T04:33:53.685981+0000","last_clean":"2024-05-09T04:33:53.685981+0000","last_became_active":"2024-05-09T04:33:51.173158+0000","last_became_peered":"2024-05-09T04:33:51.173158+0000","last_unstale":"2024-05-09T04:33:53.685981+0000","last_undegraded":"2024-05-09T04:33:53.685981+0000","last_fullsized":"2024-05-09T04:33:53.685981+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:08:08.955016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T04:33:52.217209+0000","last_change":"2024-05-09T04:33:52.214221+0000","last_active":"2024-05-09T04:33:52.217209+0000","last_peered":"2024-05-09T04:33:52.217209+0000","last_clean":"2024-05-09T04:33:52.217209+0000","last_became_active":"2024-05-09T04:33:52.213773+0000","last_became_peered":"2024-05-09T04:33:52.213773+0000","last_unstale":"2024-05-09T04:33:52.217209+0000","last_undegraded":"2024-05-09T04:33:52.217209+0000","last_fullsized":"2024-05-09T04:33:52.217209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:24:37.669022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.194274+0000","last_change":"2024-05-09T04:33:51.172542+0000","last_active":"2024-05-09T04:33:53.194274+0000","last_peered":"2024-05-09T04:33:53.194274+0000","last_clean":"2024-05-09T04:33:53.194274+0000","last_became_active":"2024-05-09T04:33:51.172232+0000","last_became_peered":"2024-05-09T04:33:51.172232+0000","last_unstale":"2024-05-09T04:33:53.194274+0000","last_undegraded":"2024-05-09T04:33:53.194274+0000","last_fullsized":"2024-05-09T04:33:53.194274+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:15.760572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.194183+0000","last_change":"2024-05-09T04:33:52.195027+0000","last_active":"2024-05-09T04:33:53.194183+0000","last_peered":"2024-05-09T04:33:53.194183+0000","last_clean":"2024-05-09T04:33:53.194183+0000","last_became_active":"2024-05-09T04:33:52.194590+0000","last_became_peered":"2024-05-09T04:33:52.194590+0000","last_unstale":"2024-05-09T04:33:53.194183+0000","last_undegraded":"2024-05-09T04:33:53.194183+0000","last_fullsized":"2024-05-09T04:33:53.194183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:34:40.257159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.211870+0000","last_change":"2024-05-09T04:33:51.177887+0000","last_active":"2024-05-09T04:33:54.211870+0000","last_peered":"2024-05-09T04:33:54.211870+0000","last_clean":"2024-05-09T04:33:54.211870+0000","last_became_active":"2024-05-09T04:33:51.175721+0000","last_became_peered":"2024-05-09T04:33:51.175721+0000","last_unstale":"2024-05-09T04:33:54.211870+0000","last_undegraded":"2024-05-09T04:33:54.211870+0000","last_fullsized":"2024-05-09T04:33:54.211870+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:02:50.965016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T04:33:52.217184+0000","last_change":"2024-05-09T04:33:52.211924+0000","last_active":"2024-05-09T04:33:52.217184+0000","last_peered":"2024-05-09T04:33:52.217184+0000","last_clean":"2024-05-09T04:33:52.217184+0000","last_became_active":"2024-05-09T04:33:52.211546+0000","last_became_peered":"2024-05-09T04:33:52.211546+0000","last_unstale":"2024-05-09T04:33:52.217184+0000","last_undegraded":"2024-05-09T04:33:52.217184+0000","last_fullsized":"2024-05-09T04:33:52.217184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:17.801441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.685619+0000","last_change":"2024-05-09T04:33:51.176697+0000","last_active":"2024-05-09T04:33:53.685619+0000","last_peered":"2024-05-09T04:33:53.685619+0000","last_clean":"2024-05-09T04:33:53.685619+0000","last_became_active":"2024-05-09T04:33:51.176351+0000","last_became_peered":"2024-05-09T04:33:51.176351+0000","last_unstale":"2024-05-09T04:33:53.685619+0000","last_undegraded":"2024-05-09T04:33:53.685619+0000","last_fullsized":"2024-05-09T04:33:53.685619+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:04:10.949412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.685570+0000","last_change":"2024-05-09T04:33:52.187596+0000","last_active":"2024-05-09T04:33:53.685570+0000","last_peered":"2024-05-09T04:33:53.685570+0000","last_clean":"2024-05-09T04:33:53.685570+0000","last_became_active":"2024-05-09T04:33:52.187287+0000","last_became_peered":"2024-05-09T04:33:52.187287+0000","last_unstale":"2024-05-09T04:33:53.685570+0000","last_undegraded":"2024-05-09T04:33:53.685570+0000","last_fullsized":"2024-05-09T04:33:53.685570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:34:40.257159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.732380+0000","last_change":"2024-05-09T04:33:51.166963+0000","last_active":"2024-05-09T04:33:54.732380+0000","last_peered":"2024-05-09T04:33:54.732380+0000","last_clean":"2024-05-09T04:33:54.732380+0000","last_became_active":"2024-05-09T04:33:51.165585+0000","last_became_peered":"2024-05-09T04:33:51.165585+0000","last_unstale":"2024-05-09T04:33:54.732380+0000","last_undegraded":"2024-05-09T04:33:54.732380+0000","last_fullsized":"2024-05-09T04:33:54.732380+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:04:10.949412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T04:33:52.215096+0000","last_change":"2024-05-09T04:33:52.207666+0000","last_active":"2024-05-09T04:33:52.215096+0000","last_peered":"2024-05-09T04:33:52.215096+0000","last_clean":"2024-05-09T04:33:52.215096+0000","last_became_active":"2024-05-09T04:33:52.207429+0000","last_became_peered":"2024-05-09T04:33:52.207429+0000","last_unstale":"2024-05-09T04:33:52.215096+0000","last_undegraded":"2024-05-09T04:33:52.215096+0000","last_fullsized":"2024-05-09T04:33:52.215096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:29:08.279802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.196757+0000","last_change":"2024-05-09T04:33:51.172081+0000","last_active":"2024-05-09T04:33:53.196757+0000","last_peered":"2024-05-09T04:33:53.196757+0000","last_clean":"2024-05-09T04:33:53.196757+0000","last_became_active":"2024-05-09T04:33:51.171700+0000","last_became_peered":"2024-05-09T04:33:51.171700+0000","last_unstale":"2024-05-09T04:33:53.196757+0000","last_undegraded":"2024-05-09T04:33:53.196757+0000","last_fullsized":"2024-05-09T04:33:53.196757+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:15.760572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T04:33:52.215044+0000","last_change":"2024-05-09T04:33:52.206889+0000","last_active":"2024-05-09T04:33:52.215044+0000","last_peered":"2024-05-09T04:33:52.215044+0000","last_clean":"2024-05-09T04:33:52.215044+0000","last_became_active":"2024-05-09T04:33:52.206452+0000","last_became_peered":"2024-05-09T04:33:52.206452+0000","last_unstale":"2024-05-09T04:33:52.215044+0000","last_undegraded":"2024-05-09T04:33:52.215044+0000","last_fullsized":"2024-05-09T04:33:52.215044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:34:29.852986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.732880+0000","last_change":"2024-05-09T04:33:51.167434+0000","last_active":"2024-05-09T04:33:54.732880+0000","last_peered":"2024-05-09T04:33:54.732880+0000","last_clean":"2024-05-09T04:33:54.732880+0000","last_became_active":"2024-05-09T04:33:51.167117+0000","last_became_peered":"2024-05-09T04:33:51.167117+0000","last_unstale":"2024-05-09T04:33:54.732880+0000","last_undegraded":"2024-05-09T04:33:54.732880+0000","last_fullsized":"2024-05-09T04:33:54.732880+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:02:50.965016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T04:33:52.215060+0000","last_change":"2024-05-09T04:33:52.207191+0000","last_active":"2024-05-09T04:33:52.215060+0000","last_peered":"2024-05-09T04:33:52.215060+0000","last_clean":"2024-05-09T04:33:52.215060+0000","last_became_active":"2024-05-09T04:33:52.206932+0000","last_became_peered":"2024-05-09T04:33:52.206932+0000","last_unstale":"2024-05-09T04:33:52.215060+0000","last_undegraded":"2024-05-09T04:33:52.215060+0000","last_fullsized":"2024-05-09T04:33:52.215060+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:59:28.002585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.154386+0000","last_change":"2024-05-09T04:33:51.163656+0000","last_active":"2024-05-09T04:33:54.154386+0000","last_peered":"2024-05-09T04:33:54.154386+0000","last_clean":"2024-05-09T04:33:54.154386+0000","last_became_active":"2024-05-09T04:33:51.163342+0000","last_became_peered":"2024-05-09T04:33:51.163342+0000","last_unstale":"2024-05-09T04:33:54.154386+0000","last_undegraded":"2024-05-09T04:33:54.154386+0000","last_fullsized":"2024-05-09T04:33:54.154386+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:19:47.722161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.212177+0000","last_change":"2024-05-09T04:33:52.205802+0000","last_active":"2024-05-09T04:33:54.212177+0000","last_peered":"2024-05-09T04:33:54.212177+0000","last_clean":"2024-05-09T04:33:54.212177+0000","last_became_active":"2024-05-09T04:33:52.205470+0000","last_became_peered":"2024-05-09T04:33:52.205470+0000","last_unstale":"2024-05-09T04:33:54.212177+0000","last_undegraded":"2024-05-09T04:33:54.212177+0000","last_fullsized":"2024-05-09T04:33:54.212177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:40:40.749511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.732648+0000","last_change":"2024-05-09T04:33:51.180702+0000","last_active":"2024-05-09T04:33:54.732648+0000","last_peered":"2024-05-09T04:33:54.732648+0000","last_clean":"2024-05-09T04:33:54.732648+0000","last_became_active":"2024-05-09T04:33:51.180322+0000","last_became_peered":"2024-05-09T04:33:51.180322+0000","last_unstale":"2024-05-09T04:33:54.732648+0000","last_undegraded":"2024-05-09T04:33:54.732648+0000","last_fullsized":"2024-05-09T04:33:54.732648+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:26:18.474267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T04:33:52.213552+0000","last_change":"2024-05-09T04:33:52.204685+0000","last_active":"2024-05-09T04:33:52.213552+0000","last_peered":"2024-05-09T04:33:52.213552+0000","last_clean":"2024-05-09T04:33:52.213552+0000","last_became_active":"2024-05-09T04:33:52.204353+0000","last_became_peered":"2024-05-09T04:33:52.204353+0000","last_unstale":"2024-05-09T04:33:52.213552+0000","last_undegraded":"2024-05-09T04:33:52.213552+0000","last_fullsized":"2024-05-09T04:33:52.213552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:53:58.795910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":108,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T04:33:52.176553+0000","last_change":"2024-05-09T04:33:41.092063+0000","last_active":"2024-05-09T04:33:52.176553+0000","last_peered":"2024-05-09T04:33:52.176553+0000","last_clean":"2024-05-09T04:33:52.176553+0000","last_became_active":"2024-05-09T04:33:41.091194+0000","last_became_peered":"2024-05-09T04:33:41.091194+0000","last_unstale":"2024-05-09T04:33:52.176553+0000","last_undegraded":"2024-05-09T04:33:52.176553+0000","last_fullsized":"2024-05-09T04:33:52.176553+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:40.045339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:40.045339+0000","last_clean_scrub_stamp":"2024-05-09T04:33:40.045339+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:04.908456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.211276+0000","last_change":"2024-05-09T04:33:51.179637+0000","last_active":"2024-05-09T04:33:54.211276+0000","last_peered":"2024-05-09T04:33:54.211276+0000","last_clean":"2024-05-09T04:33:54.211276+0000","last_became_active":"2024-05-09T04:33:51.179311+0000","last_became_peered":"2024-05-09T04:33:51.179311+0000","last_unstale":"2024-05-09T04:33:54.211276+0000","last_undegraded":"2024-05-09T04:33:54.211276+0000","last_fullsized":"2024-05-09T04:33:54.211276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:08:08.955016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.196827+0000","last_change":"2024-05-09T04:33:52.203706+0000","last_active":"2024-05-09T04:33:53.196827+0000","last_peered":"2024-05-09T04:33:53.196827+0000","last_clean":"2024-05-09T04:33:53.196827+0000","last_became_active":"2024-05-09T04:33:52.200053+0000","last_became_peered":"2024-05-09T04:33:52.200053+0000","last_unstale":"2024-05-09T04:33:53.196827+0000","last_undegraded":"2024-05-09T04:33:53.196827+0000","last_fullsized":"2024-05-09T04:33:53.196827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:29:08.279802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T04:33:52.176597+0000","last_change":"2024-05-09T04:33:51.185731+0000","last_active":"2024-05-09T04:33:52.176597+0000","last_peered":"2024-05-09T04:33:52.176597+0000","last_clean":"2024-05-09T04:33:52.176597+0000","last_became_active":"2024-05-09T04:33:51.184982+0000","last_became_peered":"2024-05-09T04:33:51.184982+0000","last_unstale":"2024-05-09T04:33:52.176597+0000","last_undegraded":"2024-05-09T04:33:52.176597+0000","last_fullsized":"2024-05-09T04:33:52.176597+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:29:11.409173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.193579+0000","last_change":"2024-05-09T04:33:52.208347+0000","last_active":"2024-05-09T04:33:53.193579+0000","last_peered":"2024-05-09T04:33:53.193579+0000","last_clean":"2024-05-09T04:33:53.193579+0000","last_became_active":"2024-05-09T04:33:52.206631+0000","last_became_peered":"2024-05-09T04:33:52.206631+0000","last_unstale":"2024-05-09T04:33:53.193579+0000","last_undegraded":"2024-05-09T04:33:53.193579+0000","last_fullsized":"2024-05-09T04:33:53.193579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:00:54.458043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.732806+0000","last_change":"2024-05-09T04:33:51.167013+0000","last_active":"2024-05-09T04:33:54.732806+0000","last_peered":"2024-05-09T04:33:54.732806+0000","last_clean":"2024-05-09T04:33:54.732806+0000","last_became_active":"2024-05-09T04:33:51.165450+0000","last_became_peered":"2024-05-09T04:33:51.165450+0000","last_unstale":"2024-05-09T04:33:54.732806+0000","last_undegraded":"2024-05-09T04:33:54.732806+0000","last_fullsized":"2024-05-09T04:33:54.732806+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:08:08.955016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.154238+0000","last_change":"2024-05-09T04:33:52.190418+0000","last_active":"2024-05-09T04:33:54.154238+0000","last_peered":"2024-05-09T04:33:54.154238+0000","last_clean":"2024-05-09T04:33:54.154238+0000","last_became_active":"2024-05-09T04:33:52.190069+0000","last_became_peered":"2024-05-09T04:33:52.190069+0000","last_unstale":"2024-05-09T04:33:54.154238+0000","last_undegraded":"2024-05-09T04:33:54.154238+0000","last_fullsized":"2024-05-09T04:33:54.154238+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:29:08.279802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.212222+0000","last_change":"2024-05-09T04:33:51.181463+0000","last_active":"2024-05-09T04:33:54.212222+0000","last_peered":"2024-05-09T04:33:54.212222+0000","last_clean":"2024-05-09T04:33:54.212222+0000","last_became_active":"2024-05-09T04:33:51.181213+0000","last_became_peered":"2024-05-09T04:33:51.181213+0000","last_unstale":"2024-05-09T04:33:54.212222+0000","last_undegraded":"2024-05-09T04:33:54.212222+0000","last_fullsized":"2024-05-09T04:33:54.212222+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:50:51.015799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.195331+0000","last_change":"2024-05-09T04:33:52.188609+0000","last_active":"2024-05-09T04:33:53.195331+0000","last_peered":"2024-05-09T04:33:53.195331+0000","last_clean":"2024-05-09T04:33:53.195331+0000","last_became_active":"2024-05-09T04:33:52.188379+0000","last_became_peered":"2024-05-09T04:33:52.188379+0000","last_unstale":"2024-05-09T04:33:53.195331+0000","last_undegraded":"2024-05-09T04:33:53.195331+0000","last_fullsized":"2024-05-09T04:33:53.195331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:19:48.733290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.153924+0000","last_change":"2024-05-09T04:33:51.164123+0000","last_active":"2024-05-09T04:33:54.153924+0000","last_peered":"2024-05-09T04:33:54.153924+0000","last_clean":"2024-05-09T04:33:54.153924+0000","last_became_active":"2024-05-09T04:33:51.163686+0000","last_became_peered":"2024-05-09T04:33:51.163686+0000","last_unstale":"2024-05-09T04:33:54.153924+0000","last_undegraded":"2024-05-09T04:33:54.153924+0000","last_fullsized":"2024-05-09T04:33:54.153924+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:29:11.409173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.212248+0000","last_change":"2024-05-09T04:33:52.209697+0000","last_active":"2024-05-09T04:33:54.212248+0000","last_peered":"2024-05-09T04:33:54.212248+0000","last_clean":"2024-05-09T04:33:54.212248+0000","last_became_active":"2024-05-09T04:33:52.209347+0000","last_became_peered":"2024-05-09T04:33:52.209347+0000","last_unstale":"2024-05-09T04:33:54.212248+0000","last_undegraded":"2024-05-09T04:33:54.212248+0000","last_fullsized":"2024-05-09T04:33:54.212248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:19:08.318788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T04:33:52.179499+0000","last_change":"2024-05-09T04:33:51.184584+0000","last_active":"2024-05-09T04:33:52.179499+0000","last_peered":"2024-05-09T04:33:52.179499+0000","last_clean":"2024-05-09T04:33:52.179499+0000","last_became_active":"2024-05-09T04:33:51.184369+0000","last_became_peered":"2024-05-09T04:33:51.184369+0000","last_unstale":"2024-05-09T04:33:52.179499+0000","last_undegraded":"2024-05-09T04:33:52.179499+0000","last_fullsized":"2024-05-09T04:33:52.179499+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:22:40.919482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.154095+0000","last_change":"2024-05-09T04:33:52.189698+0000","last_active":"2024-05-09T04:33:54.154095+0000","last_peered":"2024-05-09T04:33:54.154095+0000","last_clean":"2024-05-09T04:33:54.154095+0000","last_became_active":"2024-05-09T04:33:52.189388+0000","last_became_peered":"2024-05-09T04:33:52.189388+0000","last_unstale":"2024-05-09T04:33:54.154095+0000","last_undegraded":"2024-05-09T04:33:54.154095+0000","last_fullsized":"2024-05-09T04:33:54.154095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:59:28.002585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.179780+0000","last_change":"2024-05-09T04:33:51.173756+0000","last_active":"2024-05-09T04:33:54.179780+0000","last_peered":"2024-05-09T04:33:54.179780+0000","last_clean":"2024-05-09T04:33:54.179780+0000","last_became_active":"2024-05-09T04:33:51.173416+0000","last_became_peered":"2024-05-09T04:33:51.173416+0000","last_unstale":"2024-05-09T04:33:54.179780+0000","last_undegraded":"2024-05-09T04:33:54.179780+0000","last_fullsized":"2024-05-09T04:33:54.179780+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:02:50.965016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.197927+0000","last_change":"2024-05-09T04:33:52.205567+0000","last_active":"2024-05-09T04:33:53.197927+0000","last_peered":"2024-05-09T04:33:53.197927+0000","last_clean":"2024-05-09T04:33:53.197927+0000","last_became_active":"2024-05-09T04:33:52.205252+0000","last_became_peered":"2024-05-09T04:33:52.205252+0000","last_unstale":"2024-05-09T04:33:53.197927+0000","last_undegraded":"2024-05-09T04:33:53.197927+0000","last_fullsized":"2024-05-09T04:33:53.197927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:13:19.513750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.686413+0000","last_change":"2024-05-09T04:33:51.181117+0000","last_active":"2024-05-09T04:33:53.686413+0000","last_peered":"2024-05-09T04:33:53.686413+0000","last_clean":"2024-05-09T04:33:53.686413+0000","last_became_active":"2024-05-09T04:33:51.179062+0000","last_became_peered":"2024-05-09T04:33:51.179062+0000","last_unstale":"2024-05-09T04:33:53.686413+0000","last_undegraded":"2024-05-09T04:33:53.686413+0000","last_fullsized":"2024-05-09T04:33:53.686413+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:19:47.722161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.195642+0000","last_change":"2024-05-09T04:33:52.181414+0000","last_active":"2024-05-09T04:33:53.195642+0000","last_peered":"2024-05-09T04:33:53.195642+0000","last_clean":"2024-05-09T04:33:53.195642+0000","last_became_active":"2024-05-09T04:33:52.180058+0000","last_became_peered":"2024-05-09T04:33:52.180058+0000","last_unstale":"2024-05-09T04:33:53.195642+0000","last_undegraded":"2024-05-09T04:33:53.195642+0000","last_fullsized":"2024-05-09T04:33:53.195642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:17.801441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T04:33:52.176491+0000","last_change":"2024-05-09T04:33:51.183265+0000","last_active":"2024-05-09T04:33:52.176491+0000","last_peered":"2024-05-09T04:33:52.176491+0000","last_clean":"2024-05-09T04:33:52.176491+0000","last_became_active":"2024-05-09T04:33:51.181781+0000","last_became_peered":"2024-05-09T04:33:51.181781+0000","last_unstale":"2024-05-09T04:33:52.176491+0000","last_undegraded":"2024-05-09T04:33:52.176491+0000","last_fullsized":"2024-05-09T04:33:52.176491+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:19:47.722161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.194563+0000","last_change":"2024-05-09T04:33:52.203247+0000","last_active":"2024-05-09T04:33:53.194563+0000","last_peered":"2024-05-09T04:33:53.194563+0000","last_clean":"2024-05-09T04:33:53.194563+0000","last_became_active":"2024-05-09T04:33:52.203006+0000","last_became_peered":"2024-05-09T04:33:52.203006+0000","last_unstale":"2024-05-09T04:33:53.194563+0000","last_undegraded":"2024-05-09T04:33:53.194563+0000","last_fullsized":"2024-05-09T04:33:53.194563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:02:51.976144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.154676+0000","last_change":"2024-05-09T04:33:51.172718+0000","last_active":"2024-05-09T04:33:54.154676+0000","last_peered":"2024-05-09T04:33:54.154676+0000","last_clean":"2024-05-09T04:33:54.154676+0000","last_became_active":"2024-05-09T04:33:51.172131+0000","last_became_peered":"2024-05-09T04:33:51.172131+0000","last_unstale":"2024-05-09T04:33:54.154676+0000","last_undegraded":"2024-05-09T04:33:54.154676+0000","last_fullsized":"2024-05-09T04:33:54.154676+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:08:08.955016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.196487+0000","last_change":"2024-05-09T04:33:52.201413+0000","last_active":"2024-05-09T04:33:53.196487+0000","last_peered":"2024-05-09T04:33:53.196487+0000","last_clean":"2024-05-09T04:33:53.196487+0000","last_became_active":"2024-05-09T04:33:52.199032+0000","last_became_peered":"2024-05-09T04:33:52.199032+0000","last_unstale":"2024-05-09T04:33:53.196487+0000","last_undegraded":"2024-05-09T04:33:53.196487+0000","last_fullsized":"2024-05-09T04:33:53.196487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:34:40.257159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.154633+0000","last_change":"2024-05-09T04:33:51.159715+0000","last_active":"2024-05-09T04:33:54.154633+0000","last_peered":"2024-05-09T04:33:54.154633+0000","last_clean":"2024-05-09T04:33:54.154633+0000","last_became_active":"2024-05-09T04:33:51.159395+0000","last_became_peered":"2024-05-09T04:33:51.159395+0000","last_unstale":"2024-05-09T04:33:54.154633+0000","last_undegraded":"2024-05-09T04:33:54.154633+0000","last_fullsized":"2024-05-09T04:33:54.154633+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:19:43.170318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.197360+0000","last_change":"2024-05-09T04:33:52.198684+0000","last_active":"2024-05-09T04:33:53.197360+0000","last_peered":"2024-05-09T04:33:53.197360+0000","last_clean":"2024-05-09T04:33:53.197360+0000","last_became_active":"2024-05-09T04:33:52.196451+0000","last_became_peered":"2024-05-09T04:33:52.196451+0000","last_unstale":"2024-05-09T04:33:53.197360+0000","last_undegraded":"2024-05-09T04:33:53.197360+0000","last_fullsized":"2024-05-09T04:33:53.197360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:17.801441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T04:33:52.218009+0000","last_change":"2024-05-09T04:33:51.186050+0000","last_active":"2024-05-09T04:33:52.218009+0000","last_peered":"2024-05-09T04:33:52.218009+0000","last_clean":"2024-05-09T04:33:52.218009+0000","last_became_active":"2024-05-09T04:33:51.185720+0000","last_became_peered":"2024-05-09T04:33:51.185720+0000","last_unstale":"2024-05-09T04:33:52.218009+0000","last_undegraded":"2024-05-09T04:33:52.218009+0000","last_fullsized":"2024-05-09T04:33:52.218009+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:11:05.364073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.685574+0000","last_change":"2024-05-09T04:33:52.191123+0000","last_active":"2024-05-09T04:33:53.685574+0000","last_peered":"2024-05-09T04:33:53.685574+0000","last_clean":"2024-05-09T04:33:53.685574+0000","last_became_active":"2024-05-09T04:33:52.189005+0000","last_became_peered":"2024-05-09T04:33:52.189005+0000","last_unstale":"2024-05-09T04:33:53.685574+0000","last_undegraded":"2024-05-09T04:33:53.685574+0000","last_fullsized":"2024-05-09T04:33:53.685574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:34:29.852986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.212085+0000","last_change":"2024-05-09T04:33:52.208515+0000","last_active":"2024-05-09T04:33:54.212085+0000","last_peered":"2024-05-09T04:33:54.212085+0000","last_clean":"2024-05-09T04:33:54.212085+0000","last_became_active":"2024-05-09T04:33:52.208236+0000","last_became_peered":"2024-05-09T04:33:52.208236+0000","last_unstale":"2024-05-09T04:33:54.212085+0000","last_undegraded":"2024-05-09T04:33:54.212085+0000","last_fullsized":"2024-05-09T04:33:54.212085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:19:13.480834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T04:33:52.215642+0000","last_change":"2024-05-09T04:33:51.187919+0000","last_active":"2024-05-09T04:33:52.215642+0000","last_peered":"2024-05-09T04:33:52.215642+0000","last_clean":"2024-05-09T04:33:52.215642+0000","last_became_active":"2024-05-09T04:33:51.187590+0000","last_became_peered":"2024-05-09T04:33:51.187590+0000","last_unstale":"2024-05-09T04:33:52.215642+0000","last_undegraded":"2024-05-09T04:33:52.215642+0000","last_fullsized":"2024-05-09T04:33:52.215642+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:50:51.015799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.211737+0000","last_change":"2024-05-09T04:33:52.209870+0000","last_active":"2024-05-09T04:33:54.211737+0000","last_peered":"2024-05-09T04:33:54.211737+0000","last_clean":"2024-05-09T04:33:54.211737+0000","last_became_active":"2024-05-09T04:33:52.209199+0000","last_became_peered":"2024-05-09T04:33:52.209199+0000","last_unstale":"2024-05-09T04:33:54.211737+0000","last_undegraded":"2024-05-09T04:33:54.211737+0000","last_fullsized":"2024-05-09T04:33:54.211737+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:10:42.676456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T04:33:52.211755+0000","last_change":"2024-05-09T04:33:51.187870+0000","last_active":"2024-05-09T04:33:52.211755+0000","last_peered":"2024-05-09T04:33:52.211755+0000","last_clean":"2024-05-09T04:33:52.211755+0000","last_became_active":"2024-05-09T04:33:51.187575+0000","last_became_peered":"2024-05-09T04:33:51.187575+0000","last_unstale":"2024-05-09T04:33:52.211755+0000","last_undegraded":"2024-05-09T04:33:52.211755+0000","last_fullsized":"2024-05-09T04:33:52.211755+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:19:43.170318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.195161+0000","last_change":"2024-05-09T04:33:52.201406+0000","last_active":"2024-05-09T04:33:53.195161+0000","last_peered":"2024-05-09T04:33:53.195161+0000","last_clean":"2024-05-09T04:33:53.195161+0000","last_became_active":"2024-05-09T04:33:52.200809+0000","last_became_peered":"2024-05-09T04:33:52.200809+0000","last_unstale":"2024-05-09T04:33:53.195161+0000","last_undegraded":"2024-05-09T04:33:53.195161+0000","last_fullsized":"2024-05-09T04:33:53.195161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:59:28.002585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.197985+0000","last_change":"2024-05-09T04:33:51.168105+0000","last_active":"2024-05-09T04:33:53.197985+0000","last_peered":"2024-05-09T04:33:53.197985+0000","last_clean":"2024-05-09T04:33:53.197985+0000","last_became_active":"2024-05-09T04:33:51.167790+0000","last_became_peered":"2024-05-09T04:33:51.167790+0000","last_unstale":"2024-05-09T04:33:53.197985+0000","last_undegraded":"2024-05-09T04:33:53.197985+0000","last_fullsized":"2024-05-09T04:33:53.197985+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:08:08.955016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.197883+0000","last_change":"2024-05-09T04:33:52.205526+0000","last_active":"2024-05-09T04:33:53.197883+0000","last_peered":"2024-05-09T04:33:53.197883+0000","last_clean":"2024-05-09T04:33:53.197883+0000","last_became_active":"2024-05-09T04:33:52.205231+0000","last_became_peered":"2024-05-09T04:33:52.205231+0000","last_unstale":"2024-05-09T04:33:53.197883+0000","last_undegraded":"2024-05-09T04:33:53.197883+0000","last_fullsized":"2024-05-09T04:33:53.197883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:02:17.813339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.195818+0000","last_change":"2024-05-09T04:33:51.176461+0000","last_active":"2024-05-09T04:33:53.195818+0000","last_peered":"2024-05-09T04:33:53.195818+0000","last_clean":"2024-05-09T04:33:53.195818+0000","last_became_active":"2024-05-09T04:33:51.176175+0000","last_became_peered":"2024-05-09T04:33:51.176175+0000","last_unstale":"2024-05-09T04:33:53.195818+0000","last_undegraded":"2024-05-09T04:33:53.195818+0000","last_fullsized":"2024-05-09T04:33:53.195818+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:01:20.702014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":790528,"data_stored":655584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":192512,"data_stored":64634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":200704,"data_stored":64794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T04:33:56.615 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T04:33:56.839 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.839+0000 7fd50aa00640 1 Processor -- start 2024-05-09T04:33:56.839 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.840+0000 7fd50aa00640 1 -- start start 2024-05-09T04:33:56.841 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.841+0000 7fd50aa00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fd5040d3180 0x7fd5040d5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:56.841 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.841+0000 7fd50aa00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd5040d22f0 0x7fd5040d3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:56.841 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.841+0000 7fd50aa00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5040d3e70 con 0x7fd5040b4760 2024-05-09T04:33:56.841 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.841+0000 7fd50aa00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fd5040d4150 con 0x7fd5040d3180 2024-05-09T04:33:56.842 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.841+0000 7fd50aa00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fd5040b6200 con 0x7fd5040d22f0 2024-05-09T04:33:56.842 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.841+0000 7fd509600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fd5040d3180 0x7fd5040d5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:56.842 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.841+0000 7fd50a000640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7fd5040b4760 0x7fd5040b4bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:33902/0 (socket says 172.21.15.145:33902) 2024-05-09T04:33:56.842 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.841+0000 7fd509600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fd5040d3180 0x7fd5040d5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:44968/0 (socket says 172.21.15.145:44968) 2024-05-09T04:33:56.842 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.841+0000 7fd50a000640 1 -- 172.21.15.145:0/1444906635 learned_addr learned my addr 172.21.15.145:0/1444906635 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:33:56.842 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.841+0000 7fd508c00640 1 --2- 172.21.15.145:0/1444906635 >> v2:172.21.15.192:3300/0 conn(0x7fd5040d22f0 0x7fd5040d3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:56.842 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.841+0000 7fd509600640 1 -- 172.21.15.145:0/1444906635 >> v2:172.21.15.192:3300/0 conn(0x7fd5040d22f0 msgr2=0x7fd5040d3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:56.842 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.841+0000 7fd509600640 1 --2- 172.21.15.145:0/1444906635 >> v2:172.21.15.192:3300/0 conn(0x7fd5040d22f0 0x7fd5040d3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:56.842 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.841+0000 7fd509600640 1 -- 172.21.15.145:0/1444906635 >> v1:172.21.15.145:6789/0 conn(0x7fd5040b4760 legacy=0x7fd5040b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:56.842 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.841+0000 7fd509600640 1 -- 172.21.15.145:0/1444906635 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5040b6450 con 0x7fd5040d3180 2024-05-09T04:33:56.842 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.841+0000 7fd503e00640 1 -- 172.21.15.145:0/1444906635 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2992027843 0 0) 0x7fd5040d3e70 con 0x7fd5040b4760 2024-05-09T04:33:56.842 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.842+0000 7fd508c00640 1 --2- 172.21.15.145:0/1444906635 >> v2:172.21.15.192:3300/0 conn(0x7fd5040d22f0 0x7fd5040d3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:33:56.843 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.843+0000 7fd509600640 1 --2- 172.21.15.145:0/1444906635 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd5040d3180 0x7fd5040d5ad0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd50408d980 tx=0x7fd4f802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83d239bfae54029d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:56.848 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.843+0000 7fd503e00640 1 -- 172.21.15.145:0/1444906635 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f8037070 con 0x7fd5040d3180 2024-05-09T04:33:56.848 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.843+0000 7fd503e00640 1 -- 172.21.15.145:0/1444906635 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4f8032440 con 0x7fd5040d3180 2024-05-09T04:33:56.849 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.843+0000 7fd503e00640 1 -- 172.21.15.145:0/1444906635 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4f802ae60 con 0x7fd5040d3180 2024-05-09T04:33:56.849 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.844+0000 7fd50aa00640 1 -- 172.21.15.145:0/1444906635 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd5040d3180 msgr2=0x7fd5040d5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:56.849 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.844+0000 7fd50aa00640 1 --2- 172.21.15.145:0/1444906635 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd5040d3180 0x7fd5040d5ad0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd50408d980 tx=0x7fd4f802a110 comp rx=0 tx=0).stop 2024-05-09T04:33:56.849 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.844+0000 7fd50aa00640 1 -- 172.21.15.145:0/1444906635 shutdown_connections 2024-05-09T04:33:56.849 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.844+0000 7fd50aa00640 1 --2- 172.21.15.145:0/1444906635 >> v2:172.21.15.192:3300/0 conn(0x7fd5040d22f0 0x7fd5040d3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:56.849 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.844+0000 7fd50aa00640 1 --2- 172.21.15.145:0/1444906635 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd5040d3180 0x7fd5040d5ad0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:56.849 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.844+0000 7fd50aa00640 1 -- 172.21.15.145:0/1444906635 >> 172.21.15.145:0/1444906635 conn(0x7fd5040d4fb0 msgr2=0x7fd5040d53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:56.849 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.844+0000 7fd50aa00640 1 -- 172.21.15.145:0/1444906635 shutdown_connections 2024-05-09T04:33:56.849 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.844+0000 7fd50aa00640 1 -- 172.21.15.145:0/1444906635 wait complete. 2024-05-09T04:33:56.849 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.845+0000 7fd50aa00640 1 Processor -- start 2024-05-09T04:33:56.849 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.846+0000 7fd50aa00640 1 -- start start 2024-05-09T04:33:56.849 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.846+0000 7fd50aa00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd5040b4760 0x7fd5040cd620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:56.849 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.846+0000 7fd50aa00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd5040d22f0 0x7fd50415d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:56.849 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.846+0000 7fd50aa00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd5040d3180 0x7fd504159d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:56.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.846+0000 7fd50aa00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd5040b52e0 con 0x7fd5040d3180 2024-05-09T04:33:56.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.846+0000 7fd50aa00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd5040d2780 con 0x7fd5040d22f0 2024-05-09T04:33:56.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.846+0000 7fd50aa00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd5040d3aa0 con 0x7fd5040b4760 2024-05-09T04:33:56.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.847+0000 7fd50a000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd5040d3180 0x7fd504159d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:56.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.847+0000 7fd50a000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd5040d3180 0x7fd504159d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:44974/0 (socket says 172.21.15.145:44974) 2024-05-09T04:33:56.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.847+0000 7fd50a000640 1 -- 172.21.15.145:0/2559111217 learned_addr learned my addr 172.21.15.145:0/2559111217 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:56.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.847+0000 7fd509600640 1 --2- 172.21.15.145:0/2559111217 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd5040b4760 0x7fd5040cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:56.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.847+0000 7fd508c00640 1 --2- 172.21.15.145:0/2559111217 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd5040d22f0 0x7fd50415d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:56.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.847+0000 7fd50a000640 1 -- 172.21.15.145:0/2559111217 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd5040b4760 msgr2=0x7fd5040cd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:56.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.847+0000 7fd50a000640 1 --2- 172.21.15.145:0/2559111217 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd5040b4760 0x7fd5040cd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:56.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.847+0000 7fd50a000640 1 -- 172.21.15.145:0/2559111217 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd5040d22f0 msgr2=0x7fd50415d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:56.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.847+0000 7fd50a000640 1 --2- 172.21.15.145:0/2559111217 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd5040d22f0 0x7fd50415d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:56.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.847+0000 7fd50a000640 1 -- 172.21.15.145:0/2559111217 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5040cc580 con 0x7fd5040d3180 2024-05-09T04:33:56.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.847+0000 7fd509600640 1 --2- 172.21.15.145:0/2559111217 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd5040b4760 0x7fd5040cd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:33:56.851 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.848+0000 7fd50a000640 1 --2- 172.21.15.145:0/2559111217 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd5040d3180 0x7fd504159d70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc00c7a0 tx=0x7fd4fc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:56.851 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.848+0000 7fd501600640 1 -- 172.21.15.145:0/2559111217 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc018070 con 0x7fd5040d3180 2024-05-09T04:33:56.851 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.848+0000 7fd501600640 1 -- 172.21.15.145:0/2559111217 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4fc00a5e0 con 0x7fd5040d3180 2024-05-09T04:33:56.851 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.848+0000 7fd501600640 1 -- 172.21.15.145:0/2559111217 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc00aa50 con 0x7fd5040d3180 2024-05-09T04:33:56.851 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.848+0000 7fd50aa00640 1 -- 172.21.15.145:0/2559111217 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5040cc860 con 0x7fd5040d3180 2024-05-09T04:33:56.851 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.848+0000 7fd50aa00640 1 -- 172.21.15.145:0/2559111217 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5040cce20 con 0x7fd5040d3180 2024-05-09T04:33:56.852 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.849+0000 7fd50aa00640 1 -- 172.21.15.145:0/2559111217 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd50406a310 con 0x7fd5040d3180 2024-05-09T04:33:56.852 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.850+0000 7fd501600640 1 -- 172.21.15.145:0/2559111217 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd4fc00abf0 con 0x7fd5040d3180 2024-05-09T04:33:56.852 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.851+0000 7fd501600640 1 --2- 172.21.15.145:0/2559111217 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fd4e4076300 0x7fd4e40787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:56.852 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.851+0000 7fd501600640 1 -- 172.21.15.145:0/2559111217 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd4fc096ae0 con 0x7fd5040d3180 2024-05-09T04:33:56.852 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.851+0000 7fd509600640 1 --2- 172.21.15.145:0/2559111217 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fd4e4076300 0x7fd4e40787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:56.852 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.852+0000 7fd509600640 1 --2- 172.21.15.145:0/2559111217 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fd4e4076300 0x7fd4e40787c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8034000 tx=0x7fd4f802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:56.856 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:56.856+0000 7fd501600640 1 -- 172.21.15.145:0/2559111217 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd4fc060430 con 0x7fd5040d3180 2024-05-09T04:33:56.957 INFO:tasks.ceph.mds.d.smithi192.stderr:2024-05-09T04:33:56.957+0000 7f441a800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-09T04:33:57.046 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.046+0000 7fd50aa00640 1 -- 172.21.15.145:0/2559111217 --> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd504162b50 con 0x7fd4e4076300 2024-05-09T04:33:57.066 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.066+0000 7fd501600640 1 -- 172.21.15.145:0/2559111217 <== mgr.4104 v2:172.21.15.145:6832/2725855396 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251924 (secure 0 0 0) 0x7fd504162b50 con 0x7fd4e4076300 2024-05-09T04:33:57.069 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:33:57.070 INFO:teuthology.orchestra.run.smithi145.stderr:dumped all 2024-05-09T04:33:57.072 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.072+0000 7fd50aa00640 1 -- 172.21.15.145:0/2559111217 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fd4e4076300 msgr2=0x7fd4e40787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:57.072 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.072+0000 7fd50aa00640 1 --2- 172.21.15.145:0/2559111217 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fd4e4076300 0x7fd4e40787c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8034000 tx=0x7fd4f802e000 comp rx=0 tx=0).stop 2024-05-09T04:33:57.072 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.072+0000 7fd50aa00640 1 -- 172.21.15.145:0/2559111217 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd5040d3180 msgr2=0x7fd504159d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:57.072 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.072+0000 7fd50aa00640 1 --2- 172.21.15.145:0/2559111217 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd5040d3180 0x7fd504159d70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc00c7a0 tx=0x7fd4fc00cc60 comp rx=0 tx=0).stop 2024-05-09T04:33:57.072 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.072+0000 7fd50aa00640 1 -- 172.21.15.145:0/2559111217 shutdown_connections 2024-05-09T04:33:57.072 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.072+0000 7fd50aa00640 1 --2- 172.21.15.145:0/2559111217 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fd4e4076300 0x7fd4e40787c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:57.072 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.072+0000 7fd50aa00640 1 --2- 172.21.15.145:0/2559111217 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd5040d3180 0x7fd504159d70 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:57.072 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.072+0000 7fd50aa00640 1 --2- 172.21.15.145:0/2559111217 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd5040d22f0 0x7fd50415d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:57.072 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.072+0000 7fd50aa00640 1 --2- 172.21.15.145:0/2559111217 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd5040b4760 0x7fd5040cd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:57.072 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.072+0000 7fd50aa00640 1 -- 172.21.15.145:0/2559111217 >> 172.21.15.145:0/2559111217 conn(0x7fd5040d4fb0 msgr2=0x7fd5040d5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:57.072 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.072+0000 7fd50aa00640 1 -- 172.21.15.145:0/2559111217 shutdown_connections 2024-05-09T04:33:57.072 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.072+0000 7fd50aa00640 1 -- 172.21.15.145:0/2559111217 wait complete. 2024-05-09T04:33:57.091 INFO:teuthology.orchestra.run.smithi145.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-05-09T04:33:55.757551+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":132,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218976,"kb_used_data":2788,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755744,"statfs":{"total":773094113280,"available":772869881856,"internally_reserved":0,"allocated":2854912,"data_stored":1734128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.625557"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T04:33:52.204965+0000","last_change":"2024-05-09T04:33:52.200596+0000","last_active":"2024-05-09T04:33:52.204965+0000","last_peered":"2024-05-09T04:33:52.204965+0000","last_clean":"2024-05-09T04:33:52.204965+0000","last_became_active":"2024-05-09T04:33:52.199781+0000","last_became_peered":"2024-05-09T04:33:52.199781+0000","last_unstale":"2024-05-09T04:33:52.204965+0000","last_undegraded":"2024-05-09T04:33:52.204965+0000","last_fullsized":"2024-05-09T04:33:52.204965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:33:25.042543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.154485+0000","last_change":"2024-05-09T04:33:52.186325+0000","last_active":"2024-05-09T04:33:54.154485+0000","last_peered":"2024-05-09T04:33:54.154485+0000","last_clean":"2024-05-09T04:33:54.154485+0000","last_became_active":"2024-05-09T04:33:52.186047+0000","last_became_peered":"2024-05-09T04:33:52.186047+0000","last_unstale":"2024-05-09T04:33:54.154485+0000","last_undegraded":"2024-05-09T04:33:54.154485+0000","last_fullsized":"2024-05-09T04:33:54.154485+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:24:37.669022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.194578+0000","last_change":"2024-05-09T04:33:52.187398+0000","last_active":"2024-05-09T04:33:53.194578+0000","last_peered":"2024-05-09T04:33:53.194578+0000","last_clean":"2024-05-09T04:33:53.194578+0000","last_became_active":"2024-05-09T04:33:52.186979+0000","last_became_peered":"2024-05-09T04:33:52.186979+0000","last_unstale":"2024-05-09T04:33:53.194578+0000","last_undegraded":"2024-05-09T04:33:53.194578+0000","last_fullsized":"2024-05-09T04:33:53.194578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:34:29.852986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.211667+0000","last_change":"2024-05-09T04:33:52.210098+0000","last_active":"2024-05-09T04:33:54.211667+0000","last_peered":"2024-05-09T04:33:54.211667+0000","last_clean":"2024-05-09T04:33:54.211667+0000","last_became_active":"2024-05-09T04:33:52.209575+0000","last_became_peered":"2024-05-09T04:33:52.209575+0000","last_unstale":"2024-05-09T04:33:54.211667+0000","last_undegraded":"2024-05-09T04:33:54.211667+0000","last_fullsized":"2024-05-09T04:33:54.211667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:24:41.478222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T04:33:52.202014+0000","last_change":"2024-05-09T04:33:52.195969+0000","last_active":"2024-05-09T04:33:52.202014+0000","last_peered":"2024-05-09T04:33:52.202014+0000","last_clean":"2024-05-09T04:33:52.202014+0000","last_became_active":"2024-05-09T04:33:52.195578+0000","last_became_peered":"2024-05-09T04:33:52.195578+0000","last_unstale":"2024-05-09T04:33:52.202014+0000","last_undegraded":"2024-05-09T04:33:52.202014+0000","last_fullsized":"2024-05-09T04:33:52.202014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:27:41.648285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.211122+0000","last_change":"2024-05-09T04:33:52.198619+0000","last_active":"2024-05-09T04:33:54.211122+0000","last_peered":"2024-05-09T04:33:54.211122+0000","last_clean":"2024-05-09T04:33:54.211122+0000","last_became_active":"2024-05-09T04:33:52.198345+0000","last_became_peered":"2024-05-09T04:33:52.198345+0000","last_unstale":"2024-05-09T04:33:54.211122+0000","last_undegraded":"2024-05-09T04:33:54.211122+0000","last_fullsized":"2024-05-09T04:33:54.211122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:24:37.669022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T04:33:52.207328+0000","last_change":"2024-05-09T04:33:52.202397+0000","last_active":"2024-05-09T04:33:52.207328+0000","last_peered":"2024-05-09T04:33:52.207328+0000","last_clean":"2024-05-09T04:33:52.207328+0000","last_became_active":"2024-05-09T04:33:52.202176+0000","last_became_peered":"2024-05-09T04:33:52.202176+0000","last_unstale":"2024-05-09T04:33:52.207328+0000","last_undegraded":"2024-05-09T04:33:52.207328+0000","last_fullsized":"2024-05-09T04:33:52.207328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:35:43.478085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.196047+0000","last_change":"2024-05-09T04:33:52.207259+0000","last_active":"2024-05-09T04:33:53.196047+0000","last_peered":"2024-05-09T04:33:53.196047+0000","last_clean":"2024-05-09T04:33:53.196047+0000","last_became_active":"2024-05-09T04:33:52.204206+0000","last_became_peered":"2024-05-09T04:33:52.204206+0000","last_unstale":"2024-05-09T04:33:53.196047+0000","last_undegraded":"2024-05-09T04:33:53.196047+0000","last_fullsized":"2024-05-09T04:33:53.196047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:35:43.478085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T04:33:52.213659+0000","last_change":"2024-05-09T04:33:52.205582+0000","last_active":"2024-05-09T04:33:52.213659+0000","last_peered":"2024-05-09T04:33:52.213659+0000","last_clean":"2024-05-09T04:33:52.213659+0000","last_became_active":"2024-05-09T04:33:52.205237+0000","last_became_peered":"2024-05-09T04:33:52.205237+0000","last_unstale":"2024-05-09T04:33:52.213659+0000","last_undegraded":"2024-05-09T04:33:52.213659+0000","last_fullsized":"2024-05-09T04:33:52.213659+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:34:40.257159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.686517+0000","last_change":"2024-05-09T04:33:52.198058+0000","last_active":"2024-05-09T04:33:53.686517+0000","last_peered":"2024-05-09T04:33:53.686517+0000","last_clean":"2024-05-09T04:33:53.686517+0000","last_became_active":"2024-05-09T04:33:52.197701+0000","last_became_peered":"2024-05-09T04:33:52.197701+0000","last_unstale":"2024-05-09T04:33:53.686517+0000","last_undegraded":"2024-05-09T04:33:53.686517+0000","last_fullsized":"2024-05-09T04:33:53.686517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:24:37.669022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.686063+0000","last_change":"2024-05-09T04:33:52.194755+0000","last_active":"2024-05-09T04:33:53.686063+0000","last_peered":"2024-05-09T04:33:53.686063+0000","last_clean":"2024-05-09T04:33:53.686063+0000","last_became_active":"2024-05-09T04:33:52.194270+0000","last_became_peered":"2024-05-09T04:33:52.194270+0000","last_unstale":"2024-05-09T04:33:53.686063+0000","last_undegraded":"2024-05-09T04:33:53.686063+0000","last_fullsized":"2024-05-09T04:33:53.686063+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:59:28.002585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.195766+0000","last_change":"2024-05-09T04:33:52.200331+0000","last_active":"2024-05-09T04:33:53.195766+0000","last_peered":"2024-05-09T04:33:53.195766+0000","last_clean":"2024-05-09T04:33:53.195766+0000","last_became_active":"2024-05-09T04:33:52.199739+0000","last_became_peered":"2024-05-09T04:33:52.199739+0000","last_unstale":"2024-05-09T04:33:53.195766+0000","last_undegraded":"2024-05-09T04:33:53.195766+0000","last_fullsized":"2024-05-09T04:33:53.195766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:29:12.420301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.196580+0000","last_change":"2024-05-09T04:33:52.211817+0000","last_active":"2024-05-09T04:33:53.196580+0000","last_peered":"2024-05-09T04:33:53.196580+0000","last_clean":"2024-05-09T04:33:53.196580+0000","last_became_active":"2024-05-09T04:33:52.211533+0000","last_became_peered":"2024-05-09T04:33:52.211533+0000","last_unstale":"2024-05-09T04:33:53.196580+0000","last_undegraded":"2024-05-09T04:33:53.196580+0000","last_fullsized":"2024-05-09T04:33:53.196580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:34:29.852986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.193922+0000","last_change":"2024-05-09T04:33:52.177443+0000","last_active":"2024-05-09T04:33:53.193922+0000","last_peered":"2024-05-09T04:33:53.193922+0000","last_clean":"2024-05-09T04:33:53.193922+0000","last_became_active":"2024-05-09T04:33:52.177115+0000","last_became_peered":"2024-05-09T04:33:52.177115+0000","last_unstale":"2024-05-09T04:33:53.193922+0000","last_undegraded":"2024-05-09T04:33:53.193922+0000","last_fullsized":"2024-05-09T04:33:53.193922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:33:25.042543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.154598+0000","last_change":"2024-05-09T04:33:52.209143+0000","last_active":"2024-05-09T04:33:54.154598+0000","last_peered":"2024-05-09T04:33:54.154598+0000","last_clean":"2024-05-09T04:33:54.154598+0000","last_became_active":"2024-05-09T04:33:52.206547+0000","last_became_peered":"2024-05-09T04:33:52.206547+0000","last_unstale":"2024-05-09T04:33:54.154598+0000","last_undegraded":"2024-05-09T04:33:54.154598+0000","last_fullsized":"2024-05-09T04:33:54.154598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:02:17.813339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.211033+0000","last_change":"2024-05-09T04:33:52.198040+0000","last_active":"2024-05-09T04:33:54.211033+0000","last_peered":"2024-05-09T04:33:54.211033+0000","last_clean":"2024-05-09T04:33:54.211033+0000","last_became_active":"2024-05-09T04:33:52.197706+0000","last_became_peered":"2024-05-09T04:33:52.197706+0000","last_unstale":"2024-05-09T04:33:54.211033+0000","last_undegraded":"2024-05-09T04:33:54.211033+0000","last_fullsized":"2024-05-09T04:33:54.211033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:17.801441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.212667+0000","last_change":"2024-05-09T04:33:52.195546+0000","last_active":"2024-05-09T04:33:54.212667+0000","last_peered":"2024-05-09T04:33:54.212667+0000","last_clean":"2024-05-09T04:33:54.212667+0000","last_became_active":"2024-05-09T04:33:52.195138+0000","last_became_peered":"2024-05-09T04:33:52.195138+0000","last_unstale":"2024-05-09T04:33:54.212667+0000","last_undegraded":"2024-05-09T04:33:54.212667+0000","last_fullsized":"2024-05-09T04:33:54.212667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:42.767983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.732773+0000","last_change":"2024-05-09T04:33:52.192303+0000","last_active":"2024-05-09T04:33:54.732773+0000","last_peered":"2024-05-09T04:33:54.732773+0000","last_clean":"2024-05-09T04:33:54.732773+0000","last_became_active":"2024-05-09T04:33:52.191878+0000","last_became_peered":"2024-05-09T04:33:52.191878+0000","last_unstale":"2024-05-09T04:33:54.732773+0000","last_undegraded":"2024-05-09T04:33:54.732773+0000","last_fullsized":"2024-05-09T04:33:54.732773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:22:41.930610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T04:33:52.194033+0000","last_change":"2024-05-09T04:33:52.190589+0000","last_active":"2024-05-09T04:33:52.194033+0000","last_peered":"2024-05-09T04:33:52.194033+0000","last_clean":"2024-05-09T04:33:52.194033+0000","last_became_active":"2024-05-09T04:33:52.190312+0000","last_became_peered":"2024-05-09T04:33:52.190312+0000","last_unstale":"2024-05-09T04:33:52.194033+0000","last_undegraded":"2024-05-09T04:33:52.194033+0000","last_fullsized":"2024-05-09T04:33:52.194033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:00:54.458043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.686133+0000","last_change":"2024-05-09T04:33:52.195370+0000","last_active":"2024-05-09T04:33:53.686133+0000","last_peered":"2024-05-09T04:33:53.686133+0000","last_clean":"2024-05-09T04:33:53.686133+0000","last_became_active":"2024-05-09T04:33:52.194414+0000","last_became_peered":"2024-05-09T04:33:52.194414+0000","last_unstale":"2024-05-09T04:33:53.686133+0000","last_undegraded":"2024-05-09T04:33:53.686133+0000","last_fullsized":"2024-05-09T04:33:53.686133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:29:08.279802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.193862+0000","last_change":"2024-05-09T04:33:52.201922+0000","last_active":"2024-05-09T04:33:53.193862+0000","last_peered":"2024-05-09T04:33:53.193862+0000","last_clean":"2024-05-09T04:33:53.193862+0000","last_became_active":"2024-05-09T04:33:52.201635+0000","last_became_peered":"2024-05-09T04:33:52.201635+0000","last_unstale":"2024-05-09T04:33:53.193862+0000","last_undegraded":"2024-05-09T04:33:53.193862+0000","last_fullsized":"2024-05-09T04:33:53.193862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:27:41.648285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.211533+0000","last_change":"2024-05-09T04:33:52.195480+0000","last_active":"2024-05-09T04:33:54.211533+0000","last_peered":"2024-05-09T04:33:54.211533+0000","last_clean":"2024-05-09T04:33:54.211533+0000","last_became_active":"2024-05-09T04:33:52.195195+0000","last_became_peered":"2024-05-09T04:33:52.195195+0000","last_unstale":"2024-05-09T04:33:54.211533+0000","last_undegraded":"2024-05-09T04:33:54.211533+0000","last_fullsized":"2024-05-09T04:33:54.211533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:02:17.813339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.196691+0000","last_change":"2024-05-09T04:33:52.208038+0000","last_active":"2024-05-09T04:33:53.196691+0000","last_peered":"2024-05-09T04:33:53.196691+0000","last_clean":"2024-05-09T04:33:53.196691+0000","last_became_active":"2024-05-09T04:33:52.207755+0000","last_became_peered":"2024-05-09T04:33:52.207755+0000","last_unstale":"2024-05-09T04:33:53.196691+0000","last_undegraded":"2024-05-09T04:33:53.196691+0000","last_fullsized":"2024-05-09T04:33:53.196691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:59:28.002585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.211317+0000","last_change":"2024-05-09T04:33:52.209997+0000","last_active":"2024-05-09T04:33:54.211317+0000","last_peered":"2024-05-09T04:33:54.211317+0000","last_clean":"2024-05-09T04:33:54.211317+0000","last_became_active":"2024-05-09T04:33:52.209702+0000","last_became_peered":"2024-05-09T04:33:52.209702+0000","last_unstale":"2024-05-09T04:33:54.211317+0000","last_undegraded":"2024-05-09T04:33:54.211317+0000","last_fullsized":"2024-05-09T04:33:54.211317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:56.927838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.197678+0000","last_change":"2024-05-09T04:33:52.207838+0000","last_active":"2024-05-09T04:33:53.197678+0000","last_peered":"2024-05-09T04:33:53.197678+0000","last_clean":"2024-05-09T04:33:53.197678+0000","last_became_active":"2024-05-09T04:33:52.207601+0000","last_became_peered":"2024-05-09T04:33:52.207601+0000","last_unstale":"2024-05-09T04:33:53.197678+0000","last_undegraded":"2024-05-09T04:33:53.197678+0000","last_fullsized":"2024-05-09T04:33:53.197678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:56.927838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.686401+0000","last_change":"2024-05-09T04:33:52.196143+0000","last_active":"2024-05-09T04:33:53.686401+0000","last_peered":"2024-05-09T04:33:53.686401+0000","last_clean":"2024-05-09T04:33:53.686401+0000","last_became_active":"2024-05-09T04:33:52.195814+0000","last_became_peered":"2024-05-09T04:33:52.195814+0000","last_unstale":"2024-05-09T04:33:53.686401+0000","last_undegraded":"2024-05-09T04:33:53.686401+0000","last_fullsized":"2024-05-09T04:33:53.686401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:17.801441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.154429+0000","last_change":"2024-05-09T04:33:52.205497+0000","last_active":"2024-05-09T04:33:54.154429+0000","last_peered":"2024-05-09T04:33:54.154429+0000","last_clean":"2024-05-09T04:33:54.154429+0000","last_became_active":"2024-05-09T04:33:52.205219+0000","last_became_peered":"2024-05-09T04:33:52.205219+0000","last_unstale":"2024-05-09T04:33:54.154429+0000","last_undegraded":"2024-05-09T04:33:54.154429+0000","last_fullsized":"2024-05-09T04:33:54.154429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:17.801441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.197606+0000","last_change":"2024-05-09T04:33:52.213356+0000","last_active":"2024-05-09T04:33:53.197606+0000","last_peered":"2024-05-09T04:33:53.197606+0000","last_clean":"2024-05-09T04:33:53.197606+0000","last_became_active":"2024-05-09T04:33:52.212783+0000","last_became_peered":"2024-05-09T04:33:52.212783+0000","last_unstale":"2024-05-09T04:33:53.197606+0000","last_undegraded":"2024-05-09T04:33:53.197606+0000","last_fullsized":"2024-05-09T04:33:53.197606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:24:37.669022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.194008+0000","last_change":"2024-05-09T04:33:52.209567+0000","last_active":"2024-05-09T04:33:53.194008+0000","last_peered":"2024-05-09T04:33:53.194008+0000","last_clean":"2024-05-09T04:33:53.194008+0000","last_became_active":"2024-05-09T04:33:52.209260+0000","last_became_peered":"2024-05-09T04:33:52.209260+0000","last_unstale":"2024-05-09T04:33:53.194008+0000","last_undegraded":"2024-05-09T04:33:53.194008+0000","last_fullsized":"2024-05-09T04:33:53.194008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:35:43.478085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.211635+0000","last_change":"2024-05-09T04:33:52.210744+0000","last_active":"2024-05-09T04:33:54.211635+0000","last_peered":"2024-05-09T04:33:54.211635+0000","last_clean":"2024-05-09T04:33:54.211635+0000","last_became_active":"2024-05-09T04:33:52.210421+0000","last_became_peered":"2024-05-09T04:33:52.210421+0000","last_unstale":"2024-05-09T04:33:54.211635+0000","last_undegraded":"2024-05-09T04:33:54.211635+0000","last_fullsized":"2024-05-09T04:33:54.211635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:13:19.513750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.732744+0000","last_change":"2024-05-09T04:33:52.193832+0000","last_active":"2024-05-09T04:33:54.732744+0000","last_peered":"2024-05-09T04:33:54.732744+0000","last_clean":"2024-05-09T04:33:54.732744+0000","last_became_active":"2024-05-09T04:33:52.193531+0000","last_became_peered":"2024-05-09T04:33:52.193531+0000","last_unstale":"2024-05-09T04:33:54.732744+0000","last_undegraded":"2024-05-09T04:33:54.732744+0000","last_fullsized":"2024-05-09T04:33:54.732744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:19:44.181447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.685556+0000","last_change":"2024-05-09T04:33:52.198677+0000","last_active":"2024-05-09T04:33:53.685556+0000","last_peered":"2024-05-09T04:33:53.685556+0000","last_clean":"2024-05-09T04:33:53.685556+0000","last_became_active":"2024-05-09T04:33:52.198334+0000","last_became_peered":"2024-05-09T04:33:52.198334+0000","last_unstale":"2024-05-09T04:33:53.685556+0000","last_undegraded":"2024-05-09T04:33:53.685556+0000","last_fullsized":"2024-05-09T04:33:53.685556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:53:58.795910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.194924+0000","last_change":"2024-05-09T04:33:52.192680+0000","last_active":"2024-05-09T04:33:53.194924+0000","last_peered":"2024-05-09T04:33:53.194924+0000","last_clean":"2024-05-09T04:33:53.194924+0000","last_became_active":"2024-05-09T04:33:52.192454+0000","last_became_peered":"2024-05-09T04:33:52.192454+0000","last_unstale":"2024-05-09T04:33:53.194924+0000","last_undegraded":"2024-05-09T04:33:53.194924+0000","last_fullsized":"2024-05-09T04:33:53.194924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:50:52.026927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.196881+0000","last_change":"2024-05-09T04:33:51.177063+0000","last_active":"2024-05-09T04:33:53.196881+0000","last_peered":"2024-05-09T04:33:53.196881+0000","last_clean":"2024-05-09T04:33:53.196881+0000","last_became_active":"2024-05-09T04:33:51.176788+0000","last_became_peered":"2024-05-09T04:33:51.176788+0000","last_unstale":"2024-05-09T04:33:53.196881+0000","last_undegraded":"2024-05-09T04:33:53.196881+0000","last_fullsized":"2024-05-09T04:33:53.196881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:04:10.949412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.194154+0000","last_change":"2024-05-09T04:33:52.190946+0000","last_active":"2024-05-09T04:33:53.194154+0000","last_peered":"2024-05-09T04:33:53.194154+0000","last_clean":"2024-05-09T04:33:53.194154+0000","last_became_active":"2024-05-09T04:33:52.190598+0000","last_became_peered":"2024-05-09T04:33:52.190598+0000","last_unstale":"2024-05-09T04:33:53.194154+0000","last_undegraded":"2024-05-09T04:33:53.194154+0000","last_fullsized":"2024-05-09T04:33:53.194154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:53:58.795910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.685499+0000","last_change":"2024-05-09T04:33:51.177930+0000","last_active":"2024-05-09T04:33:53.685499+0000","last_peered":"2024-05-09T04:33:53.685499+0000","last_clean":"2024-05-09T04:33:53.685499+0000","last_became_active":"2024-05-09T04:33:51.174452+0000","last_became_peered":"2024-05-09T04:33:51.174452+0000","last_unstale":"2024-05-09T04:33:53.685499+0000","last_undegraded":"2024-05-09T04:33:53.685499+0000","last_fullsized":"2024-05-09T04:33:53.685499+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:26:18.474267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.196968+0000","last_change":"2024-05-09T04:33:51.169721+0000","last_active":"2024-05-09T04:33:53.196968+0000","last_peered":"2024-05-09T04:33:53.196968+0000","last_clean":"2024-05-09T04:33:53.196968+0000","last_became_active":"2024-05-09T04:33:51.169422+0000","last_became_peered":"2024-05-09T04:33:51.169422+0000","last_unstale":"2024-05-09T04:33:53.196968+0000","last_undegraded":"2024-05-09T04:33:53.196968+0000","last_fullsized":"2024-05-09T04:33:53.196968+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:36:52.136732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.196263+0000","last_change":"2024-05-09T04:33:52.208855+0000","last_active":"2024-05-09T04:33:53.196263+0000","last_peered":"2024-05-09T04:33:53.196263+0000","last_clean":"2024-05-09T04:33:53.196263+0000","last_became_active":"2024-05-09T04:33:52.208588+0000","last_became_peered":"2024-05-09T04:33:52.208588+0000","last_unstale":"2024-05-09T04:33:53.196263+0000","last_undegraded":"2024-05-09T04:33:53.196263+0000","last_fullsized":"2024-05-09T04:33:53.196263+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:53:58.795910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.211333+0000","last_change":"2024-05-09T04:33:51.178180+0000","last_active":"2024-05-09T04:33:54.211333+0000","last_peered":"2024-05-09T04:33:54.211333+0000","last_clean":"2024-05-09T04:33:54.211333+0000","last_became_active":"2024-05-09T04:33:51.175516+0000","last_became_peered":"2024-05-09T04:33:51.175516+0000","last_unstale":"2024-05-09T04:33:54.211333+0000","last_undegraded":"2024-05-09T04:33:54.211333+0000","last_fullsized":"2024-05-09T04:33:54.211333+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:26:18.474267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.154507+0000","last_change":"2024-05-09T04:33:52.196488+0000","last_active":"2024-05-09T04:33:54.154507+0000","last_peered":"2024-05-09T04:33:54.154507+0000","last_clean":"2024-05-09T04:33:54.154507+0000","last_became_active":"2024-05-09T04:33:52.196175+0000","last_became_peered":"2024-05-09T04:33:52.196175+0000","last_unstale":"2024-05-09T04:33:54.154507+0000","last_undegraded":"2024-05-09T04:33:54.154507+0000","last_fullsized":"2024-05-09T04:33:54.154507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:56.927838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.686280+0000","last_change":"2024-05-09T04:33:51.185310+0000","last_active":"2024-05-09T04:33:53.686280+0000","last_peered":"2024-05-09T04:33:53.686280+0000","last_clean":"2024-05-09T04:33:53.686280+0000","last_became_active":"2024-05-09T04:33:51.185016+0000","last_became_peered":"2024-05-09T04:33:51.185016+0000","last_unstale":"2024-05-09T04:33:53.686280+0000","last_undegraded":"2024-05-09T04:33:53.686280+0000","last_fullsized":"2024-05-09T04:33:53.686280+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:50:51.015799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.212510+0000","last_change":"2024-05-09T04:33:52.193316+0000","last_active":"2024-05-09T04:33:54.212510+0000","last_peered":"2024-05-09T04:33:54.212510+0000","last_clean":"2024-05-09T04:33:54.212510+0000","last_became_active":"2024-05-09T04:33:52.192939+0000","last_became_peered":"2024-05-09T04:33:52.192939+0000","last_unstale":"2024-05-09T04:33:54.212510+0000","last_undegraded":"2024-05-09T04:33:54.212510+0000","last_fullsized":"2024-05-09T04:33:54.212510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:46:44.064608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.212393+0000","last_change":"2024-05-09T04:33:51.183116+0000","last_active":"2024-05-09T04:33:54.212393+0000","last_peered":"2024-05-09T04:33:54.212393+0000","last_clean":"2024-05-09T04:33:54.212393+0000","last_became_active":"2024-05-09T04:33:51.182750+0000","last_became_peered":"2024-05-09T04:33:51.182750+0000","last_unstale":"2024-05-09T04:33:54.212393+0000","last_undegraded":"2024-05-09T04:33:54.212393+0000","last_fullsized":"2024-05-09T04:33:54.212393+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:04:10.949412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.195562+0000","last_change":"2024-05-09T04:33:52.201494+0000","last_active":"2024-05-09T04:33:53.195562+0000","last_peered":"2024-05-09T04:33:53.195562+0000","last_clean":"2024-05-09T04:33:53.195562+0000","last_became_active":"2024-05-09T04:33:52.197939+0000","last_became_peered":"2024-05-09T04:33:52.197939+0000","last_unstale":"2024-05-09T04:33:53.195562+0000","last_undegraded":"2024-05-09T04:33:53.195562+0000","last_fullsized":"2024-05-09T04:33:53.195562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:29:08.279802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.154034+0000","last_change":"2024-05-09T04:33:51.178564+0000","last_active":"2024-05-09T04:33:54.154034+0000","last_peered":"2024-05-09T04:33:54.154034+0000","last_clean":"2024-05-09T04:33:54.154034+0000","last_became_active":"2024-05-09T04:33:51.177732+0000","last_became_peered":"2024-05-09T04:33:51.177732+0000","last_unstale":"2024-05-09T04:33:54.154034+0000","last_undegraded":"2024-05-09T04:33:54.154034+0000","last_fullsized":"2024-05-09T04:33:54.154034+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:50:51.015799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.196170+0000","last_change":"2024-05-09T04:33:52.200019+0000","last_active":"2024-05-09T04:33:53.196170+0000","last_peered":"2024-05-09T04:33:53.196170+0000","last_clean":"2024-05-09T04:33:53.196170+0000","last_became_active":"2024-05-09T04:33:52.199279+0000","last_became_peered":"2024-05-09T04:33:52.199279+0000","last_unstale":"2024-05-09T04:33:53.196170+0000","last_undegraded":"2024-05-09T04:33:53.196170+0000","last_fullsized":"2024-05-09T04:33:53.196170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:19:44.181447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.685981+0000","last_change":"2024-05-09T04:33:51.173622+0000","last_active":"2024-05-09T04:33:53.685981+0000","last_peered":"2024-05-09T04:33:53.685981+0000","last_clean":"2024-05-09T04:33:53.685981+0000","last_became_active":"2024-05-09T04:33:51.173158+0000","last_became_peered":"2024-05-09T04:33:51.173158+0000","last_unstale":"2024-05-09T04:33:53.685981+0000","last_undegraded":"2024-05-09T04:33:53.685981+0000","last_fullsized":"2024-05-09T04:33:53.685981+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:08:08.955016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T04:33:52.217209+0000","last_change":"2024-05-09T04:33:52.214221+0000","last_active":"2024-05-09T04:33:52.217209+0000","last_peered":"2024-05-09T04:33:52.217209+0000","last_clean":"2024-05-09T04:33:52.217209+0000","last_became_active":"2024-05-09T04:33:52.213773+0000","last_became_peered":"2024-05-09T04:33:52.213773+0000","last_unstale":"2024-05-09T04:33:52.217209+0000","last_undegraded":"2024-05-09T04:33:52.217209+0000","last_fullsized":"2024-05-09T04:33:52.217209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:24:37.669022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.194274+0000","last_change":"2024-05-09T04:33:51.172542+0000","last_active":"2024-05-09T04:33:53.194274+0000","last_peered":"2024-05-09T04:33:53.194274+0000","last_clean":"2024-05-09T04:33:53.194274+0000","last_became_active":"2024-05-09T04:33:51.172232+0000","last_became_peered":"2024-05-09T04:33:51.172232+0000","last_unstale":"2024-05-09T04:33:53.194274+0000","last_undegraded":"2024-05-09T04:33:53.194274+0000","last_fullsized":"2024-05-09T04:33:53.194274+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:15.760572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.194183+0000","last_change":"2024-05-09T04:33:52.195027+0000","last_active":"2024-05-09T04:33:53.194183+0000","last_peered":"2024-05-09T04:33:53.194183+0000","last_clean":"2024-05-09T04:33:53.194183+0000","last_became_active":"2024-05-09T04:33:52.194590+0000","last_became_peered":"2024-05-09T04:33:52.194590+0000","last_unstale":"2024-05-09T04:33:53.194183+0000","last_undegraded":"2024-05-09T04:33:53.194183+0000","last_fullsized":"2024-05-09T04:33:53.194183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:34:40.257159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.211870+0000","last_change":"2024-05-09T04:33:51.177887+0000","last_active":"2024-05-09T04:33:54.211870+0000","last_peered":"2024-05-09T04:33:54.211870+0000","last_clean":"2024-05-09T04:33:54.211870+0000","last_became_active":"2024-05-09T04:33:51.175721+0000","last_became_peered":"2024-05-09T04:33:51.175721+0000","last_unstale":"2024-05-09T04:33:54.211870+0000","last_undegraded":"2024-05-09T04:33:54.211870+0000","last_fullsized":"2024-05-09T04:33:54.211870+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:02:50.965016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T04:33:52.217184+0000","last_change":"2024-05-09T04:33:52.211924+0000","last_active":"2024-05-09T04:33:52.217184+0000","last_peered":"2024-05-09T04:33:52.217184+0000","last_clean":"2024-05-09T04:33:52.217184+0000","last_became_active":"2024-05-09T04:33:52.211546+0000","last_became_peered":"2024-05-09T04:33:52.211546+0000","last_unstale":"2024-05-09T04:33:52.217184+0000","last_undegraded":"2024-05-09T04:33:52.217184+0000","last_fullsized":"2024-05-09T04:33:52.217184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:17.801441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.685619+0000","last_change":"2024-05-09T04:33:51.176697+0000","last_active":"2024-05-09T04:33:53.685619+0000","last_peered":"2024-05-09T04:33:53.685619+0000","last_clean":"2024-05-09T04:33:53.685619+0000","last_became_active":"2024-05-09T04:33:51.176351+0000","last_became_peered":"2024-05-09T04:33:51.176351+0000","last_unstale":"2024-05-09T04:33:53.685619+0000","last_undegraded":"2024-05-09T04:33:53.685619+0000","last_fullsized":"2024-05-09T04:33:53.685619+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:04:10.949412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.685570+0000","last_change":"2024-05-09T04:33:52.187596+0000","last_active":"2024-05-09T04:33:53.685570+0000","last_peered":"2024-05-09T04:33:53.685570+0000","last_clean":"2024-05-09T04:33:53.685570+0000","last_became_active":"2024-05-09T04:33:52.187287+0000","last_became_peered":"2024-05-09T04:33:52.187287+0000","last_unstale":"2024-05-09T04:33:53.685570+0000","last_undegraded":"2024-05-09T04:33:53.685570+0000","last_fullsized":"2024-05-09T04:33:53.685570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:34:40.257159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.732380+0000","last_change":"2024-05-09T04:33:51.166963+0000","last_active":"2024-05-09T04:33:54.732380+0000","last_peered":"2024-05-09T04:33:54.732380+0000","last_clean":"2024-05-09T04:33:54.732380+0000","last_became_active":"2024-05-09T04:33:51.165585+0000","last_became_peered":"2024-05-09T04:33:51.165585+0000","last_unstale":"2024-05-09T04:33:54.732380+0000","last_undegraded":"2024-05-09T04:33:54.732380+0000","last_fullsized":"2024-05-09T04:33:54.732380+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:04:10.949412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T04:33:52.215096+0000","last_change":"2024-05-09T04:33:52.207666+0000","last_active":"2024-05-09T04:33:52.215096+0000","last_peered":"2024-05-09T04:33:52.215096+0000","last_clean":"2024-05-09T04:33:52.215096+0000","last_became_active":"2024-05-09T04:33:52.207429+0000","last_became_peered":"2024-05-09T04:33:52.207429+0000","last_unstale":"2024-05-09T04:33:52.215096+0000","last_undegraded":"2024-05-09T04:33:52.215096+0000","last_fullsized":"2024-05-09T04:33:52.215096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:29:08.279802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.196757+0000","last_change":"2024-05-09T04:33:51.172081+0000","last_active":"2024-05-09T04:33:53.196757+0000","last_peered":"2024-05-09T04:33:53.196757+0000","last_clean":"2024-05-09T04:33:53.196757+0000","last_became_active":"2024-05-09T04:33:51.171700+0000","last_became_peered":"2024-05-09T04:33:51.171700+0000","last_unstale":"2024-05-09T04:33:53.196757+0000","last_undegraded":"2024-05-09T04:33:53.196757+0000","last_fullsized":"2024-05-09T04:33:53.196757+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:15.760572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T04:33:52.215044+0000","last_change":"2024-05-09T04:33:52.206889+0000","last_active":"2024-05-09T04:33:52.215044+0000","last_peered":"2024-05-09T04:33:52.215044+0000","last_clean":"2024-05-09T04:33:52.215044+0000","last_became_active":"2024-05-09T04:33:52.206452+0000","last_became_peered":"2024-05-09T04:33:52.206452+0000","last_unstale":"2024-05-09T04:33:52.215044+0000","last_undegraded":"2024-05-09T04:33:52.215044+0000","last_fullsized":"2024-05-09T04:33:52.215044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:34:29.852986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.732880+0000","last_change":"2024-05-09T04:33:51.167434+0000","last_active":"2024-05-09T04:33:54.732880+0000","last_peered":"2024-05-09T04:33:54.732880+0000","last_clean":"2024-05-09T04:33:54.732880+0000","last_became_active":"2024-05-09T04:33:51.167117+0000","last_became_peered":"2024-05-09T04:33:51.167117+0000","last_unstale":"2024-05-09T04:33:54.732880+0000","last_undegraded":"2024-05-09T04:33:54.732880+0000","last_fullsized":"2024-05-09T04:33:54.732880+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:02:50.965016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T04:33:52.215060+0000","last_change":"2024-05-09T04:33:52.207191+0000","last_active":"2024-05-09T04:33:52.215060+0000","last_peered":"2024-05-09T04:33:52.215060+0000","last_clean":"2024-05-09T04:33:52.215060+0000","last_became_active":"2024-05-09T04:33:52.206932+0000","last_became_peered":"2024-05-09T04:33:52.206932+0000","last_unstale":"2024-05-09T04:33:52.215060+0000","last_undegraded":"2024-05-09T04:33:52.215060+0000","last_fullsized":"2024-05-09T04:33:52.215060+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:59:28.002585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.154386+0000","last_change":"2024-05-09T04:33:51.163656+0000","last_active":"2024-05-09T04:33:54.154386+0000","last_peered":"2024-05-09T04:33:54.154386+0000","last_clean":"2024-05-09T04:33:54.154386+0000","last_became_active":"2024-05-09T04:33:51.163342+0000","last_became_peered":"2024-05-09T04:33:51.163342+0000","last_unstale":"2024-05-09T04:33:54.154386+0000","last_undegraded":"2024-05-09T04:33:54.154386+0000","last_fullsized":"2024-05-09T04:33:54.154386+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:19:47.722161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.212177+0000","last_change":"2024-05-09T04:33:52.205802+0000","last_active":"2024-05-09T04:33:54.212177+0000","last_peered":"2024-05-09T04:33:54.212177+0000","last_clean":"2024-05-09T04:33:54.212177+0000","last_became_active":"2024-05-09T04:33:52.205470+0000","last_became_peered":"2024-05-09T04:33:52.205470+0000","last_unstale":"2024-05-09T04:33:54.212177+0000","last_undegraded":"2024-05-09T04:33:54.212177+0000","last_fullsized":"2024-05-09T04:33:54.212177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:40:40.749511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.732648+0000","last_change":"2024-05-09T04:33:51.180702+0000","last_active":"2024-05-09T04:33:54.732648+0000","last_peered":"2024-05-09T04:33:54.732648+0000","last_clean":"2024-05-09T04:33:54.732648+0000","last_became_active":"2024-05-09T04:33:51.180322+0000","last_became_peered":"2024-05-09T04:33:51.180322+0000","last_unstale":"2024-05-09T04:33:54.732648+0000","last_undegraded":"2024-05-09T04:33:54.732648+0000","last_fullsized":"2024-05-09T04:33:54.732648+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:26:18.474267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T04:33:52.213552+0000","last_change":"2024-05-09T04:33:52.204685+0000","last_active":"2024-05-09T04:33:52.213552+0000","last_peered":"2024-05-09T04:33:52.213552+0000","last_clean":"2024-05-09T04:33:52.213552+0000","last_became_active":"2024-05-09T04:33:52.204353+0000","last_became_peered":"2024-05-09T04:33:52.204353+0000","last_unstale":"2024-05-09T04:33:52.213552+0000","last_undegraded":"2024-05-09T04:33:52.213552+0000","last_fullsized":"2024-05-09T04:33:52.213552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:53:58.795910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":108,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T04:33:52.176553+0000","last_change":"2024-05-09T04:33:41.092063+0000","last_active":"2024-05-09T04:33:52.176553+0000","last_peered":"2024-05-09T04:33:52.176553+0000","last_clean":"2024-05-09T04:33:52.176553+0000","last_became_active":"2024-05-09T04:33:41.091194+0000","last_became_peered":"2024-05-09T04:33:41.091194+0000","last_unstale":"2024-05-09T04:33:52.176553+0000","last_undegraded":"2024-05-09T04:33:52.176553+0000","last_fullsized":"2024-05-09T04:33:52.176553+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:40.045339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:40.045339+0000","last_clean_scrub_stamp":"2024-05-09T04:33:40.045339+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:04.908456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.211276+0000","last_change":"2024-05-09T04:33:51.179637+0000","last_active":"2024-05-09T04:33:54.211276+0000","last_peered":"2024-05-09T04:33:54.211276+0000","last_clean":"2024-05-09T04:33:54.211276+0000","last_became_active":"2024-05-09T04:33:51.179311+0000","last_became_peered":"2024-05-09T04:33:51.179311+0000","last_unstale":"2024-05-09T04:33:54.211276+0000","last_undegraded":"2024-05-09T04:33:54.211276+0000","last_fullsized":"2024-05-09T04:33:54.211276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:08:08.955016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.196827+0000","last_change":"2024-05-09T04:33:52.203706+0000","last_active":"2024-05-09T04:33:53.196827+0000","last_peered":"2024-05-09T04:33:53.196827+0000","last_clean":"2024-05-09T04:33:53.196827+0000","last_became_active":"2024-05-09T04:33:52.200053+0000","last_became_peered":"2024-05-09T04:33:52.200053+0000","last_unstale":"2024-05-09T04:33:53.196827+0000","last_undegraded":"2024-05-09T04:33:53.196827+0000","last_fullsized":"2024-05-09T04:33:53.196827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:29:08.279802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T04:33:52.176597+0000","last_change":"2024-05-09T04:33:51.185731+0000","last_active":"2024-05-09T04:33:52.176597+0000","last_peered":"2024-05-09T04:33:52.176597+0000","last_clean":"2024-05-09T04:33:52.176597+0000","last_became_active":"2024-05-09T04:33:51.184982+0000","last_became_peered":"2024-05-09T04:33:51.184982+0000","last_unstale":"2024-05-09T04:33:52.176597+0000","last_undegraded":"2024-05-09T04:33:52.176597+0000","last_fullsized":"2024-05-09T04:33:52.176597+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:29:11.409173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.193579+0000","last_change":"2024-05-09T04:33:52.208347+0000","last_active":"2024-05-09T04:33:53.193579+0000","last_peered":"2024-05-09T04:33:53.193579+0000","last_clean":"2024-05-09T04:33:53.193579+0000","last_became_active":"2024-05-09T04:33:52.206631+0000","last_became_peered":"2024-05-09T04:33:52.206631+0000","last_unstale":"2024-05-09T04:33:53.193579+0000","last_undegraded":"2024-05-09T04:33:53.193579+0000","last_fullsized":"2024-05-09T04:33:53.193579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:00:54.458043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.732806+0000","last_change":"2024-05-09T04:33:51.167013+0000","last_active":"2024-05-09T04:33:54.732806+0000","last_peered":"2024-05-09T04:33:54.732806+0000","last_clean":"2024-05-09T04:33:54.732806+0000","last_became_active":"2024-05-09T04:33:51.165450+0000","last_became_peered":"2024-05-09T04:33:51.165450+0000","last_unstale":"2024-05-09T04:33:54.732806+0000","last_undegraded":"2024-05-09T04:33:54.732806+0000","last_fullsized":"2024-05-09T04:33:54.732806+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:08:08.955016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.154238+0000","last_change":"2024-05-09T04:33:52.190418+0000","last_active":"2024-05-09T04:33:54.154238+0000","last_peered":"2024-05-09T04:33:54.154238+0000","last_clean":"2024-05-09T04:33:54.154238+0000","last_became_active":"2024-05-09T04:33:52.190069+0000","last_became_peered":"2024-05-09T04:33:52.190069+0000","last_unstale":"2024-05-09T04:33:54.154238+0000","last_undegraded":"2024-05-09T04:33:54.154238+0000","last_fullsized":"2024-05-09T04:33:54.154238+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:29:08.279802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.212222+0000","last_change":"2024-05-09T04:33:51.181463+0000","last_active":"2024-05-09T04:33:54.212222+0000","last_peered":"2024-05-09T04:33:54.212222+0000","last_clean":"2024-05-09T04:33:54.212222+0000","last_became_active":"2024-05-09T04:33:51.181213+0000","last_became_peered":"2024-05-09T04:33:51.181213+0000","last_unstale":"2024-05-09T04:33:54.212222+0000","last_undegraded":"2024-05-09T04:33:54.212222+0000","last_fullsized":"2024-05-09T04:33:54.212222+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:50:51.015799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.195331+0000","last_change":"2024-05-09T04:33:52.188609+0000","last_active":"2024-05-09T04:33:53.195331+0000","last_peered":"2024-05-09T04:33:53.195331+0000","last_clean":"2024-05-09T04:33:53.195331+0000","last_became_active":"2024-05-09T04:33:52.188379+0000","last_became_peered":"2024-05-09T04:33:52.188379+0000","last_unstale":"2024-05-09T04:33:53.195331+0000","last_undegraded":"2024-05-09T04:33:53.195331+0000","last_fullsized":"2024-05-09T04:33:53.195331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:19:48.733290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.153924+0000","last_change":"2024-05-09T04:33:51.164123+0000","last_active":"2024-05-09T04:33:54.153924+0000","last_peered":"2024-05-09T04:33:54.153924+0000","last_clean":"2024-05-09T04:33:54.153924+0000","last_became_active":"2024-05-09T04:33:51.163686+0000","last_became_peered":"2024-05-09T04:33:51.163686+0000","last_unstale":"2024-05-09T04:33:54.153924+0000","last_undegraded":"2024-05-09T04:33:54.153924+0000","last_fullsized":"2024-05-09T04:33:54.153924+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:29:11.409173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.212248+0000","last_change":"2024-05-09T04:33:52.209697+0000","last_active":"2024-05-09T04:33:54.212248+0000","last_peered":"2024-05-09T04:33:54.212248+0000","last_clean":"2024-05-09T04:33:54.212248+0000","last_became_active":"2024-05-09T04:33:52.209347+0000","last_became_peered":"2024-05-09T04:33:52.209347+0000","last_unstale":"2024-05-09T04:33:54.212248+0000","last_undegraded":"2024-05-09T04:33:54.212248+0000","last_fullsized":"2024-05-09T04:33:54.212248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:19:08.318788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T04:33:52.179499+0000","last_change":"2024-05-09T04:33:51.184584+0000","last_active":"2024-05-09T04:33:52.179499+0000","last_peered":"2024-05-09T04:33:52.179499+0000","last_clean":"2024-05-09T04:33:52.179499+0000","last_became_active":"2024-05-09T04:33:51.184369+0000","last_became_peered":"2024-05-09T04:33:51.184369+0000","last_unstale":"2024-05-09T04:33:52.179499+0000","last_undegraded":"2024-05-09T04:33:52.179499+0000","last_fullsized":"2024-05-09T04:33:52.179499+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:22:40.919482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.154095+0000","last_change":"2024-05-09T04:33:52.189698+0000","last_active":"2024-05-09T04:33:54.154095+0000","last_peered":"2024-05-09T04:33:54.154095+0000","last_clean":"2024-05-09T04:33:54.154095+0000","last_became_active":"2024-05-09T04:33:52.189388+0000","last_became_peered":"2024-05-09T04:33:52.189388+0000","last_unstale":"2024-05-09T04:33:54.154095+0000","last_undegraded":"2024-05-09T04:33:54.154095+0000","last_fullsized":"2024-05-09T04:33:54.154095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:59:28.002585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.179780+0000","last_change":"2024-05-09T04:33:51.173756+0000","last_active":"2024-05-09T04:33:54.179780+0000","last_peered":"2024-05-09T04:33:54.179780+0000","last_clean":"2024-05-09T04:33:54.179780+0000","last_became_active":"2024-05-09T04:33:51.173416+0000","last_became_peered":"2024-05-09T04:33:51.173416+0000","last_unstale":"2024-05-09T04:33:54.179780+0000","last_undegraded":"2024-05-09T04:33:54.179780+0000","last_fullsized":"2024-05-09T04:33:54.179780+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:02:50.965016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.197927+0000","last_change":"2024-05-09T04:33:52.205567+0000","last_active":"2024-05-09T04:33:53.197927+0000","last_peered":"2024-05-09T04:33:53.197927+0000","last_clean":"2024-05-09T04:33:53.197927+0000","last_became_active":"2024-05-09T04:33:52.205252+0000","last_became_peered":"2024-05-09T04:33:52.205252+0000","last_unstale":"2024-05-09T04:33:53.197927+0000","last_undegraded":"2024-05-09T04:33:53.197927+0000","last_fullsized":"2024-05-09T04:33:53.197927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:13:19.513750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.686413+0000","last_change":"2024-05-09T04:33:51.181117+0000","last_active":"2024-05-09T04:33:53.686413+0000","last_peered":"2024-05-09T04:33:53.686413+0000","last_clean":"2024-05-09T04:33:53.686413+0000","last_became_active":"2024-05-09T04:33:51.179062+0000","last_became_peered":"2024-05-09T04:33:51.179062+0000","last_unstale":"2024-05-09T04:33:53.686413+0000","last_undegraded":"2024-05-09T04:33:53.686413+0000","last_fullsized":"2024-05-09T04:33:53.686413+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:19:47.722161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.195642+0000","last_change":"2024-05-09T04:33:52.181414+0000","last_active":"2024-05-09T04:33:53.195642+0000","last_peered":"2024-05-09T04:33:53.195642+0000","last_clean":"2024-05-09T04:33:53.195642+0000","last_became_active":"2024-05-09T04:33:52.180058+0000","last_became_peered":"2024-05-09T04:33:52.180058+0000","last_unstale":"2024-05-09T04:33:53.195642+0000","last_undegraded":"2024-05-09T04:33:53.195642+0000","last_fullsized":"2024-05-09T04:33:53.195642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:17.801441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T04:33:52.176491+0000","last_change":"2024-05-09T04:33:51.183265+0000","last_active":"2024-05-09T04:33:52.176491+0000","last_peered":"2024-05-09T04:33:52.176491+0000","last_clean":"2024-05-09T04:33:52.176491+0000","last_became_active":"2024-05-09T04:33:51.181781+0000","last_became_peered":"2024-05-09T04:33:51.181781+0000","last_unstale":"2024-05-09T04:33:52.176491+0000","last_undegraded":"2024-05-09T04:33:52.176491+0000","last_fullsized":"2024-05-09T04:33:52.176491+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:19:47.722161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.194563+0000","last_change":"2024-05-09T04:33:52.203247+0000","last_active":"2024-05-09T04:33:53.194563+0000","last_peered":"2024-05-09T04:33:53.194563+0000","last_clean":"2024-05-09T04:33:53.194563+0000","last_became_active":"2024-05-09T04:33:52.203006+0000","last_became_peered":"2024-05-09T04:33:52.203006+0000","last_unstale":"2024-05-09T04:33:53.194563+0000","last_undegraded":"2024-05-09T04:33:53.194563+0000","last_fullsized":"2024-05-09T04:33:53.194563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:02:51.976144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.154676+0000","last_change":"2024-05-09T04:33:51.172718+0000","last_active":"2024-05-09T04:33:54.154676+0000","last_peered":"2024-05-09T04:33:54.154676+0000","last_clean":"2024-05-09T04:33:54.154676+0000","last_became_active":"2024-05-09T04:33:51.172131+0000","last_became_peered":"2024-05-09T04:33:51.172131+0000","last_unstale":"2024-05-09T04:33:54.154676+0000","last_undegraded":"2024-05-09T04:33:54.154676+0000","last_fullsized":"2024-05-09T04:33:54.154676+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:08:08.955016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.196487+0000","last_change":"2024-05-09T04:33:52.201413+0000","last_active":"2024-05-09T04:33:53.196487+0000","last_peered":"2024-05-09T04:33:53.196487+0000","last_clean":"2024-05-09T04:33:53.196487+0000","last_became_active":"2024-05-09T04:33:52.199032+0000","last_became_peered":"2024-05-09T04:33:52.199032+0000","last_unstale":"2024-05-09T04:33:53.196487+0000","last_undegraded":"2024-05-09T04:33:53.196487+0000","last_fullsized":"2024-05-09T04:33:53.196487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:34:40.257159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.154633+0000","last_change":"2024-05-09T04:33:51.159715+0000","last_active":"2024-05-09T04:33:54.154633+0000","last_peered":"2024-05-09T04:33:54.154633+0000","last_clean":"2024-05-09T04:33:54.154633+0000","last_became_active":"2024-05-09T04:33:51.159395+0000","last_became_peered":"2024-05-09T04:33:51.159395+0000","last_unstale":"2024-05-09T04:33:54.154633+0000","last_undegraded":"2024-05-09T04:33:54.154633+0000","last_fullsized":"2024-05-09T04:33:54.154633+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:19:43.170318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.197360+0000","last_change":"2024-05-09T04:33:52.198684+0000","last_active":"2024-05-09T04:33:53.197360+0000","last_peered":"2024-05-09T04:33:53.197360+0000","last_clean":"2024-05-09T04:33:53.197360+0000","last_became_active":"2024-05-09T04:33:52.196451+0000","last_became_peered":"2024-05-09T04:33:52.196451+0000","last_unstale":"2024-05-09T04:33:53.197360+0000","last_undegraded":"2024-05-09T04:33:53.197360+0000","last_fullsized":"2024-05-09T04:33:53.197360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:17.801441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T04:33:52.218009+0000","last_change":"2024-05-09T04:33:51.186050+0000","last_active":"2024-05-09T04:33:52.218009+0000","last_peered":"2024-05-09T04:33:52.218009+0000","last_clean":"2024-05-09T04:33:52.218009+0000","last_became_active":"2024-05-09T04:33:51.185720+0000","last_became_peered":"2024-05-09T04:33:51.185720+0000","last_unstale":"2024-05-09T04:33:52.218009+0000","last_undegraded":"2024-05-09T04:33:52.218009+0000","last_fullsized":"2024-05-09T04:33:52.218009+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:11:05.364073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.685574+0000","last_change":"2024-05-09T04:33:52.191123+0000","last_active":"2024-05-09T04:33:53.685574+0000","last_peered":"2024-05-09T04:33:53.685574+0000","last_clean":"2024-05-09T04:33:53.685574+0000","last_became_active":"2024-05-09T04:33:52.189005+0000","last_became_peered":"2024-05-09T04:33:52.189005+0000","last_unstale":"2024-05-09T04:33:53.685574+0000","last_undegraded":"2024-05-09T04:33:53.685574+0000","last_fullsized":"2024-05-09T04:33:53.685574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:34:29.852986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.212085+0000","last_change":"2024-05-09T04:33:52.208515+0000","last_active":"2024-05-09T04:33:54.212085+0000","last_peered":"2024-05-09T04:33:54.212085+0000","last_clean":"2024-05-09T04:33:54.212085+0000","last_became_active":"2024-05-09T04:33:52.208236+0000","last_became_peered":"2024-05-09T04:33:52.208236+0000","last_unstale":"2024-05-09T04:33:54.212085+0000","last_undegraded":"2024-05-09T04:33:54.212085+0000","last_fullsized":"2024-05-09T04:33:54.212085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:19:13.480834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T04:33:52.215642+0000","last_change":"2024-05-09T04:33:51.187919+0000","last_active":"2024-05-09T04:33:52.215642+0000","last_peered":"2024-05-09T04:33:52.215642+0000","last_clean":"2024-05-09T04:33:52.215642+0000","last_became_active":"2024-05-09T04:33:51.187590+0000","last_became_peered":"2024-05-09T04:33:51.187590+0000","last_unstale":"2024-05-09T04:33:52.215642+0000","last_undegraded":"2024-05-09T04:33:52.215642+0000","last_fullsized":"2024-05-09T04:33:52.215642+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:50:51.015799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.211737+0000","last_change":"2024-05-09T04:33:52.209870+0000","last_active":"2024-05-09T04:33:54.211737+0000","last_peered":"2024-05-09T04:33:54.211737+0000","last_clean":"2024-05-09T04:33:54.211737+0000","last_became_active":"2024-05-09T04:33:52.209199+0000","last_became_peered":"2024-05-09T04:33:52.209199+0000","last_unstale":"2024-05-09T04:33:54.211737+0000","last_undegraded":"2024-05-09T04:33:54.211737+0000","last_fullsized":"2024-05-09T04:33:54.211737+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:10:42.676456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-05-09T04:33:52.211755+0000","last_change":"2024-05-09T04:33:51.187870+0000","last_active":"2024-05-09T04:33:52.211755+0000","last_peered":"2024-05-09T04:33:52.211755+0000","last_clean":"2024-05-09T04:33:52.211755+0000","last_became_active":"2024-05-09T04:33:51.187575+0000","last_became_peered":"2024-05-09T04:33:51.187575+0000","last_unstale":"2024-05-09T04:33:52.211755+0000","last_undegraded":"2024-05-09T04:33:52.211755+0000","last_fullsized":"2024-05-09T04:33:52.211755+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:19:43.170318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.195161+0000","last_change":"2024-05-09T04:33:52.201406+0000","last_active":"2024-05-09T04:33:53.195161+0000","last_peered":"2024-05-09T04:33:53.195161+0000","last_clean":"2024-05-09T04:33:53.195161+0000","last_became_active":"2024-05-09T04:33:52.200809+0000","last_became_peered":"2024-05-09T04:33:52.200809+0000","last_unstale":"2024-05-09T04:33:53.195161+0000","last_undegraded":"2024-05-09T04:33:53.195161+0000","last_fullsized":"2024-05-09T04:33:53.195161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:59:28.002585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.197985+0000","last_change":"2024-05-09T04:33:51.168105+0000","last_active":"2024-05-09T04:33:53.197985+0000","last_peered":"2024-05-09T04:33:53.197985+0000","last_clean":"2024-05-09T04:33:53.197985+0000","last_became_active":"2024-05-09T04:33:51.167790+0000","last_became_peered":"2024-05-09T04:33:51.167790+0000","last_unstale":"2024-05-09T04:33:53.197985+0000","last_undegraded":"2024-05-09T04:33:53.197985+0000","last_fullsized":"2024-05-09T04:33:53.197985+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:08:08.955016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.197883+0000","last_change":"2024-05-09T04:33:52.205526+0000","last_active":"2024-05-09T04:33:53.197883+0000","last_peered":"2024-05-09T04:33:53.197883+0000","last_clean":"2024-05-09T04:33:53.197883+0000","last_became_active":"2024-05-09T04:33:52.205231+0000","last_became_peered":"2024-05-09T04:33:52.205231+0000","last_unstale":"2024-05-09T04:33:53.197883+0000","last_undegraded":"2024-05-09T04:33:53.197883+0000","last_fullsized":"2024-05-09T04:33:53.197883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:02:17.813339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.195818+0000","last_change":"2024-05-09T04:33:51.176461+0000","last_active":"2024-05-09T04:33:53.195818+0000","last_peered":"2024-05-09T04:33:53.195818+0000","last_clean":"2024-05-09T04:33:53.195818+0000","last_became_active":"2024-05-09T04:33:51.176175+0000","last_became_peered":"2024-05-09T04:33:51.176175+0000","last_unstale":"2024-05-09T04:33:53.195818+0000","last_undegraded":"2024-05-09T04:33:53.195818+0000","last_fullsized":"2024-05-09T04:33:53.195818+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:01:20.702014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":790528,"data_stored":655584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":192512,"data_stored":64634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":200704,"data_stored":64794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T04:33:57.094 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-09T04:33:57.094 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-09T04:33:57.095 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-05-09T04:33:57.095 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-05-09T04:33:57.095 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-05-09T04:33:57.335 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.335+0000 7f1fd5a00640 1 Processor -- start 2024-05-09T04:33:57.335 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.335+0000 7f1fd5a00640 1 -- start start 2024-05-09T04:33:57.337 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.337+0000 7f1fd5a00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f1fd00d3180 0x7f1fd00d5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:57.337 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.337+0000 7f1fd5a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fd00b4760 0x7f1fd00b4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:57.337 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.337+0000 7f1fd5a00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fd00d3e70 con 0x7f1fd00d22f0 2024-05-09T04:33:57.337 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.337+0000 7f1fd5a00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f1fd00d4150 con 0x7f1fd00d3180 2024-05-09T04:33:57.337 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.337+0000 7f1fd5a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f1fd00b6200 con 0x7f1fd00b4760 2024-05-09T04:33:57.338 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.338+0000 7f1fd5000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fd00b4760 0x7f1fd00b4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:57.338 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.338+0000 7f1fd5000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fd00b4760 0x7f1fd00b4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.145:47272/0 (socket says 172.21.15.145:47272) 2024-05-09T04:33:57.338 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.338+0000 7f1fd5000640 1 -- 172.21.15.145:0/3962189431 learned_addr learned my addr 172.21.15.145:0/3962189431 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:57.338 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.338+0000 7f1fcfe00640 1 --2- 172.21.15.145:0/3962189431 >> v2:172.21.15.192:3300/0 conn(0x7f1fd00d3180 0x7f1fd00d5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:57.338 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.339+0000 7f1fd5000640 1 -- 172.21.15.145:0/3962189431 >> v2:172.21.15.192:3300/0 conn(0x7f1fd00d3180 msgr2=0x7f1fd00d5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:57.338 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.339+0000 7f1fd5000640 1 --2- 172.21.15.145:0/3962189431 >> v2:172.21.15.192:3300/0 conn(0x7f1fd00d3180 0x7f1fd00d5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:57.339 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.339+0000 7f1fd5000640 1 -- 172.21.15.145:0/3962189431 >> v1:172.21.15.192:6789/0 conn(0x7f1fd00d22f0 legacy=0x7f1fd00d3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:57.339 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.339+0000 7f1fd5000640 1 -- 172.21.15.145:0/3962189431 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fd00b6450 con 0x7f1fd00b4760 2024-05-09T04:33:57.339 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.340+0000 7f1fd5000640 1 --2- 172.21.15.145:0/3962189431 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fd00b4760 0x7f1fd00b4bd0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1fc8009e50 tx=0x7f1fc802a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36c3344db832476 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:57.340 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.340+0000 7f1fcf400640 1 -- 172.21.15.145:0/3962189431 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc8036070 con 0x7f1fd00b4760 2024-05-09T04:33:57.340 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.340+0000 7f1fcf400640 1 -- 172.21.15.145:0/3962189431 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fc8031600 con 0x7f1fd00b4760 2024-05-09T04:33:57.340 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.340+0000 7f1fcf400640 1 -- 172.21.15.145:0/3962189431 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc80319d0 con 0x7f1fd00b4760 2024-05-09T04:33:57.341 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.341+0000 7f1fd5a00640 1 -- 172.21.15.145:0/3962189431 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fd00b4760 msgr2=0x7f1fd00b4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:57.341 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.341+0000 7f1fd5a00640 1 --2- 172.21.15.145:0/3962189431 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fd00b4760 0x7f1fd00b4bd0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1fc8009e50 tx=0x7f1fc802a680 comp rx=0 tx=0).stop 2024-05-09T04:33:57.341 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.341+0000 7f1fd5a00640 1 -- 172.21.15.145:0/3962189431 shutdown_connections 2024-05-09T04:33:57.341 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.341+0000 7f1fd5a00640 1 --2- 172.21.15.145:0/3962189431 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fd00b4760 0x7f1fd00b4bd0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:57.341 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.341+0000 7f1fd5a00640 1 --2- 172.21.15.145:0/3962189431 >> v2:172.21.15.192:3300/0 conn(0x7f1fd00d3180 0x7f1fd00d5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:57.342 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.341+0000 7f1fd5a00640 1 -- 172.21.15.145:0/3962189431 >> 172.21.15.145:0/3962189431 conn(0x7f1fd00d4fb0 msgr2=0x7f1fd00d53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:57.342 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.341+0000 7f1fd5a00640 1 -- 172.21.15.145:0/3962189431 shutdown_connections 2024-05-09T04:33:57.342 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.341+0000 7f1fd5a00640 1 -- 172.21.15.145:0/3962189431 wait complete. 2024-05-09T04:33:57.343 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.343+0000 7f1fd5a00640 1 Processor -- start 2024-05-09T04:33:57.343 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.343+0000 7f1fd5a00640 1 -- start start 2024-05-09T04:33:57.345 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.344+0000 7f1fd5a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1fd00b4760 0x7f1fd015de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:57.345 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.344+0000 7f1fd5a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fd00d22f0 0x7f1fd0158250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:57.345 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.344+0000 7f1fd5a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1fd00d3180 0x7f1fd015a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:57.345 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.344+0000 7f1fd5a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd00d2780 con 0x7f1fd00d3180 2024-05-09T04:33:57.345 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.344+0000 7f1fd5a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd00d1e60 con 0x7f1fd00b4760 2024-05-09T04:33:57.345 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.344+0000 7f1fd5a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f1fd00b3fa0 con 0x7f1fd00d22f0 2024-05-09T04:33:57.345 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.344+0000 7f1fd5000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1fd00d3180 0x7f1fd015a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:57.345 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.344+0000 7f1fd5000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1fd00d3180 0x7f1fd015a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:44984/0 (socket says 172.21.15.145:44984) 2024-05-09T04:33:57.345 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.344+0000 7f1fd5000640 1 -- 172.21.15.145:0/2784968107 learned_addr learned my addr 172.21.15.145:0/2784968107 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:57.345 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.345+0000 7f1fcfe00640 1 --2- 172.21.15.145:0/2784968107 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1fd00b4760 0x7f1fd015de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:57.345 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.345+0000 7f1fc7400640 1 --2- 172.21.15.145:0/2784968107 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fd00d22f0 0x7f1fd0158250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:57.345 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.345+0000 7f1fd5000640 1 -- 172.21.15.145:0/2784968107 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fd00d22f0 msgr2=0x7f1fd0158250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:57.346 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.345+0000 7f1fd5000640 1 --2- 172.21.15.145:0/2784968107 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fd00d22f0 0x7f1fd0158250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:57.346 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.345+0000 7f1fd5000640 1 -- 172.21.15.145:0/2784968107 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1fd00b4760 msgr2=0x7f1fd015de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:57.346 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.345+0000 7f1fd5000640 1 --2- 172.21.15.145:0/2784968107 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1fd00b4760 0x7f1fd015de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:57.346 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.345+0000 7f1fd5000640 1 -- 172.21.15.145:0/2784968107 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fd0158790 con 0x7f1fd00d3180 2024-05-09T04:33:57.346 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.345+0000 7f1fd5000640 1 --2- 172.21.15.145:0/2784968107 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1fd00d3180 0x7f1fd015a6b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1fc8034040 tx=0x7f1fc8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:57.347 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.346+0000 7f1fccc00640 1 -- 172.21.15.145:0/2784968107 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc8036050 con 0x7f1fd00d3180 2024-05-09T04:33:57.347 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.346+0000 7f1fccc00640 1 -- 172.21.15.145:0/2784968107 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fc8042070 con 0x7f1fd00d3180 2024-05-09T04:33:57.347 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.346+0000 7f1fccc00640 1 -- 172.21.15.145:0/2784968107 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc8009510 con 0x7f1fd00d3180 2024-05-09T04:33:57.347 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.346+0000 7f1fd5a00640 1 -- 172.21.15.145:0/2784968107 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fd015af50 con 0x7f1fd00d3180 2024-05-09T04:33:57.347 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.346+0000 7f1fd5a00640 1 -- 172.21.15.145:0/2784968107 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fd015b510 con 0x7f1fd00d3180 2024-05-09T04:33:57.347 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.347+0000 7f1fd5a00640 1 -- 172.21.15.145:0/2784968107 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fd006a310 con 0x7f1fd00d3180 2024-05-09T04:33:57.347 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.348+0000 7f1fccc00640 1 -- 172.21.15.145:0/2784968107 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1fc8007c70 con 0x7f1fd00d3180 2024-05-09T04:33:57.349 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.349+0000 7f1fccc00640 1 --2- 172.21.15.145:0/2784968107 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f1fac0762d0 0x7f1fac078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:57.349 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.349+0000 7f1fccc00640 1 -- 172.21.15.145:0/2784968107 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1fc80b8490 con 0x7f1fd00d3180 2024-05-09T04:33:57.349 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.350+0000 7f1fcfe00640 1 --2- 172.21.15.145:0/2784968107 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f1fac0762d0 0x7f1fac078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:57.350 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.350+0000 7f1fcfe00640 1 --2- 172.21.15.145:0/2784968107 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f1fac0762d0 0x7f1fac078790 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0009a30 tx=0x7f1fc0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:57.353 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.354+0000 7f1fccc00640 1 -- 172.21.15.145:0/2784968107 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f1fc8081de0 con 0x7f1fd00d3180 2024-05-09T04:33:57.600 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.601+0000 7f1fd5a00640 1 -- 172.21.15.145:0/2784968107 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f1fd0162b50 con 0x7f1fd00d3180 2024-05-09T04:33:57.607 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.607+0000 7f1fccc00640 1 -- 172.21.15.145:0/2784968107 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+189528 (secure 0 0 0) 0x7f1fc80816e0 con 0x7f1fd00d3180 2024-05-09T04:33:57.609 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:33:57.611 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.611+0000 7f1fd5a00640 1 -- 172.21.15.145:0/2784968107 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f1fac0762d0 msgr2=0x7f1fac078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:57.611 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.611+0000 7f1fd5a00640 1 --2- 172.21.15.145:0/2784968107 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f1fac0762d0 0x7f1fac078790 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0009a30 tx=0x7f1fc0008040 comp rx=0 tx=0).stop 2024-05-09T04:33:57.611 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.611+0000 7f1fd5a00640 1 -- 172.21.15.145:0/2784968107 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1fd00d3180 msgr2=0x7f1fd015a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:57.611 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.611+0000 7f1fd5a00640 1 --2- 172.21.15.145:0/2784968107 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1fd00d3180 0x7f1fd015a6b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1fc8034040 tx=0x7f1fc8004a40 comp rx=0 tx=0).stop 2024-05-09T04:33:57.611 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.611+0000 7f1fd5a00640 1 -- 172.21.15.145:0/2784968107 shutdown_connections 2024-05-09T04:33:57.611 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.611+0000 7f1fd5a00640 1 --2- 172.21.15.145:0/2784968107 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f1fac0762d0 0x7f1fac078790 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:57.611 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.611+0000 7f1fd5a00640 1 --2- 172.21.15.145:0/2784968107 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1fd00d3180 0x7f1fd015a6b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:57.611 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.611+0000 7f1fd5a00640 1 --2- 172.21.15.145:0/2784968107 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fd00d22f0 0x7f1fd0158250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:57.611 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.611+0000 7f1fd5a00640 1 --2- 172.21.15.145:0/2784968107 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1fd00b4760 0x7f1fd015de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:57.611 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.611+0000 7f1fd5a00640 1 -- 172.21.15.145:0/2784968107 >> 172.21.15.145:0/2784968107 conn(0x7f1fd00d4fb0 msgr2=0x7f1fd00b2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:57.611 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.611+0000 7f1fd5a00640 1 -- 172.21.15.145:0/2784968107 shutdown_connections 2024-05-09T04:33:57.612 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.611+0000 7f1fd5a00640 1 -- 172.21.15.145:0/2784968107 wait complete. 2024-05-09T04:33:57.628 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":5,"flags":0,"active_gid":4104,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6832","nonce":2725855396},{"type":"v1","addr":"172.21.15.145:6833","nonce":2725855396}]},"active_addr":"172.21.15.145:6833/2725855396","active_change":"2024-05-09T04:33:31.732594+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4108,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.145:0","nonce":1658670739}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.145:0","nonce":3420781009}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.145:0","nonce":1994309777}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.145:0","nonce":3479260548}]}]} 2024-05-09T04:33:57.632 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-05-09T04:33:57.632 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-05-09T04:33:57.632 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T04:33:57.820 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.820+0000 7fd289800640 1 Processor -- start 2024-05-09T04:33:57.820 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.820+0000 7fd289800640 1 -- start start 2024-05-09T04:33:57.823 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.822+0000 7fd289800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fd2841572c0 0x7fd28415deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:57.823 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.822+0000 7fd289800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd2840cff30 0x7fd2840d0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:57.823 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.822+0000 7fd289800640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2840d0890 con 0x7fd2840ae710 2024-05-09T04:33:57.823 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.822+0000 7fd289800640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fd2840b6290 con 0x7fd2841572c0 2024-05-09T04:33:57.823 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.822+0000 7fd289800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fd284161930 con 0x7fd2840cff30 2024-05-09T04:33:57.823 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.823+0000 7fd283e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fd2841572c0 0x7fd28415deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:57.823 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.823+0000 7fd283e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fd2841572c0 0x7fd28415deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:44992/0 (socket says 172.21.15.145:44992) 2024-05-09T04:33:57.823 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.823+0000 7fd283400640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7fd2840ae710 0x7fd284161220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.145:42450/0 (socket says 172.21.15.145:42450) 2024-05-09T04:33:57.823 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.823+0000 7fd283e00640 1 -- 172.21.15.145:0/4014546274 learned_addr learned my addr 172.21.15.145:0/4014546274 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:57.823 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.823+0000 7fd288e00640 1 --2- 172.21.15.145:0/4014546274 >> v2:172.21.15.192:3300/0 conn(0x7fd2840cff30 0x7fd2840d0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:57.823 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.823+0000 7fd283e00640 1 -- 172.21.15.145:0/4014546274 >> v2:172.21.15.192:3300/0 conn(0x7fd2840cff30 msgr2=0x7fd2840d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:57.823 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.823+0000 7fd283e00640 1 --2- 172.21.15.145:0/4014546274 >> v2:172.21.15.192:3300/0 conn(0x7fd2840cff30 0x7fd2840d0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:57.824 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.823+0000 7fd283e00640 1 -- 172.21.15.145:0/4014546274 >> v1:172.21.15.192:6789/0 conn(0x7fd2840ae710 legacy=0x7fd284161220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:57.824 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.823+0000 7fd283e00640 1 -- 172.21.15.145:0/4014546274 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2840aecd0 con 0x7fd2841572c0 2024-05-09T04:33:57.824 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.824+0000 7fd283e00640 1 --2- 172.21.15.145:0/4014546274 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd2841572c0 0x7fd28415deb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd2840cd670 tx=0x7fd27402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ae8cf1d69495ae0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:57.824 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.825+0000 7fd282a00640 1 -- 172.21.15.145:0/4014546274 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd274036070 con 0x7fd2841572c0 2024-05-09T04:33:57.825 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.825+0000 7fd282a00640 1 -- 172.21.15.145:0/4014546274 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd274031600 con 0x7fd2841572c0 2024-05-09T04:33:57.825 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.825+0000 7fd282a00640 1 -- 172.21.15.145:0/4014546274 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2740319d0 con 0x7fd2841572c0 2024-05-09T04:33:57.825 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.825+0000 7fd289800640 1 -- 172.21.15.145:0/4014546274 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd2841572c0 msgr2=0x7fd28415deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:57.825 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.825+0000 7fd289800640 1 --2- 172.21.15.145:0/4014546274 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd2841572c0 0x7fd28415deb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd2840cd670 tx=0x7fd27402a730 comp rx=0 tx=0).stop 2024-05-09T04:33:57.826 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.825+0000 7fd289800640 1 -- 172.21.15.145:0/4014546274 shutdown_connections 2024-05-09T04:33:57.826 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.825+0000 7fd289800640 1 --2- 172.21.15.145:0/4014546274 >> v2:172.21.15.192:3300/0 conn(0x7fd2840cff30 0x7fd2840d0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:57.826 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.825+0000 7fd289800640 1 --2- 172.21.15.145:0/4014546274 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd2841572c0 0x7fd28415deb0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:57.826 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.825+0000 7fd289800640 1 -- 172.21.15.145:0/4014546274 >> 172.21.15.145:0/4014546274 conn(0x7fd28406c8a0 msgr2=0x7fd28406ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:57.826 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.826+0000 7fd289800640 1 -- 172.21.15.145:0/4014546274 shutdown_connections 2024-05-09T04:33:57.826 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.826+0000 7fd289800640 1 -- 172.21.15.145:0/4014546274 wait complete. 2024-05-09T04:33:57.827 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.827+0000 7fd289800640 1 Processor -- start 2024-05-09T04:33:57.827 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.827+0000 7fd289800640 1 -- start start 2024-05-09T04:33:57.829 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.828+0000 7fd289800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2840ae710 0x7fd284152250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:57.829 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.828+0000 7fd289800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd2840cff30 0x7fd2841e7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:57.829 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.828+0000 7fd289800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd2841572c0 0x7fd28409c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:57.829 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.828+0000 7fd289800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd284160a20 con 0x7fd2841572c0 2024-05-09T04:33:57.829 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.828+0000 7fd289800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd2840af290 con 0x7fd2840ae710 2024-05-09T04:33:57.829 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.828+0000 7fd289800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd284160d00 con 0x7fd2840cff30 2024-05-09T04:33:57.830 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.828+0000 7fd288e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd2841572c0 0x7fd28409c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:57.830 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.829+0000 7fd288e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd2841572c0 0x7fd28409c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:44996/0 (socket says 172.21.15.145:44996) 2024-05-09T04:33:57.830 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.829+0000 7fd288e00640 1 -- 172.21.15.145:0/1166948826 learned_addr learned my addr 172.21.15.145:0/1166948826 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:57.830 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.829+0000 7fd283e00640 1 --2- 172.21.15.145:0/1166948826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2840ae710 0x7fd284152250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:57.830 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.829+0000 7fd288e00640 1 -- 172.21.15.145:0/1166948826 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd2840cff30 msgr2=0x7fd2841e7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:57.830 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.829+0000 7fd288e00640 1 --2- 172.21.15.145:0/1166948826 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd2840cff30 0x7fd2841e7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:57.830 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.829+0000 7fd288e00640 1 -- 172.21.15.145:0/1166948826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2840ae710 msgr2=0x7fd284152250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:57.830 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.829+0000 7fd288e00640 1 --2- 172.21.15.145:0/1166948826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2840ae710 0x7fd284152250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:57.830 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.829+0000 7fd288e00640 1 -- 172.21.15.145:0/1166948826 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2841e7c80 con 0x7fd2841572c0 2024-05-09T04:33:57.830 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.829+0000 7fd283e00640 1 --2- 172.21.15.145:0/1166948826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2840ae710 0x7fd284152250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:33:57.830 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.829+0000 7fd288e00640 1 --2- 172.21.15.145:0/1166948826 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd2841572c0 0x7fd28409c050 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd27c00d800 tx=0x7fd27c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:57.830 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.829+0000 7fd27be00640 1 -- 172.21.15.145:0/1166948826 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd27c00b6c0 con 0x7fd2841572c0 2024-05-09T04:33:57.830 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.829+0000 7fd27be00640 1 -- 172.21.15.145:0/1166948826 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd27c019070 con 0x7fd2841572c0 2024-05-09T04:33:57.830 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.829+0000 7fd27be00640 1 -- 172.21.15.145:0/1166948826 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd27c00bac0 con 0x7fd2841572c0 2024-05-09T04:33:57.830 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.829+0000 7fd289800640 1 -- 172.21.15.145:0/1166948826 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2841e7f60 con 0x7fd2841572c0 2024-05-09T04:33:57.831 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.830+0000 7fd289800640 1 -- 172.21.15.145:0/1166948826 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2841e8520 con 0x7fd2841572c0 2024-05-09T04:33:57.831 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.831+0000 7fd289800640 1 -- 172.21.15.145:0/1166948826 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd28415dd30 con 0x7fd2841572c0 2024-05-09T04:33:57.831 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.832+0000 7fd27be00640 1 -- 172.21.15.145:0/1166948826 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd27c00bc60 con 0x7fd2841572c0 2024-05-09T04:33:57.832 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.833+0000 7fd27be00640 1 --2- 172.21.15.145:0/1166948826 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fd2600762d0 0x7fd260078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:57.832 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.833+0000 7fd27be00640 1 -- 172.21.15.145:0/1166948826 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd27c097b20 con 0x7fd2841572c0 2024-05-09T04:33:57.832 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.833+0000 7fd283e00640 1 --2- 172.21.15.145:0/1166948826 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fd2600762d0 0x7fd260078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:57.833 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.834+0000 7fd283e00640 1 --2- 172.21.15.145:0/1166948826 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fd2600762d0 0x7fd260078790 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd274033000 tx=0x7fd274009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:57.837 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:57.837+0000 7fd27be00640 1 -- 172.21.15.145:0/1166948826 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd27c0613e0 con 0x7fd2841572c0 2024-05-09T04:33:58.043 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.043+0000 7fd289800640 1 -- 172.21.15.145:0/1166948826 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd28406a310 con 0x7fd2841572c0 2024-05-09T04:33:58.044 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.045+0000 7fd27be00640 1 -- 172.21.15.145:0/1166948826 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16653 (secure 0 0 0) 0x7fd27c060ce0 con 0x7fd2841572c0 2024-05-09T04:33:58.045 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:33:58.045 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":20,"fsid":"957ee6c1-073b-44cf-9b93-9491b3fdfc1d","created":"2024-05-09T04:33:26.544367+0000","modified":"2024-05-09T04:33:53.178020+0000","last_up_change":"2024-05-09T04:33:39.039318+0000","last_in_change":"2024-05-09T04:33:29.902361+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T04:33:39.795344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T04:33:49.651743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T04:33:50.596065+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22e550ad-847b-4952-80e6-63d859ed1a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6801","nonce":3182572912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6803","nonce":3182572912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6807","nonce":3182572912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6805","nonce":3182572912}]},"public_addr":"172.21.15.145:6801/3182572912","cluster_addr":"172.21.15.145:6803/3182572912","heartbeat_back_addr":"172.21.15.145:6807/3182572912","heartbeat_front_addr":"172.21.15.145:6805/3182572912","state":["exists","up"]},{"osd":1,"uuid":"0f6b9fca-0c41-4145-8d74-598ddb42f2a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6825","nonce":1845225555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6827","nonce":1845225555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6831","nonce":1845225555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6829","nonce":1845225555}]},"public_addr":"172.21.15.145:6825/1845225555","cluster_addr":"172.21.15.145:6827/1845225555","heartbeat_back_addr":"172.21.15.145:6831/1845225555","heartbeat_front_addr":"172.21.15.145:6829/1845225555","state":["exists","up"]},{"osd":2,"uuid":"b3b1f67e-5790-483f-8610-e05a5bce4c30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6817","nonce":2168958993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6819","nonce":2168958993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6823","nonce":2168958993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6821","nonce":2168958993}]},"public_addr":"172.21.15.145:6817/2168958993","cluster_addr":"172.21.15.145:6819/2168958993","heartbeat_back_addr":"172.21.15.145:6823/2168958993","heartbeat_front_addr":"172.21.15.145:6821/2168958993","state":["exists","up"]},{"osd":3,"uuid":"884787a5-e037-4d31-9aae-743030eba4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6809","nonce":337009826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6811","nonce":337009826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6815","nonce":337009826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6813","nonce":337009826}]},"public_addr":"172.21.15.145:6809/337009826","cluster_addr":"172.21.15.145:6811/337009826","heartbeat_back_addr":"172.21.15.145:6815/337009826","heartbeat_front_addr":"172.21.15.145:6813/337009826","state":["exists","up"]},{"osd":4,"uuid":"0759fc1e-c5fc-4a2f-86bb-e8eccd83f833","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6818","nonce":3117700995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6821","nonce":3117700995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6827","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6829","nonce":3117700995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6823","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6825","nonce":3117700995}]},"public_addr":"172.21.15.192:6818/3117700995","cluster_addr":"172.21.15.192:6821/3117700995","heartbeat_back_addr":"172.21.15.192:6829/3117700995","heartbeat_front_addr":"172.21.15.192:6825/3117700995","state":["exists","up"]},{"osd":5,"uuid":"9443cc22-9379-46e3-9173-98b592bf2c16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6801","nonce":1328373268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6803","nonce":1328373268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6807","nonce":1328373268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6805","nonce":1328373268}]},"public_addr":"172.21.15.192:6801/1328373268","cluster_addr":"172.21.15.192:6803/1328373268","heartbeat_back_addr":"172.21.15.192:6807/1328373268","heartbeat_front_addr":"172.21.15.192:6805/1328373268","state":["exists","up"]},{"osd":6,"uuid":"d05a3311-505a-45b5-85b9-aa3113e96987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6817","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6820","nonce":1868011057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6824","nonce":1868011057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6831","nonce":1868011057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6828","nonce":1868011057}]},"public_addr":"172.21.15.192:6820/1868011057","cluster_addr":"172.21.15.192:6824/1868011057","heartbeat_back_addr":"172.21.15.192:6831/1868011057","heartbeat_front_addr":"172.21.15.192:6828/1868011057","state":["exists","up"]},{"osd":7,"uuid":"c7c23dbc-2029-43ec-9f53-25f3d58019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6809","nonce":3368512157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6811","nonce":3368512157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6815","nonce":3368512157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6813","nonce":3368512157}]},"public_addr":"172.21.15.192:6809/3368512157","cluster_addr":"172.21.15.192:6811/3368512157","heartbeat_back_addr":"172.21.15.192:6815/3368512157","heartbeat_front_addr":"172.21.15.192:6813/3368512157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.091188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.032567+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:34.797468+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.228839+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.622400+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.473103+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.566092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.349395+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T04:33:58.048 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.048+0000 7fd289800640 1 -- 172.21.15.145:0/1166948826 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fd2600762d0 msgr2=0x7fd260078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.048 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.048+0000 7fd289800640 1 --2- 172.21.15.145:0/1166948826 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fd2600762d0 0x7fd260078790 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd274033000 tx=0x7fd274009000 comp rx=0 tx=0).stop 2024-05-09T04:33:58.048 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.048+0000 7fd289800640 1 -- 172.21.15.145:0/1166948826 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd2841572c0 msgr2=0x7fd28409c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.048 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.048+0000 7fd289800640 1 --2- 172.21.15.145:0/1166948826 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd2841572c0 0x7fd28409c050 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd27c00d800 tx=0x7fd27c00dcc0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.048 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.048+0000 7fd289800640 1 -- 172.21.15.145:0/1166948826 shutdown_connections 2024-05-09T04:33:58.048 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.048+0000 7fd289800640 1 --2- 172.21.15.145:0/1166948826 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fd2600762d0 0x7fd260078790 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.048 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.048+0000 7fd289800640 1 --2- 172.21.15.145:0/1166948826 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd2841572c0 0x7fd28409c050 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.048 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.048+0000 7fd289800640 1 --2- 172.21.15.145:0/1166948826 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd2840cff30 0x7fd2841e7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.048 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.048+0000 7fd289800640 1 --2- 172.21.15.145:0/1166948826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2840ae710 0x7fd284152250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.048 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.048+0000 7fd289800640 1 -- 172.21.15.145:0/1166948826 >> 172.21.15.145:0/1166948826 conn(0x7fd28406c8a0 msgr2=0x7fd28406e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:58.048 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.048+0000 7fd289800640 1 -- 172.21.15.145:0/1166948826 shutdown_connections 2024-05-09T04:33:58.048 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.048+0000 7fd289800640 1 -- 172.21.15.145:0/1166948826 wait complete. 2024-05-09T04:33:58.066 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-05-09T04:33:58.066 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T04:33:58.267 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.267+0000 7f36a6600640 1 Processor -- start 2024-05-09T04:33:58.267 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.267+0000 7f36a6600640 1 -- start start 2024-05-09T04:33:58.269 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.268+0000 7f36a6600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f36a0163270 0x7f36a006e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.269 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.269+0000 7f36a6600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36a00af130 0x7f36a0073560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.269 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.269+0000 7f36a6600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36a00af890 con 0x7f36a0073aa0 2024-05-09T04:33:58.269 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.269+0000 7f36a6600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f36a00afb70 con 0x7f36a0163270 2024-05-09T04:33:58.269 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.269+0000 7f36a6600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f36a0072a80 con 0x7f36a00af130 2024-05-09T04:33:58.269 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.269+0000 7f36a5c00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f36a0073aa0 0x7f36a00b47d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:33904/0 (socket says 172.21.15.145:33904) 2024-05-09T04:33:58.269 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.269+0000 7f36a5c00640 1 -- 172.21.15.145:0/3688904137 learned_addr learned my addr 172.21.15.145:0/3688904137 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:33:58.271 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.269+0000 7f369f400640 1 -- 172.21.15.145:0/3688904137 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1299645215 0 0) 0x7f36a00af890 con 0x7f36a0073aa0 2024-05-09T04:33:58.271 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.269+0000 7f369fe00640 1 --2- 172.21.15.145:0/3688904137 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36a00af130 0x7f36a0073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:58.271 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.269+0000 7f36a5200640 1 --2- 172.21.15.145:0/3688904137 >> v2:172.21.15.192:3300/0 conn(0x7f36a0163270 0x7f36a006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:58.271 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.269+0000 7f369f400640 1 -- 172.21.15.145:0/3688904137 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3688003520 con 0x7f36a0073aa0 2024-05-09T04:33:58.271 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.270+0000 7f369f400640 1 -- 172.21.15.145:0/3688904137 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2782636502 0 0) 0x7f3688003520 con 0x7f36a0073aa0 2024-05-09T04:33:58.271 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.270+0000 7f369f400640 1 -- 172.21.15.145:0/3688904137 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36a00af130 msgr2=0x7f36a0073560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:58.271 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.270+0000 7f369f400640 1 --2- 172.21.15.145:0/3688904137 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36a00af130 0x7f36a0073560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.271 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.270+0000 7f369f400640 1 -- 172.21.15.145:0/3688904137 >> v2:172.21.15.192:3300/0 conn(0x7f36a0163270 msgr2=0x7f36a006e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:58.271 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.270+0000 7f369f400640 1 --2- 172.21.15.145:0/3688904137 >> v2:172.21.15.192:3300/0 conn(0x7f36a0163270 0x7f36a006e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.271 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.270+0000 7f369f400640 1 -- 172.21.15.145:0/3688904137 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36a0072d60 con 0x7f36a0073aa0 2024-05-09T04:33:58.271 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.270+0000 7f369f400640 1 -- 172.21.15.145:0/3688904137 <== mon.0 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3474413261 0 0) 0x7f3698001a90 con 0x7f36a0073aa0 2024-05-09T04:33:58.271 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.270+0000 7f36a5200640 1 --2- 172.21.15.145:0/3688904137 >> v2:172.21.15.192:3300/0 conn(0x7f36a0163270 0x7f36a006e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:33:58.271 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.270+0000 7f369f400640 1 -- 172.21.15.145:0/3688904137 >> v1:172.21.15.145:6789/0 conn(0x7f36a0073aa0 legacy=0x7f36a00b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.271 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.270+0000 7f369f400640 1 --2- 172.21.15.145:0/3688904137 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3688003a50 0x7f3688003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.272 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.270+0000 7f369f400640 1 --2- 172.21.15.145:0/3688904137 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36a00af130 0x7f3688004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.272 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.270+0000 7f369f400640 1 --2- 172.21.15.145:0/3688904137 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3688004d40 0x7f36880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.272 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.270+0000 7f369f400640 1 -- 172.21.15.145:0/3688904137 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f36a00afb70 con 0x7f3688003a50 2024-05-09T04:33:58.272 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.270+0000 7f369f400640 1 -- 172.21.15.145:0/3688904137 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f36a0072a80 con 0x7f3688004d40 2024-05-09T04:33:58.272 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.270+0000 7f369f400640 1 -- 172.21.15.145:0/3688904137 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f368800bb70 con 0x7f36a00af130 2024-05-09T04:33:58.272 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.270+0000 7f36a5200640 1 --2- 172.21.15.145:0/3688904137 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3688003a50 0x7f3688003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:58.272 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.270+0000 7f36a5200640 1 -- 172.21.15.145:0/3688904137 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36a00af130 msgr2=0x7f3688004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:58.272 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.270+0000 7f36a5200640 1 --2- 172.21.15.145:0/3688904137 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36a00af130 0x7f3688004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.272 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.270+0000 7f36a5200640 1 -- 172.21.15.145:0/3688904137 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3688004d40 msgr2=0x7f36880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:58.272 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.270+0000 7f36a5200640 1 --2- 172.21.15.145:0/3688904137 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3688004d40 0x7f36880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.272 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.271+0000 7f36a5200640 1 -- 172.21.15.145:0/3688904137 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3688003520 con 0x7f3688003a50 2024-05-09T04:33:58.273 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.271+0000 7f36a5200640 1 --2- 172.21.15.145:0/3688904137 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3688003a50 0x7f3688003e70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f36940098a0 tx=0x7f3688005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49e380d346b08861 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:58.273 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.272+0000 7f369f400640 1 -- 172.21.15.145:0/3688904137 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369400f040 con 0x7f3688003a50 2024-05-09T04:33:58.274 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.272+0000 7f369f400640 1 -- 172.21.15.145:0/3688904137 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f369402ad20 con 0x7f3688003a50 2024-05-09T04:33:58.274 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.272+0000 7f369f400640 1 -- 172.21.15.145:0/3688904137 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3694004250 con 0x7f3688003a50 2024-05-09T04:33:58.274 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.272+0000 7f36a6600640 1 -- 172.21.15.145:0/3688904137 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3688003a50 msgr2=0x7f3688003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.274 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.272+0000 7f36a6600640 1 --2- 172.21.15.145:0/3688904137 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3688003a50 0x7f3688003e70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f36940098a0 tx=0x7f3688005830 comp rx=0 tx=0).stop 2024-05-09T04:33:58.274 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.272+0000 7f36a5c00640 1 -- 172.21.15.145:0/3688904137 reap_dead start 2024-05-09T04:33:58.274 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.272+0000 7f36a6600640 1 -- 172.21.15.145:0/3688904137 shutdown_connections 2024-05-09T04:33:58.274 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.272+0000 7f36a6600640 1 -- 172.21.15.145:0/3688904137 >> 172.21.15.145:0/3688904137 conn(0x7f36a006c8a0 msgr2=0x7f36a006ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:58.274 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.273+0000 7f36a6600640 1 -- 172.21.15.145:0/3688904137 shutdown_connections 2024-05-09T04:33:58.274 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.273+0000 7f36a6600640 1 -- 172.21.15.145:0/3688904137 wait complete. 2024-05-09T04:33:58.276 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.274+0000 7f36a6600640 1 Processor -- start 2024-05-09T04:33:58.276 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.274+0000 7f36a6600640 1 -- start start 2024-05-09T04:33:58.277 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.275+0000 7f36a6600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3688003a50 0x7f36a0164df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.277 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.275+0000 7f36a6600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36a015bf30 0x7f36a015f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.277 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.275+0000 7f36a6600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36a0165cd0 0x7f36a0166180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.277 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.275+0000 7f36a6600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f36a0163650 con 0x7f36a015bf30 2024-05-09T04:33:58.277 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.275+0000 7f36a6600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f36a00af5b0 con 0x7f36a0165cd0 2024-05-09T04:33:58.277 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.275+0000 7f36a6600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f36a01637d0 con 0x7f3688003a50 2024-05-09T04:33:58.277 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.276+0000 7f369fe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36a015bf30 0x7f36a015f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:58.277 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.276+0000 7f369fe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36a015bf30 0x7f36a015f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:45014/0 (socket says 172.21.15.145:45014) 2024-05-09T04:33:58.277 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.276+0000 7f369fe00640 1 -- 172.21.15.145:0/1339034254 learned_addr learned my addr 172.21.15.145:0/1339034254 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:58.277 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.276+0000 7f36a5200640 1 --2- 172.21.15.145:0/1339034254 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3688003a50 0x7f36a0164df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:58.277 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.276+0000 7f369fe00640 1 -- 172.21.15.145:0/1339034254 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3688003a50 msgr2=0x7f36a0164df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.277 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.276+0000 7f369fe00640 1 --2- 172.21.15.145:0/1339034254 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3688003a50 0x7f36a0164df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.277 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.276+0000 7f369fe00640 1 -- 172.21.15.145:0/1339034254 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36a0165cd0 msgr2=0x7f36a0166180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.277 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.276+0000 7f369fe00640 1 --2- 172.21.15.145:0/1339034254 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36a0165cd0 0x7f36a0166180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.277 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.276+0000 7f369fe00640 1 -- 172.21.15.145:0/1339034254 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36a015cfd0 con 0x7f36a015bf30 2024-05-09T04:33:58.278 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.276+0000 7f36a5200640 1 --2- 172.21.15.145:0/1339034254 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3688003a50 0x7f36a0164df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:33:58.278 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.276+0000 7f369fe00640 1 --2- 172.21.15.145:0/1339034254 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36a015bf30 0x7f36a015f8d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f368c00e7f0 tx=0x7f368c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:58.278 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.277+0000 7f369cc00640 1 -- 172.21.15.145:0/1339034254 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368c017070 con 0x7f36a015bf30 2024-05-09T04:33:58.278 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.277+0000 7f369cc00640 1 -- 172.21.15.145:0/1339034254 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f368c00c940 con 0x7f36a015bf30 2024-05-09T04:33:58.278 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.277+0000 7f36a6600640 1 -- 172.21.15.145:0/1339034254 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36a0163d50 con 0x7f36a015bf30 2024-05-09T04:33:58.278 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.277+0000 7f369cc00640 1 -- 172.21.15.145:0/1339034254 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368c00cd10 con 0x7f36a015bf30 2024-05-09T04:33:58.278 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.277+0000 7f36a6600640 1 -- 172.21.15.145:0/1339034254 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36a0164310 con 0x7f36a015bf30 2024-05-09T04:33:58.278 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.278+0000 7f36a6600640 1 -- 172.21.15.145:0/1339034254 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36a006a310 con 0x7f36a015bf30 2024-05-09T04:33:58.279 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.279+0000 7f369cc00640 1 -- 172.21.15.145:0/1339034254 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f368c01a070 con 0x7f36a015bf30 2024-05-09T04:33:58.280 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.280+0000 7f369cc00640 1 --2- 172.21.15.145:0/1339034254 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f3680076300 0x7f36800787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.280 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.280+0000 7f369cc00640 1 -- 172.21.15.145:0/1339034254 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f368c097460 con 0x7f36a015bf30 2024-05-09T04:33:58.280 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.280+0000 7f36a5200640 1 --2- 172.21.15.145:0/1339034254 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f3680076300 0x7f36800787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:58.280 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.281+0000 7f36a5200640 1 --2- 172.21.15.145:0/1339034254 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f3680076300 0x7f36800787c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3694033000 tx=0x7f369402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:58.286 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.284+0000 7f369cc00640 1 -- 172.21.15.145:0/1339034254 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f368c060db0 con 0x7f36a015bf30 2024-05-09T04:33:58.492 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.492+0000 7f36a6600640 1 -- 172.21.15.145:0/1339034254 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f36a00aec50 con 0x7f36a015bf30 2024-05-09T04:33:58.494 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.494+0000 7f369cc00640 1 -- 172.21.15.145:0/1339034254 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16653 (secure 0 0 0) 0x7f368c0606b0 con 0x7f36a015bf30 2024-05-09T04:33:58.495 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:33:58.495 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":20,"fsid":"957ee6c1-073b-44cf-9b93-9491b3fdfc1d","created":"2024-05-09T04:33:26.544367+0000","modified":"2024-05-09T04:33:53.178020+0000","last_up_change":"2024-05-09T04:33:39.039318+0000","last_in_change":"2024-05-09T04:33:29.902361+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T04:33:39.795344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T04:33:49.651743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T04:33:50.596065+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22e550ad-847b-4952-80e6-63d859ed1a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6801","nonce":3182572912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6803","nonce":3182572912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6807","nonce":3182572912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6805","nonce":3182572912}]},"public_addr":"172.21.15.145:6801/3182572912","cluster_addr":"172.21.15.145:6803/3182572912","heartbeat_back_addr":"172.21.15.145:6807/3182572912","heartbeat_front_addr":"172.21.15.145:6805/3182572912","state":["exists","up"]},{"osd":1,"uuid":"0f6b9fca-0c41-4145-8d74-598ddb42f2a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6825","nonce":1845225555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6827","nonce":1845225555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6831","nonce":1845225555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6829","nonce":1845225555}]},"public_addr":"172.21.15.145:6825/1845225555","cluster_addr":"172.21.15.145:6827/1845225555","heartbeat_back_addr":"172.21.15.145:6831/1845225555","heartbeat_front_addr":"172.21.15.145:6829/1845225555","state":["exists","up"]},{"osd":2,"uuid":"b3b1f67e-5790-483f-8610-e05a5bce4c30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6817","nonce":2168958993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6819","nonce":2168958993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6823","nonce":2168958993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6821","nonce":2168958993}]},"public_addr":"172.21.15.145:6817/2168958993","cluster_addr":"172.21.15.145:6819/2168958993","heartbeat_back_addr":"172.21.15.145:6823/2168958993","heartbeat_front_addr":"172.21.15.145:6821/2168958993","state":["exists","up"]},{"osd":3,"uuid":"884787a5-e037-4d31-9aae-743030eba4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6809","nonce":337009826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6811","nonce":337009826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6815","nonce":337009826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6813","nonce":337009826}]},"public_addr":"172.21.15.145:6809/337009826","cluster_addr":"172.21.15.145:6811/337009826","heartbeat_back_addr":"172.21.15.145:6815/337009826","heartbeat_front_addr":"172.21.15.145:6813/337009826","state":["exists","up"]},{"osd":4,"uuid":"0759fc1e-c5fc-4a2f-86bb-e8eccd83f833","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6818","nonce":3117700995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6821","nonce":3117700995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6827","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6829","nonce":3117700995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6823","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6825","nonce":3117700995}]},"public_addr":"172.21.15.192:6818/3117700995","cluster_addr":"172.21.15.192:6821/3117700995","heartbeat_back_addr":"172.21.15.192:6829/3117700995","heartbeat_front_addr":"172.21.15.192:6825/3117700995","state":["exists","up"]},{"osd":5,"uuid":"9443cc22-9379-46e3-9173-98b592bf2c16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6801","nonce":1328373268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6803","nonce":1328373268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6807","nonce":1328373268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6805","nonce":1328373268}]},"public_addr":"172.21.15.192:6801/1328373268","cluster_addr":"172.21.15.192:6803/1328373268","heartbeat_back_addr":"172.21.15.192:6807/1328373268","heartbeat_front_addr":"172.21.15.192:6805/1328373268","state":["exists","up"]},{"osd":6,"uuid":"d05a3311-505a-45b5-85b9-aa3113e96987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6817","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6820","nonce":1868011057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6824","nonce":1868011057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6831","nonce":1868011057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6828","nonce":1868011057}]},"public_addr":"172.21.15.192:6820/1868011057","cluster_addr":"172.21.15.192:6824/1868011057","heartbeat_back_addr":"172.21.15.192:6831/1868011057","heartbeat_front_addr":"172.21.15.192:6828/1868011057","state":["exists","up"]},{"osd":7,"uuid":"c7c23dbc-2029-43ec-9f53-25f3d58019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6809","nonce":3368512157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6811","nonce":3368512157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6815","nonce":3368512157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6813","nonce":3368512157}]},"public_addr":"172.21.15.192:6809/3368512157","cluster_addr":"172.21.15.192:6811/3368512157","heartbeat_back_addr":"172.21.15.192:6815/3368512157","heartbeat_front_addr":"172.21.15.192:6813/3368512157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.091188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.032567+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:34.797468+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.228839+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.622400+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.473103+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.566092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.349395+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T04:33:58.497 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.497+0000 7f36a6600640 1 -- 172.21.15.145:0/1339034254 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f3680076300 msgr2=0x7f36800787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.497 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.497+0000 7f36a6600640 1 --2- 172.21.15.145:0/1339034254 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f3680076300 0x7f36800787c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3694033000 tx=0x7f369402e000 comp rx=0 tx=0).stop 2024-05-09T04:33:58.497 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.497+0000 7f36a6600640 1 -- 172.21.15.145:0/1339034254 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36a015bf30 msgr2=0x7f36a015f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.498 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.497+0000 7f36a6600640 1 --2- 172.21.15.145:0/1339034254 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36a015bf30 0x7f36a015f8d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f368c00e7f0 tx=0x7f368c00ecb0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.498 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.497+0000 7f36a6600640 1 -- 172.21.15.145:0/1339034254 shutdown_connections 2024-05-09T04:33:58.498 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.497+0000 7f36a6600640 1 --2- 172.21.15.145:0/1339034254 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f3680076300 0x7f36800787c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.498 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.497+0000 7f36a6600640 1 --2- 172.21.15.145:0/1339034254 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36a0165cd0 0x7f36a0166180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.498 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.497+0000 7f36a6600640 1 --2- 172.21.15.145:0/1339034254 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36a015bf30 0x7f36a015f8d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.498 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.497+0000 7f36a6600640 1 --2- 172.21.15.145:0/1339034254 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3688003a50 0x7f36a0164df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.498 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.497+0000 7f36a6600640 1 -- 172.21.15.145:0/1339034254 >> 172.21.15.145:0/1339034254 conn(0x7f36a006c8a0 msgr2=0x7f36a006f390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:58.498 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.498+0000 7f36a6600640 1 -- 172.21.15.145:0/1339034254 shutdown_connections 2024-05-09T04:33:58.498 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.498+0000 7f36a6600640 1 -- 172.21.15.145:0/1339034254 wait complete. 2024-05-09T04:33:58.516 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-05-09T04:33:58.516 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-05-09T04:33:58.516 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-05-09T04:33:58.517 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-05-09T04:33:58.517 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-05-09T04:33:58.517 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-05-09T04:33:58.518 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-05-09T04:33:58.518 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-05-09T04:33:58.794 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.795+0000 7f8ae4200640 1 Processor -- start 2024-05-09T04:33:58.795 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.795+0000 7f8ae4200640 1 -- start start 2024-05-09T04:33:58.797 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.797+0000 7f8ae4200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f8ad403c090 0x7f8ad40e7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.798 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.798+0000 7f8ae4200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f8ad4053ca0 0x7f8ad40540f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.798 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.798+0000 7f8ae4200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ad4058800 con 0x7f8ad40e3150 2024-05-09T04:33:58.798 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.798+0000 7f8ae4200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f8ad4007b60 con 0x7f8ad403c090 2024-05-09T04:33:58.798 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.798+0000 7f8ae4200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f8ad4058ae0 con 0x7f8ad4053ca0 2024-05-09T04:33:58.798 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.798+0000 7f8ae1800640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f8ad40e3150 0x7f8ad4058040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:33920/0 (socket says 172.21.15.145:33920) 2024-05-09T04:33:58.798 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.798+0000 7f8ae1800640 1 -- 172.21.15.145:0/2367892878 learned_addr learned my addr 172.21.15.145:0/2367892878 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:33:58.798 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.798+0000 7f8ae0e00640 1 --2- 172.21.15.145:0/2367892878 >> v2:172.21.15.145:3300/0 conn(0x7f8ad403c090 0x7f8ad40e7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:58.799 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.799+0000 7f8ae2200640 1 --2- 172.21.15.145:0/2367892878 >> v2:172.21.15.192:3300/0 conn(0x7f8ad4053ca0 0x7f8ad40540f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:58.799 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.800+0000 7f8ae0e00640 1 -- 172.21.15.145:0/2367892878 >> v2:172.21.15.192:3300/0 conn(0x7f8ad4053ca0 msgr2=0x7f8ad40540f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:58.799 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.800+0000 7f8ae0e00640 1 --2- 172.21.15.145:0/2367892878 >> v2:172.21.15.192:3300/0 conn(0x7f8ad4053ca0 0x7f8ad40540f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.799 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.800+0000 7f8ae0e00640 1 -- 172.21.15.145:0/2367892878 >> v1:172.21.15.145:6789/0 conn(0x7f8ad40e3150 legacy=0x7f8ad4058040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.800 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.800+0000 7f8ae0e00640 1 -- 172.21.15.145:0/2367892878 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ad4006350 con 0x7f8ad403c090 2024-05-09T04:33:58.800 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.799+0000 7f8adbe00640 1 -- 172.21.15.145:0/2367892878 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2774550014 0 0) 0x7f8ad4058800 con 0x7f8ad40e3150 2024-05-09T04:33:58.800 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.801+0000 7f8ae0e00640 1 --2- 172.21.15.145:0/2367892878 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ad403c090 0x7f8ad40e7770 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4008650 tx=0x7f8ad002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4de017472eb16600 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:58.801 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.801+0000 7f8adbe00640 1 -- 172.21.15.145:0/2367892878 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad0037070 con 0x7f8ad403c090 2024-05-09T04:33:58.801 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.801+0000 7f8adbe00640 1 -- 172.21.15.145:0/2367892878 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ad0032440 con 0x7f8ad403c090 2024-05-09T04:33:58.801 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.801+0000 7f8adbe00640 1 -- 172.21.15.145:0/2367892878 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad0032810 con 0x7f8ad403c090 2024-05-09T04:33:58.802 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.802+0000 7f893ee00640 1 Processor -- start 2024-05-09T04:33:58.802 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.802+0000 7f8ae4200640 1 -- 172.21.15.145:0/2367892878 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ad403c090 msgr2=0x7f8ad40e7770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.802 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.803+0000 7f893ee00640 1 -- start start 2024-05-09T04:33:58.803 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.802+0000 7f8ae4200640 1 --2- 172.21.15.145:0/2367892878 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ad403c090 0x7f8ad40e7770 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4008650 tx=0x7f8ad002a1a0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.803 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.803+0000 7f8ae4200640 1 -- 172.21.15.145:0/2367892878 shutdown_connections 2024-05-09T04:33:58.803 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.803+0000 7f8ae4200640 1 --2- 172.21.15.145:0/2367892878 >> v2:172.21.15.192:3300/0 conn(0x7f8ad4053ca0 0x7f8ad40540f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.803 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.803+0000 7f8ae4200640 1 --2- 172.21.15.145:0/2367892878 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ad403c090 0x7f8ad40e7770 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.803 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.803+0000 7f8ae4200640 1 -- 172.21.15.145:0/2367892878 >> 172.21.15.145:0/2367892878 conn(0x7f8ad4058ed0 msgr2=0x7f8ad40592b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:58.803 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.803+0000 7f8ae4200640 1 -- 172.21.15.145:0/2367892878 shutdown_connections 2024-05-09T04:33:58.803 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.803+0000 7f8ae4200640 1 -- 172.21.15.145:0/2367892878 wait complete. 2024-05-09T04:33:58.804 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.803+0000 7f209e200640 1 Processor -- start 2024-05-09T04:33:58.804 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.804+0000 7f893ee00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f89381629e0 0x7f89380d0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.804 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.804+0000 7f893ee00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f89380d15e0 0x7f89380d1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.804 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.804+0000 7f893ee00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89380d4150 con 0x7f89380d1f60 2024-05-09T04:33:58.804 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.804+0000 7f893ee00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f89380b5090 con 0x7f89380d15e0 2024-05-09T04:33:58.804 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.804+0000 7f893ee00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f89380d0c70 con 0x7f89381629e0 2024-05-09T04:33:58.805 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.805+0000 7f893ce00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f89380d1f60 0x7f89380d2eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:33928/0 (socket says 172.21.15.145:33928) 2024-05-09T04:33:58.805 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.805+0000 7f893ce00640 1 -- 172.21.15.145:0/2752705629 learned_addr learned my addr 172.21.15.145:0/2752705629 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:33:58.806 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.806+0000 7f8ae4200640 1 Processor -- start 2024-05-09T04:33:58.806 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.804+0000 7f209e200640 1 -- start start 2024-05-09T04:33:58.806 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.806+0000 7f209e200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f209815f4e0 0x7f209816a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.806 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.806+0000 7f209e200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20980cfda0 con 0x7f20980d3cd0 2024-05-09T04:33:58.806 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.806+0000 7f209e200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20980d0080 con 0x7f209815f100 2024-05-09T04:33:58.806 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.806+0000 7f209e200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f20980b8e70 con 0x7f209815f4e0 2024-05-09T04:33:58.807 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.806+0000 7f8ae4200640 1 -- start start 2024-05-09T04:33:58.807 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.807+0000 7f8936a00640 1 -- 172.21.15.145:0/2752705629 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2123562906 0 0) 0x7f89380d4150 con 0x7f89380d1f60 2024-05-09T04:33:58.807 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.807+0000 7f209d800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f209815f4e0 0x7f209816a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:58.808 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.807+0000 7f8937e00640 1 --2- 172.21.15.145:0/2752705629 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f89381629e0 0x7f89380d0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:58.808 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.807+0000 7f8937400640 1 --2- 172.21.15.145:0/2752705629 >> v2:172.21.15.192:3300/0 conn(0x7f89380d15e0 0x7f89380d1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:58.808 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.807+0000 7f209d800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f209815f4e0 0x7f209816a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.145:55242/0 (socket says 172.21.15.145:55242) 2024-05-09T04:33:58.808 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.807+0000 7f8936a00640 1 -- 172.21.15.145:0/2752705629 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f891c003520 con 0x7f89380d1f60 2024-05-09T04:33:58.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.808+0000 7f209ce00640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f209815f100 0x7f20980d35c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.145:42460/0 (socket says 172.21.15.145:42460) 2024-05-09T04:33:58.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.808+0000 7f209d800640 1 -- 172.21.15.145:0/1322623822 learned_addr learned my addr 172.21.15.145:0/1322623822 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:58.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.808+0000 7f8ae4200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ad403c090 0x7f8ad40e0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.809+0000 7f8ae4200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ad4053ca0 0x7f8ad416d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.810 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.809+0000 7f8936a00640 1 -- 172.21.15.145:0/2752705629 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 535178479 0 0) 0x7f891c003520 con 0x7f89380d1f60 2024-05-09T04:33:58.810 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.809+0000 7f8ae4200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8ad40e3150 0x7f8ad417bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.810 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.809+0000 7f8936a00640 1 -- 172.21.15.145:0/2752705629 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f89381629e0 msgr2=0x7f89380d0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:58.810 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.809+0000 7f8936a00640 1 --2- 172.21.15.145:0/2752705629 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f89381629e0 0x7f89380d0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.810 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.809+0000 7f8936a00640 1 -- 172.21.15.145:0/2752705629 >> v2:172.21.15.192:3300/0 conn(0x7f89380d15e0 msgr2=0x7f89380d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:58.811 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.809+0000 7f8936a00640 1 --2- 172.21.15.145:0/2752705629 >> v2:172.21.15.192:3300/0 conn(0x7f89380d15e0 0x7f89380d1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.811 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.809+0000 7f8936a00640 1 -- 172.21.15.145:0/2752705629 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89380d11d0 con 0x7f89380d1f60 2024-05-09T04:33:58.811 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.809+0000 7f8936a00640 1 -- 172.21.15.145:0/2752705629 <== mon.0 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3474413261 0 0) 0x7f8930001a70 con 0x7f89380d1f60 2024-05-09T04:33:58.811 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.809+0000 7f8936a00640 1 -- 172.21.15.145:0/2752705629 >> v1:172.21.15.145:6789/0 conn(0x7f89380d1f60 legacy=0x7f89380d2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.811 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.809+0000 7f8936a00640 1 --2- 172.21.15.145:0/2752705629 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f891c003a50 0x7f891c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.811 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.809+0000 7f8936a00640 1 --2- 172.21.15.145:0/2752705629 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f891c004800 0x7f891c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.811 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.809+0000 7f8936a00640 1 --2- 172.21.15.145:0/2752705629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f891c0051c0 0x7f891c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.811 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.809+0000 7f8936a00640 1 -- 172.21.15.145:0/2752705629 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f89380b5090 con 0x7f891c004800 2024-05-09T04:33:58.811 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.809+0000 7f8936a00640 1 -- 172.21.15.145:0/2752705629 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f89380d0c70 con 0x7f891c0051c0 2024-05-09T04:33:58.811 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.809+0000 7f8936a00640 1 -- 172.21.15.145:0/2752705629 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f891c00c140 con 0x7f891c003a50 2024-05-09T04:33:58.811 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.809+0000 7f8ae4200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad40e3530 con 0x7f8ad403c090 2024-05-09T04:33:58.812 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.810+0000 7f8ae4200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad4054630 con 0x7f8ad4053ca0 2024-05-09T04:33:58.812 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.810+0000 7f8ae4200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8ad4006b50 con 0x7f8ad40e3150 2024-05-09T04:33:58.812 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.810+0000 7f8937400640 1 --2- 172.21.15.145:0/2752705629 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f891c004800 0x7f891c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:58.812 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.810+0000 7f8ae2200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8ad40e3150 0x7f8ad417bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:58.812 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.810+0000 7f3296000640 1 Processor -- start 2024-05-09T04:33:58.813 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.810+0000 7f8ae0e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ad4053ca0 0x7f8ad416d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:58.813 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.810+0000 7f8ae2200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8ad40e3150 0x7f8ad417bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.145:47320/0 (socket says 172.21.15.145:47320) 2024-05-09T04:33:58.813 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.810+0000 7f8ae2200640 1 -- 172.21.15.145:0/1795159111 learned_addr learned my addr 172.21.15.145:0/1795159111 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:58.813 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.810+0000 7f209d800640 1 -- 172.21.15.145:0/1322623822 >> v1:172.21.15.192:6789/0 conn(0x7f209815f100 legacy=0x7f20980d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:58.813 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.810+0000 7f209d800640 1 -- 172.21.15.145:0/1322623822 >> v1:172.21.15.145:6789/0 conn(0x7f20980d3cd0 legacy=0x7f20980d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:58.814 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.810+0000 7f8937e00640 1 --2- 172.21.15.145:0/2752705629 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f891c003a50 0x7f891c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:58.814 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.811+0000 7f209d800640 1 -- 172.21.15.145:0/1322623822 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20980d0360 con 0x7f209815f4e0 2024-05-09T04:33:58.814 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.810+0000 7f3296000640 1 -- start start 2024-05-09T04:33:58.815 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.811+0000 7f8ae2200640 1 -- 172.21.15.145:0/1795159111 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ad4053ca0 msgr2=0x7f8ad416d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.815 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.811+0000 7f8ae2200640 1 --2- 172.21.15.145:0/1795159111 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ad4053ca0 0x7f8ad416d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.815 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.811+0000 7f8ae2200640 1 -- 172.21.15.145:0/1795159111 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ad403c090 msgr2=0x7f8ad40e0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.815 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.811+0000 7f8ae2200640 1 --2- 172.21.15.145:0/1795159111 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8ad403c090 0x7f8ad40e0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.815 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.811+0000 7f8ae2200640 1 -- 172.21.15.145:0/1795159111 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ad40e0ad0 con 0x7f8ad40e3150 2024-05-09T04:33:58.816 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.811+0000 7f5afd400640 1 Processor -- start 2024-05-09T04:33:58.816 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.811+0000 7f5afd400640 1 -- start start 2024-05-09T04:33:58.816 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.811+0000 7f893ce00640 1 --2- 172.21.15.145:0/2752705629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f891c0051c0 0x7f891c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:58.816 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.812+0000 7f8937e00640 1 -- 172.21.15.145:0/2752705629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f891c0051c0 msgr2=0x7f891c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:58.816 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.812+0000 7f8937e00640 1 --2- 172.21.15.145:0/2752705629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f891c0051c0 0x7f891c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.816 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.812+0000 7f8937e00640 1 -- 172.21.15.145:0/2752705629 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f891c004800 msgr2=0x7f891c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:58.817 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.812+0000 7f8937e00640 1 --2- 172.21.15.145:0/2752705629 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f891c004800 0x7f891c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.817 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.812+0000 7f8937e00640 1 -- 172.21.15.145:0/2752705629 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89380d11d0 con 0x7f891c003a50 2024-05-09T04:33:58.817 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.812+0000 7f8ae2200640 1 --2- 172.21.15.145:0/1795159111 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8ad40e3150 0x7f8ad417bb30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8adc074fe0 tx=0x7f8adc069fc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:58.817 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.813+0000 7f3296000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f32880e7710 0x7f328804ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.817 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.813+0000 7f3296000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f32880589c0 0x7f3288053530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.818 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.813+0000 7f5afd400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f5af80d3450 0x7f5af80d3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.818 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.813+0000 7f5afd400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f5af80b3fe0 0x7f5af80b2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.818 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.813+0000 7f5afd400640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5af80b1c80 con 0x7f5af80b2950 2024-05-09T04:33:58.818 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.813+0000 7f5afd400640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f5af80b1f80 con 0x7f5af80b3fe0 2024-05-09T04:33:58.818 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.813+0000 7f5afd400640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f5af815f8f0 con 0x7f5af80d3450 2024-05-09T04:33:58.818 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.814+0000 7f209d800640 1 --2- 172.21.15.145:0/1322623822 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f209815f4e0 0x7f209816a120 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2090009e80 tx=0x7f209002a6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1f454a917f64b4a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:58.819 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.813+0000 7f6488800640 1 Processor -- start 2024-05-09T04:33:58.819 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.813+0000 7f8ad9600640 1 -- 172.21.15.145:0/1795159111 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8adc06c480 con 0x7f8ad40e3150 2024-05-09T04:33:58.819 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.814+0000 7f3296000640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3288057d90 con 0x7f3288056ab0 2024-05-09T04:33:58.819 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.814+0000 7f3296000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f3288006260 con 0x7f32880589c0 2024-05-09T04:33:58.820 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.814+0000 7f8937e00640 1 --2- 172.21.15.145:0/2752705629 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f891c003a50 0x7f891c003e70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f893806c7c0 tx=0x7f891c005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46e6daf9a0ca47f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:58.820 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.814+0000 7f3294c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f32880e7710 0x7f328804ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:58.820 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.814+0000 7f3294c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f32880e7710 0x7f328804ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.145:47342/0 (socket says 172.21.15.145:47342) 2024-05-09T04:33:58.820 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.814+0000 7f3294c00640 1 -- 172.21.15.145:0/795394395 learned_addr learned my addr 172.21.15.145:0/795394395 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:58.820 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.814+0000 7f5af7400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f5af80b3fe0 0x7f5af80b2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:58.821 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.814+0000 7f8ad9600640 1 -- 172.21.15.145:0/1795159111 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8adc090030 con 0x7f8ad40e3150 2024-05-09T04:33:58.821 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.814+0000 7f3296000640 1 -- 172.21.15.145:0/795394395 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f3288006540 con 0x7f32880e7710 2024-05-09T04:33:58.821 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.814+0000 7f8ad9600640 1 -- 172.21.15.145:0/1795159111 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8adc06c920 con 0x7f8ad40e3150 2024-05-09T04:33:58.822 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.814+0000 7f6488800640 1 -- start start 2024-05-09T04:33:58.822 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.814+0000 7f5af7400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f5af80b3fe0 0x7f5af80b2410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:45064/0 (socket says 172.21.15.145:45064) 2024-05-09T04:33:58.822 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.814+0000 7f5af7400640 1 -- 172.21.15.145:0/486592245 learned_addr learned my addr 172.21.15.145:0/486592245 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:58.822 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.815+0000 7f5af7e00640 1 --2- 172.21.15.145:0/486592245 >> v2:172.21.15.192:3300/0 conn(0x7f5af80d3450 0x7f5af80d3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:58.823 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.814+0000 7f2097400640 1 -- 172.21.15.145:0/1322623822 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2090036070 con 0x7f209815f4e0 2024-05-09T04:33:58.823 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.815+0000 7f2097400640 1 -- 172.21.15.145:0/1322623822 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2090031600 con 0x7f209815f4e0 2024-05-09T04:33:58.823 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.815+0000 7f2097400640 1 -- 172.21.15.145:0/1322623822 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20900319d0 con 0x7f209815f4e0 2024-05-09T04:33:58.823 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.815+0000 7f5af7400640 1 -- 172.21.15.145:0/486592245 >> v2:172.21.15.192:3300/0 conn(0x7f5af80d3450 msgr2=0x7f5af80d3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:58.823 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.815+0000 7f8ae4200640 1 -- 172.21.15.145:0/1795159111 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ad4178df0 con 0x7f8ad40e3150 2024-05-09T04:33:58.823 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.815+0000 7f8ae4200640 1 -- 172.21.15.145:0/1795159111 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ad4179390 con 0x7f8ad40e3150 2024-05-09T04:33:58.824 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.816+0000 7f209e200640 1 -- 172.21.15.145:0/1322623822 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f209815f4e0 msgr2=0x7f209816a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.824 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.816+0000 7f209e200640 1 --2- 172.21.15.145:0/1322623822 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f209815f4e0 0x7f209816a120 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2090009e80 tx=0x7f209002a6a0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.824 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.816+0000 7f209e200640 1 -- 172.21.15.145:0/1322623822 shutdown_connections 2024-05-09T04:33:58.824 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.816+0000 7f209e200640 1 --2- 172.21.15.145:0/1322623822 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f209815f4e0 0x7f209816a120 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.824 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.816+0000 7f209e200640 1 -- 172.21.15.145:0/1322623822 >> 172.21.15.145:0/1322623822 conn(0x7f209809e2e0 msgr2=0x7f20980d5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:58.824 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.814+0000 7f8936a00640 1 -- 172.21.15.145:0/2752705629 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8924030070 con 0x7f891c003a50 2024-05-09T04:33:58.824 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.816+0000 7f8936a00640 1 -- 172.21.15.145:0/2752705629 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f892402acc0 con 0x7f891c003a50 2024-05-09T04:33:58.824 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.816+0000 7f8936a00640 1 -- 172.21.15.145:0/2752705629 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892400b550 con 0x7f891c003a50 2024-05-09T04:33:58.825 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.814+0000 7f328fe00640 1 --2- 172.21.15.145:0/795394395 >> v2:172.21.15.192:3300/0 conn(0x7f32880589c0 0x7f3288053530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:58.825 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.815+0000 7f5af7400640 1 --2- 172.21.15.145:0/486592245 >> v2:172.21.15.192:3300/0 conn(0x7f5af80d3450 0x7f5af80d3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.825 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.815+0000 7f5af7400640 1 -- 172.21.15.145:0/486592245 >> v1:172.21.15.192:6789/0 conn(0x7f5af80b2950 legacy=0x7f5af80b2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:58.825 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.816+0000 7f5af7400640 1 -- 172.21.15.145:0/486592245 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5af80d3d70 con 0x7f5af80b3fe0 2024-05-09T04:33:58.826 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.816+0000 7f328f400640 1 -- 172.21.15.145:0/795394395 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1673811437 0 0) 0x7f3288057d90 con 0x7f3288056ab0 2024-05-09T04:33:58.826 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.816+0000 7f328f400640 1 -- 172.21.15.145:0/795394395 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3278003520 con 0x7f3288056ab0 2024-05-09T04:33:58.826 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.816+0000 7f6488800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f648015f610 0x7f64800d5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.826 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.816+0000 7f209e200640 1 -- 172.21.15.145:0/1322623822 shutdown_connections 2024-05-09T04:33:58.826 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.816+0000 7f209e200640 1 -- 172.21.15.145:0/1322623822 wait complete. 2024-05-09T04:33:58.827 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.816+0000 7f6488800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f64800cfc70 0x7f64800d3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.827 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.817+0000 7f6488800640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64800d0150 con 0x7f64800d3c80 2024-05-09T04:33:58.827 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.817+0000 7f6488800640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f64800d40d0 con 0x7f64800cfc70 2024-05-09T04:33:58.827 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.817+0000 7f6488800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f64800d0d60 con 0x7f648015f610 2024-05-09T04:33:58.827 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.817+0000 7f8ae4200640 1 -- 172.21.15.145:0/1795159111 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8ad4005380 con 0x7f8ad40e3150 2024-05-09T04:33:58.827 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.816+0000 7f328fe00640 1 -- 172.21.15.145:0/795394395 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f32880e7710 msgr2=0x7f328804ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:58.827 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.816+0000 7f328fe00640 1 --2- 172.21.15.145:0/795394395 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f32880e7710 0x7f328804ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.828 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.816+0000 7f328fe00640 1 -- 172.21.15.145:0/795394395 >> v1:172.21.15.192:6789/0 conn(0x7f3288056ab0 legacy=0x7f3288007020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.828 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.816+0000 7f328fe00640 1 -- 172.21.15.145:0/795394395 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3288006820 con 0x7f32880589c0 2024-05-09T04:33:58.828 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.817+0000 7f5af7400640 1 --2- 172.21.15.145:0/486592245 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5af80b3fe0 0x7f5af80b2410 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5af80b7c10 tx=0x7f5ae402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc4f9074c5eec13c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:58.828 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.817+0000 7f893ee00640 1 -- 172.21.15.145:0/2752705629 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f891c003a50 msgr2=0x7f891c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.828 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.817+0000 7f893ee00640 1 --2- 172.21.15.145:0/2752705629 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f891c003a50 0x7f891c003e70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f893806c7c0 tx=0x7f891c005830 comp rx=0 tx=0).stop 2024-05-09T04:33:58.829 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.817+0000 7f6485400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f64800cfc70 0x7f64800d3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:58.829 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.817+0000 7f6485400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f64800cfc70 0x7f64800d3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:45072/0 (socket says 172.21.15.145:45072) 2024-05-09T04:33:58.829 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.817+0000 7f6485400640 1 -- 172.21.15.145:0/2127575439 learned_addr learned my addr 172.21.15.145:0/2127575439 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:58.829 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.818+0000 7f893ce00640 1 -- 172.21.15.145:0/2752705629 reap_dead start 2024-05-09T04:33:58.829 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.818+0000 7f5af6a00640 1 -- 172.21.15.145:0/486592245 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae4037070 con 0x7f5af80b3fe0 2024-05-09T04:33:58.829 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.818+0000 7f5af6a00640 1 -- 172.21.15.145:0/486592245 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ae4032440 con 0x7f5af80b3fe0 2024-05-09T04:33:58.829 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.818+0000 7f5af6a00640 1 -- 172.21.15.145:0/486592245 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae4032810 con 0x7f5af80b3fe0 2024-05-09T04:33:58.830 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.818+0000 7f6485400640 1 -- 172.21.15.145:0/2127575439 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f648015f610 msgr2=0x7f64800d5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:58.830 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.818+0000 7f6485400640 1 --2- 172.21.15.145:0/2127575439 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f648015f610 0x7f64800d5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.830 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.818+0000 7f6485400640 1 -- 172.21.15.145:0/2127575439 >> v1:172.21.15.192:6789/0 conn(0x7f64800d3c80 legacy=0x7f64801630b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.830 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.818+0000 7f6485400640 1 -- 172.21.15.145:0/2127575439 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64800d0430 con 0x7f64800cfc70 2024-05-09T04:33:58.830 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.818+0000 7f893ee00640 1 -- 172.21.15.145:0/2752705629 shutdown_connections 2024-05-09T04:33:58.830 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.818+0000 7f893ee00640 1 -- 172.21.15.145:0/2752705629 >> 172.21.15.145:0/2752705629 conn(0x7f89380d4fb0 msgr2=0x7f89380d53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:58.831 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.819+0000 7f5afd400640 1 -- 172.21.15.145:0/486592245 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5af80b3fe0 msgr2=0x7f5af80b2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.831 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.819+0000 7f5afd400640 1 --2- 172.21.15.145:0/486592245 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5af80b3fe0 0x7f5af80b2410 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5af80b7c10 tx=0x7f5ae402a140 comp rx=0 tx=0).stop 2024-05-09T04:33:58.831 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.819+0000 7f328fe00640 1 --2- 172.21.15.145:0/795394395 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32880589c0 0x7f3288053530 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f32840097a0 tx=0x7f328402a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=740c7f4d70c3342f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:58.831 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.819+0000 7f209e200640 1 Processor -- start 2024-05-09T04:33:58.832 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.819+0000 7f5afd400640 1 -- 172.21.15.145:0/486592245 shutdown_connections 2024-05-09T04:33:58.832 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.819+0000 7f5afd400640 1 --2- 172.21.15.145:0/486592245 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5af80b3fe0 0x7f5af80b2410 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.832 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.819+0000 7f5afd400640 1 --2- 172.21.15.145:0/486592245 >> v2:172.21.15.192:3300/0 conn(0x7f5af80d3450 0x7f5af80d3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.832 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.819+0000 7f5afd400640 1 -- 172.21.15.145:0/486592245 >> 172.21.15.145:0/486592245 conn(0x7f5af80d5020 msgr2=0x7f5af80d5430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:58.832 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.819+0000 7f5afd400640 1 -- 172.21.15.145:0/486592245 shutdown_connections 2024-05-09T04:33:58.832 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.819+0000 7f893ee00640 1 -- 172.21.15.145:0/2752705629 shutdown_connections 2024-05-09T04:33:58.832 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.819+0000 7f328f400640 1 -- 172.21.15.145:0/795394395 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3284037070 con 0x7f32880589c0 2024-05-09T04:33:58.832 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.819+0000 7f328f400640 1 -- 172.21.15.145:0/795394395 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f328402adb0 con 0x7f32880589c0 2024-05-09T04:33:58.833 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.819+0000 7f328f400640 1 -- 172.21.15.145:0/795394395 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3284032640 con 0x7f32880589c0 2024-05-09T04:33:58.833 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.819+0000 7f0e51a00640 1 Processor -- start 2024-05-09T04:33:58.833 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.820+0000 7f0e51a00640 1 -- start start 2024-05-09T04:33:58.833 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.819+0000 7f209e200640 1 -- start start 2024-05-09T04:33:58.834 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.820+0000 7fe983000640 1 Processor -- start 2024-05-09T04:33:58.834 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.821+0000 7f209e200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f20980d3cd0 0x7f209814ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.834 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.821+0000 7f6485400640 1 --2- 172.21.15.145:0/2127575439 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f64800cfc70 0x7f64800d3740 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f646c009870 tx=0x7f646c02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6baa8c54651d849c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:58.835 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.821+0000 7f209e200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f209815f100 0x7f209814b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.835 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.820+0000 7f893ee00640 1 -- 172.21.15.145:0/2752705629 wait complete. 2024-05-09T04:33:58.835 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.821+0000 7f209e200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f209815f4e0 0x7f209814bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.835 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.822+0000 7f209e200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f20980d0e40 con 0x7f209815f4e0 2024-05-09T04:33:58.835 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.822+0000 7f209e200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f20980d0920 con 0x7f209815f100 2024-05-09T04:33:58.835 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.822+0000 7f209e200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f209815eae0 con 0x7f20980d3cd0 2024-05-09T04:33:58.836 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.822+0000 7f6484a00640 1 -- 172.21.15.145:0/2127575439 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f646c037070 con 0x7f64800cfc70 2024-05-09T04:33:58.836 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.822+0000 7f6484a00640 1 -- 172.21.15.145:0/2127575439 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f646c032440 con 0x7f64800cfc70 2024-05-09T04:33:58.836 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.822+0000 7f3296000640 1 -- 172.21.15.145:0/795394395 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32880589c0 msgr2=0x7f3288053530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.836 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.822+0000 7f3296000640 1 --2- 172.21.15.145:0/795394395 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32880589c0 0x7f3288053530 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f32840097a0 tx=0x7f328402a060 comp rx=0 tx=0).stop 2024-05-09T04:33:58.836 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.822+0000 7f5afd400640 1 -- 172.21.15.145:0/486592245 wait complete. 2024-05-09T04:33:58.837 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.822+0000 7f209ce00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f20980d3cd0 0x7f209814ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:58.837 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.822+0000 7f2097e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f209815f100 0x7f209814b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:58.837 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.822+0000 7f2097e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f209815f100 0x7f209814b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.145:55292/0 (socket says 172.21.15.145:55292) 2024-05-09T04:33:58.837 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.822+0000 7f2097e00640 1 -- 172.21.15.145:0/1784680129 learned_addr learned my addr 172.21.15.145:0/1784680129 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:58.837 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.822+0000 7f209d800640 1 --2- 172.21.15.145:0/1784680129 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f209815f4e0 0x7f209814bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:58.837 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.822+0000 7f893ee00640 1 Processor -- start 2024-05-09T04:33:58.838 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.820+0000 7fe983000640 1 -- start start 2024-05-09T04:33:58.838 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.822+0000 7fe983000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe97c163910 0x7fe97c0cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.838 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.822+0000 7fe983000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fe97c0d1290 0x7fe97c0cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.838 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.822+0000 7fe983000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe97c0d0db0 con 0x7fe97c0c49d0 2024-05-09T04:33:58.838 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.822+0000 7fe983000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fe97c068580 con 0x7fe97c0d1290 2024-05-09T04:33:58.838 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.822+0000 7fe983000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe97c06eb40 con 0x7fe97c163910 2024-05-09T04:33:58.839 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.823+0000 7f6488800640 1 -- 172.21.15.145:0/2127575439 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f64800cfc70 msgr2=0x7f64800d3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.839 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.823+0000 7f6484a00640 1 -- 172.21.15.145:0/2127575439 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f646c037070 con 0x7f64800cfc70 2024-05-09T04:33:58.839 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.823+0000 7f6488800640 1 --2- 172.21.15.145:0/2127575439 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f64800cfc70 0x7f64800d3740 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f646c009870 tx=0x7f646c02a250 comp rx=0 tx=0).stop 2024-05-09T04:33:58.839 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.823+0000 7f893ee00640 1 -- start start 2024-05-09T04:33:58.840 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.823+0000 7fe97be00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe97c163910 0x7fe97c0cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:58.840 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.823+0000 7fe97be00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe97c163910 0x7fe97c0cf6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.145:47366/0 (socket says 172.21.15.145:47366) 2024-05-09T04:33:58.840 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.823+0000 7fe97be00640 1 -- 172.21.15.145:0/3593262327 learned_addr learned my addr 172.21.15.145:0/3593262327 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:58.840 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.823+0000 7f3296000640 1 -- 172.21.15.145:0/795394395 shutdown_connections 2024-05-09T04:33:58.840 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.823+0000 7f3296000640 1 --2- 172.21.15.145:0/795394395 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32880589c0 0x7f3288053530 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.840 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.823+0000 7f3296000640 1 --2- 172.21.15.145:0/795394395 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f32880e7710 0x7f328804ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.841 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.823+0000 7f3296000640 1 -- 172.21.15.145:0/795394395 >> 172.21.15.145:0/795394395 conn(0x7f3288058ee0 msgr2=0x7f32880592c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:58.841 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.823+0000 7f3296000640 1 -- 172.21.15.145:0/795394395 shutdown_connections 2024-05-09T04:33:58.841 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.824+0000 7fe981000640 1 --2- 172.21.15.145:0/3593262327 >> v2:172.21.15.145:3300/0 conn(0x7fe97c0d1290 0x7fe97c0cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:58.841 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.822+0000 7f8ad9600640 1 -- 172.21.15.145:0/1795159111 <== mon.2 v2:172.21.15.192:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8adc06cac0 con 0x7f8ad40e3150 2024-05-09T04:33:58.842 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.823+0000 7f8ad9600640 1 --2- 172.21.15.145:0/1795159111 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f8abc0763a0 0x7f8abc078860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.842 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.823+0000 7f8ad9600640 1 -- 172.21.15.145:0/1795159111 <== mon.2 v2:172.21.15.192:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8adc113c30 con 0x7f8ad40e3150 2024-05-09T04:33:58.842 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.824+0000 7f8ad9600640 1 --2- 172.21.15.145:0/1795159111 >> [v2:172.21.15.145:6800/3182572912,v1:172.21.15.145:6801/3182572912] conn(0x7f8abc07e5c0 0x7f8abc0809e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.842 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.824+0000 7f8ad9600640 1 -- 172.21.15.145:0/1795159111 --> [v2:172.21.15.145:6800/3182572912,v1:172.21.15.145:6801/3182572912] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8adc090ec0 con 0x7f8abc07e5c0 2024-05-09T04:33:58.842 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.824+0000 7f8ad9600640 1 -- 172.21.15.145:0/1795159111 <== mon.2 v2:172.21.15.192:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f8adc114020 con 0x7f8ad40e3150 2024-05-09T04:33:58.842 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.824+0000 7f8ae1800640 1 --2- 172.21.15.145:0/1795159111 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f8abc0763a0 0x7f8abc078860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:58.842 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.823+0000 7f2097e00640 1 -- 172.21.15.145:0/1784680129 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f20980d3cd0 msgr2=0x7f209814ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.842 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.823+0000 7f2097e00640 1 --2- 172.21.15.145:0/1784680129 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f20980d3cd0 0x7f209814ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.842 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.823+0000 7f2097e00640 1 -- 172.21.15.145:0/1784680129 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f209815f4e0 msgr2=0x7f209814bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.842 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.823+0000 7f2097e00640 1 --2- 172.21.15.145:0/1784680129 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f209815f4e0 0x7f209814bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.843 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.823+0000 7f2097e00640 1 -- 172.21.15.145:0/1784680129 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20980cd6e0 con 0x7f209815f100 2024-05-09T04:33:58.843 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.824+0000 7f0e51a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0e440078e0 0x7f0e440edd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.843 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.824+0000 7f0e51a00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e44058570 con 0x7f0e4403e840 2024-05-09T04:33:58.843 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.824+0000 7f8ae0e00640 1 --2- 172.21.15.145:0/1795159111 >> [v2:172.21.15.145:6800/3182572912,v1:172.21.15.145:6801/3182572912] conn(0x7f8abc07e5c0 0x7f8abc0809e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:58.844 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.824+0000 7fe97be00640 1 -- 172.21.15.145:0/3593262327 >> v2:172.21.15.145:3300/0 conn(0x7fe97c0d1290 msgr2=0x7fe97c0cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:58.844 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.824+0000 7fe97be00640 1 --2- 172.21.15.145:0/3593262327 >> v2:172.21.15.145:3300/0 conn(0x7fe97c0d1290 0x7fe97c0cce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.844 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.824+0000 7fe97be00640 1 -- 172.21.15.145:0/3593262327 >> v1:172.21.15.145:6789/0 conn(0x7fe97c0c49d0 legacy=0x7fe97c0cfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.844 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.824+0000 7fe97be00640 1 -- 172.21.15.145:0/3593262327 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe97c06ee20 con 0x7fe97c163910 2024-05-09T04:33:58.844 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.824+0000 7fe981000640 1 --2- 172.21.15.145:0/3593262327 >> v2:172.21.15.145:3300/0 conn(0x7fe97c0d1290 0x7fe97c0cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:33:58.844 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.824+0000 7f0e51a00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e44058850 con 0x7f0e440e30c0 2024-05-09T04:33:58.844 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.824+0000 7f0e51a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0e44058b30 con 0x7f0e440078e0 2024-05-09T04:33:58.844 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.825+0000 7f2097e00640 1 --2- 172.21.15.145:0/1784680129 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f209815f100 0x7f209814b690 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f208400c430 tx=0x7f208400c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:58.845 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.825+0000 7f0e4be00640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f0e440e30c0 0x7f0e44057d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.145:42514/0 (socket says 172.21.15.145:42514) 2024-05-09T04:33:58.845 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.825+0000 7f0e4be00640 1 -- 172.21.15.145:0/3868217152 learned_addr learned my addr 172.21.15.145:0/3868217152 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:33:58.845 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.825+0000 7f6488800640 1 -- 172.21.15.145:0/2127575439 shutdown_connections 2024-05-09T04:33:58.845 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.825+0000 7f6488800640 1 --2- 172.21.15.145:0/2127575439 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f64800cfc70 0x7f64800d3740 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.845 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.825+0000 7f6488800640 1 --2- 172.21.15.145:0/2127575439 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f648015f610 0x7f64800d5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.845 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.825+0000 7f6488800640 1 -- 172.21.15.145:0/2127575439 >> 172.21.15.145:0/2127575439 conn(0x7f648009e4a0 msgr2=0x7f64800d5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:58.846 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.825+0000 7f8ae1800640 1 --2- 172.21.15.145:0/1795159111 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f8abc0763a0 0x7f8abc078860 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8000c00 tx=0x7f8ac80079c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:58.846 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.825+0000 7fe97be00640 1 --2- 172.21.15.145:0/3593262327 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe97c163910 0x7fe97c0cf6a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe96c009870 tx=0x7fe96c02a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=faaf3a2aa16ea4b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:58.846 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.826+0000 7f6488800640 1 -- 172.21.15.145:0/2127575439 shutdown_connections 2024-05-09T04:33:58.847 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.825+0000 7f0e51000640 1 --2- 172.21.15.145:0/3868217152 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0e440078e0 0x7f0e440edd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:58.847 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.825+0000 7f0e4aa00640 1 -- 172.21.15.145:0/3868217152 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3832000029 0 0) 0x7f0e44058570 con 0x7f0e4403e840 2024-05-09T04:33:58.847 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.825+0000 7f0e4aa00640 1 -- 172.21.15.145:0/3868217152 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e34003520 con 0x7f0e4403e840 2024-05-09T04:33:58.847 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.826+0000 7f2094c00640 1 -- 172.21.15.145:0/1784680129 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2084011790 con 0x7f209815f100 2024-05-09T04:33:58.847 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.826+0000 7f2094c00640 1 -- 172.21.15.145:0/1784680129 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2084017070 con 0x7f209815f100 2024-05-09T04:33:58.847 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.826+0000 7f2094c00640 1 -- 172.21.15.145:0/1784680129 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2084011bb0 con 0x7f209815f100 2024-05-09T04:33:58.847 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.824+0000 7f3296000640 1 -- 172.21.15.145:0/795394395 wait complete. 2024-05-09T04:33:58.848 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.826+0000 7f209e200640 1 -- 172.21.15.145:0/1784680129 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f209814c2a0 con 0x7f209815f100 2024-05-09T04:33:58.848 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.826+0000 7f6488800640 1 -- 172.21.15.145:0/2127575439 wait complete. 2024-05-09T04:33:58.848 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.826+0000 7f893ee00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f891c003a50 0x7f8938172c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.849 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.826+0000 7fe97aa00640 1 -- 172.21.15.145:0/3593262327 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96c037070 con 0x7fe97c163910 2024-05-09T04:33:58.849 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.827+0000 7f893ee00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f89380715a0 0x7f89380cc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.826+0000 7fe97aa00640 1 -- 172.21.15.145:0/3593262327 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe96c032440 con 0x7fe97c163910 2024-05-09T04:33:58.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.826+0000 7fe97aa00640 1 -- 172.21.15.145:0/3593262327 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96c032810 con 0x7fe97c163910 2024-05-09T04:33:58.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.827+0000 7fe983000640 1 -- 172.21.15.145:0/3593262327 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe97c163910 msgr2=0x7fe97c0cf6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.827+0000 7fe983000640 1 --2- 172.21.15.145:0/3593262327 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe97c163910 0x7fe97c0cf6a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe96c009870 tx=0x7fe96c02a250 comp rx=0 tx=0).stop 2024-05-09T04:33:58.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.828+0000 7fe983000640 1 -- 172.21.15.145:0/3593262327 shutdown_connections 2024-05-09T04:33:58.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.828+0000 7fe983000640 1 --2- 172.21.15.145:0/3593262327 >> v2:172.21.15.145:3300/0 conn(0x7fe97c0d1290 0x7fe97c0cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.828+0000 7fe983000640 1 --2- 172.21.15.145:0/3593262327 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe97c163910 0x7fe97c0cf6a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.828+0000 7f3296000640 1 Processor -- start 2024-05-09T04:33:58.851 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.828+0000 7f8937e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f891c003a50 0x7f8938172c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:58.851 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.828+0000 7fe983000640 1 -- 172.21.15.145:0/3593262327 >> 172.21.15.145:0/3593262327 conn(0x7fe97c0ce220 msgr2=0x7fe97c0ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:58.851 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.828+0000 7f3296000640 1 -- start start 2024-05-09T04:33:58.852 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.828+0000 7f8937e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f891c003a50 0x7f8938172c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.145:47388/0 (socket says 172.21.15.145:47388) 2024-05-09T04:33:58.852 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.828+0000 7f8937e00640 1 -- 172.21.15.145:0/2571431195 learned_addr learned my addr 172.21.15.145:0/2571431195 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:58.852 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.828+0000 7fe983000640 1 -- 172.21.15.145:0/3593262327 shutdown_connections 2024-05-09T04:33:58.852 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.828+0000 7fe983000640 1 -- 172.21.15.145:0/3593262327 wait complete. 2024-05-09T04:33:58.852 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.828+0000 7f893ee00640 1 --2- 172.21.15.145:0/2571431195 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89380caa00 0x7f8938161480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.853 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.828+0000 7f893ee00640 1 -- 172.21.15.145:0/2571431195 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f89380b4a40 con 0x7f89380715a0 2024-05-09T04:33:58.853 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.828+0000 7f893ee00640 1 -- 172.21.15.145:0/2571431195 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f89380d23f0 con 0x7f89380caa00 2024-05-09T04:33:58.853 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.826+0000 7f0e4aa00640 1 -- 172.21.15.145:0/3868217152 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 840053500 0 0) 0x7f0e44058850 con 0x7f0e440e30c0 2024-05-09T04:33:58.853 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.826+0000 7f0e4aa00640 1 -- 172.21.15.145:0/3868217152 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e44058570 con 0x7f0e440e30c0 2024-05-09T04:33:58.853 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.826+0000 7f0e51000640 1 -- 172.21.15.145:0/3868217152 >> v1:172.21.15.192:6789/0 conn(0x7f0e440e30c0 legacy=0x7f0e44057d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.853 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.826+0000 7f0e51000640 1 -- 172.21.15.145:0/3868217152 >> v1:172.21.15.145:6789/0 conn(0x7f0e4403e840 legacy=0x7f0e44053bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.853 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.826+0000 7f0e51000640 1 -- 172.21.15.145:0/3868217152 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e440063b0 con 0x7f0e440078e0 2024-05-09T04:33:58.853 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.828+0000 7f893ee00640 1 -- 172.21.15.145:0/2571431195 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f89380b4d20 con 0x7f891c003a50 2024-05-09T04:33:58.854 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.826+0000 7f5afd400640 1 Processor -- start 2024-05-09T04:33:58.854 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.826+0000 7f8ae0e00640 1 --2- 172.21.15.145:0/1795159111 >> [v2:172.21.15.145:6800/3182572912,v1:172.21.15.145:6801/3182572912] conn(0x7f8abc07e5c0 0x7f8abc0809e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:58.854 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.826+0000 7f209e200640 1 -- 172.21.15.145:0/1784680129 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f209814c860 con 0x7f209815f100 2024-05-09T04:33:58.854 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.828+0000 7f209e200640 1 -- 172.21.15.145:0/1784680129 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f20980b6540 con 0x7f209815f100 2024-05-09T04:33:58.855 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.829+0000 7f8937400640 1 --2- 172.21.15.145:0/2571431195 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f89380715a0 0x7f89380cc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:58.855 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.829+0000 7f0e51000640 1 --2- 172.21.15.145:0/3868217152 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0e440078e0 0x7f0e440edd90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c000b60 tx=0x7f0e4c0a6670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=403fbcb7afffbc0b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:58.855 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.828+0000 7f6488800640 1 Processor -- start 2024-05-09T04:33:58.856 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.829+0000 7f5afd400640 1 -- start start 2024-05-09T04:33:58.856 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.829+0000 7f0e4aa00640 1 -- 172.21.15.145:0/3868217152 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c0b2070 con 0x7f0e440078e0 2024-05-09T04:33:58.856 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.829+0000 7f6488800640 1 -- start start 2024-05-09T04:33:58.856 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.829+0000 7f0e4aa00640 1 -- 172.21.15.145:0/3868217152 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e4c0ab7d0 con 0x7f0e440078e0 2024-05-09T04:33:58.856 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.829+0000 7f0e4aa00640 1 -- 172.21.15.145:0/3868217152 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c0abb80 con 0x7f0e440078e0 2024-05-09T04:33:58.857 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.829+0000 7f8ad9600640 1 -- 172.21.15.145:0/1795159111 <== osd.0 v2:172.21.15.145:6800/3182572912 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8adc090ec0 con 0x7f8abc07e5c0 2024-05-09T04:33:58.857 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.829+0000 7f893ce00640 1 --2- 172.21.15.145:0/2571431195 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89380caa00 0x7f8938161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:58.857 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.831+0000 7f8937400640 1 -- 172.21.15.145:0/2571431195 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f891c003a50 msgr2=0x7f8938172c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.857 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.831+0000 7f8937400640 1 --2- 172.21.15.145:0/2571431195 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f891c003a50 0x7f8938172c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.857 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.831+0000 7f8937400640 1 -- 172.21.15.145:0/2571431195 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89380caa00 msgr2=0x7f8938161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.857 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.831+0000 7f8937400640 1 --2- 172.21.15.145:0/2571431195 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89380caa00 0x7f8938161480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.858 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.831+0000 7f8937400640 1 -- 172.21.15.145:0/2571431195 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8938173170 con 0x7f89380715a0 2024-05-09T04:33:58.858 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.831+0000 7f8937e00640 1 --2- 172.21.15.145:0/2571431195 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f891c003a50 0x7f8938172c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:33:58.858 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.832+0000 7f8937400640 1 --2- 172.21.15.145:0/2571431195 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f89380715a0 0x7f89380cc8a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f892800f4e0 tx=0x7f892800f8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:58.858 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.832+0000 7f892fe00640 1 -- 172.21.15.145:0/2571431195 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8928004240 con 0x7f89380715a0 2024-05-09T04:33:58.858 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.832+0000 7f893ee00640 1 -- 172.21.15.145:0/2571431195 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8938071a40 con 0x7f89380715a0 2024-05-09T04:33:58.858 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.829+0000 7f3296000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3288056ab0 0x7f3288185310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.858 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.829+0000 7f3296000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f32880589c0 0x7f3288052350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.859 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.829+0000 7f3296000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32880e7710 0x7f3288052890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.859 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.830+0000 7f3296000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f32880579a0 con 0x7f32880589c0 2024-05-09T04:33:58.859 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.830+0000 7f3296000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f328804cf70 con 0x7f32880e7710 2024-05-09T04:33:58.859 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.830+0000 7f3296000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f3288057b20 con 0x7f3288056ab0 2024-05-09T04:33:58.859 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.830+0000 7f3294c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3288056ab0 0x7f3288185310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:58.859 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.830+0000 7f3294c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3288056ab0 0x7f3288185310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.145:47404/0 (socket says 172.21.15.145:47404) 2024-05-09T04:33:58.859 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.830+0000 7f3294c00640 1 -- 172.21.15.145:0/608230380 learned_addr learned my addr 172.21.15.145:0/608230380 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:58.859 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.830+0000 7f328fe00640 1 --2- 172.21.15.145:0/608230380 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f32880589c0 0x7f3288052350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:58.859 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.831+0000 7f3295600640 1 --2- 172.21.15.145:0/608230380 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32880e7710 0x7f3288052890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:58.859 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.831+0000 7f3294c00640 1 -- 172.21.15.145:0/608230380 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32880e7710 msgr2=0x7f3288052890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.859 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.831+0000 7f3294c00640 1 --2- 172.21.15.145:0/608230380 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32880e7710 0x7f3288052890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.859 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.831+0000 7f3294c00640 1 -- 172.21.15.145:0/608230380 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f32880589c0 msgr2=0x7f3288052350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.860 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.832+0000 7f5afd400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5af80b2950 0x7f5af815d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.860 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.832+0000 7f5afd400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5af80b3fe0 0x7f5af80bab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.860 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.832+0000 7f5afd400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5af80d3450 0x7f5af815bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.860 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.832+0000 7f5afd400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5af80d3180 con 0x7f5af80d3450 2024-05-09T04:33:58.860 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.832+0000 7f5afd400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5af80d5eb0 con 0x7f5af80b3fe0 2024-05-09T04:33:58.860 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.832+0000 7f5afd400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f5af80d5b40 con 0x7f5af80b2950 2024-05-09T04:33:58.860 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.831+0000 7f6488800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f64800cfc70 0x7f64800ce290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.860 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.831+0000 7f6488800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f64800d3c80 0x7f648015c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.860 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.831+0000 7f6488800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f648015f610 0x7f64800ca730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.860 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.831+0000 7f6488800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f648015f9f0 con 0x7f64800cfc70 2024-05-09T04:33:58.860 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.831+0000 7f6488800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f64800d6060 con 0x7f648015f610 2024-05-09T04:33:58.861 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.831+0000 7f6488800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f64800d09f0 con 0x7f64800d3c80 2024-05-09T04:33:58.861 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.831+0000 7f6485e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f64800cfc70 0x7f64800ce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:58.861 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.831+0000 7f6485400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f64800d3c80 0x7f648015c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:58.861 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.832+0000 7f6485400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f64800d3c80 0x7f648015c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.145:47410/0 (socket says 172.21.15.145:47410) 2024-05-09T04:33:58.861 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.832+0000 7f6485400640 1 -- 172.21.15.145:0/3748979104 learned_addr learned my addr 172.21.15.145:0/3748979104 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:58.861 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.832+0000 7f6486800640 1 --2- 172.21.15.145:0/3748979104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f648015f610 0x7f64800ca730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:58.861 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.832+0000 7f893ee00640 1 -- 172.21.15.145:0/2571431195 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89380722e0 con 0x7f89380715a0 2024-05-09T04:33:58.862 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.832+0000 7f6485400640 1 -- 172.21.15.145:0/3748979104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f648015f610 msgr2=0x7f64800ca730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.862 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.832+0000 7f6485400640 1 --2- 172.21.15.145:0/3748979104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f648015f610 0x7f64800ca730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.862 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.832+0000 7f6485400640 1 -- 172.21.15.145:0/3748979104 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f64800cfc70 msgr2=0x7f64800ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.862 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.832+0000 7f6485400640 1 --2- 172.21.15.145:0/3748979104 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f64800cfc70 0x7f64800ce290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.862 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.831+0000 7fe983000640 1 Processor -- start 2024-05-09T04:33:58.862 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.833+0000 7f6485400640 1 -- 172.21.15.145:0/3748979104 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64800ce970 con 0x7f64800d3c80 2024-05-09T04:33:58.863 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.833+0000 7f892fe00640 1 -- 172.21.15.145:0/2571431195 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89280043e0 con 0x7f89380715a0 2024-05-09T04:33:58.863 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.833+0000 7f892fe00640 1 -- 172.21.15.145:0/2571431195 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8928003810 con 0x7f89380715a0 2024-05-09T04:33:58.863 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.831+0000 7fe983000640 1 -- start start 2024-05-09T04:33:58.863 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.832+0000 7fe983000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe97c0c49d0 0x7fe97c1660d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.863 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.833+0000 7fe983000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe97c0d1290 0x7fe97c0c6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.863 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.833+0000 7fe983000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe97c163910 0x7fe97c14f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.864 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.833+0000 7f5af7e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5af80b2950 0x7f5af815d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:58.864 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.833+0000 7f5af7e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5af80b2950 0x7f5af815d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.145:47414/0 (socket says 172.21.15.145:47414) 2024-05-09T04:33:58.864 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.833+0000 7f5af7e00640 1 -- 172.21.15.145:0/1180297608 learned_addr learned my addr 172.21.15.145:0/1180297608 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:58.864 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.830+0000 7f0e51a00640 1 -- 172.21.15.145:0/3868217152 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0e440078e0 msgr2=0x7f0e440edd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.864 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.830+0000 7f0e51a00640 1 --2- 172.21.15.145:0/3868217152 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0e440078e0 0x7f0e440edd90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c000b60 tx=0x7f0e4c0a6670 comp rx=0 tx=0).stop 2024-05-09T04:33:58.864 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.832+0000 7f0e51a00640 1 -- 172.21.15.145:0/3868217152 shutdown_connections 2024-05-09T04:33:58.864 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.832+0000 7f0e51a00640 1 --2- 172.21.15.145:0/3868217152 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0e440078e0 0x7f0e440edd90 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.864 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.832+0000 7f0e51a00640 1 -- 172.21.15.145:0/3868217152 >> 172.21.15.145:0/3868217152 conn(0x7f0e440559e0 msgr2=0x7f0e44055df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:58.864 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.833+0000 7f0e51a00640 1 -- 172.21.15.145:0/3868217152 shutdown_connections 2024-05-09T04:33:58.865 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.833+0000 7f0e51a00640 1 -- 172.21.15.145:0/3868217152 wait complete. 2024-05-09T04:33:58.865 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.833+0000 7fe983000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe97c06f280 con 0x7fe97c0d1290 2024-05-09T04:33:58.865 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.833+0000 7fe983000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe97c06f100 con 0x7fe97c0c49d0 2024-05-09T04:33:58.865 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.833+0000 7fe983000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe97c0d02f0 con 0x7fe97c163910 2024-05-09T04:33:58.865 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.833+0000 7f6485400640 1 --2- 172.21.15.145:0/3748979104 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f64800d3c80 0x7f648015c700 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f646c0090b0 tx=0x7f646c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:58.865 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.834+0000 7f893ee00640 1 -- 172.21.15.145:0/2571431195 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8910001030 con 0x7f89380715a0 2024-05-09T04:33:58.866 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.833+0000 7f3294c00640 1 --2- 172.21.15.145:0/608230380 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f32880589c0 0x7f3288052350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.866 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.833+0000 7f3294c00640 1 -- 172.21.15.145:0/608230380 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3288052ea0 con 0x7f3288056ab0 2024-05-09T04:33:58.866 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.834+0000 7f5af7e00640 1 -- 172.21.15.145:0/1180297608 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5af80b3fe0 msgr2=0x7f5af80bab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.866 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.834+0000 7f5af7e00640 1 --2- 172.21.15.145:0/1180297608 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5af80b3fe0 0x7f5af80bab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.867 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.834+0000 7fe981000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe97c163910 0x7fe97c14f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:58.867 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.834+0000 7fe981000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe97c163910 0x7fe97c14f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.145:47428/0 (socket says 172.21.15.145:47428) 2024-05-09T04:33:58.867 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.834+0000 7fe981000640 1 -- 172.21.15.145:0/551572541 learned_addr learned my addr 172.21.15.145:0/551572541 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:58.867 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.834+0000 7fe97be00640 1 --2- 172.21.15.145:0/551572541 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe97c0c49d0 0x7fe97c1660d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:58.867 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.834+0000 7f5af7e00640 1 -- 172.21.15.145:0/1180297608 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5af80d3450 msgr2=0x7f5af815bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.867 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.834+0000 7f5af7e00640 1 --2- 172.21.15.145:0/1180297608 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5af80d3450 0x7f5af815bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.867 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.834+0000 7f5af7e00640 1 -- 172.21.15.145:0/1180297608 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5af815c6e0 con 0x7f5af80b2950 2024-05-09T04:33:58.867 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.834+0000 7f328fe00640 1 --2- 172.21.15.145:0/608230380 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f32880589c0 0x7f3288052350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:33:58.868 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.834+0000 7f647a000640 1 -- 172.21.15.145:0/3748979104 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f646c037040 con 0x7f64800d3c80 2024-05-09T04:33:58.868 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.835+0000 7f647a000640 1 -- 172.21.15.145:0/3748979104 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f646c0327f0 con 0x7f64800d3c80 2024-05-09T04:33:58.868 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.835+0000 7f647a000640 1 -- 172.21.15.145:0/3748979104 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f646c032bc0 con 0x7f64800d3c80 2024-05-09T04:33:58.868 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.835+0000 7f5af7e00640 1 --2- 172.21.15.145:0/1180297608 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5af80b2950 0x7f5af815d780 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5af0007e40 tx=0x7f5af000e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:58.868 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.835+0000 7fe97be00640 1 -- 172.21.15.145:0/551572541 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe97c163910 msgr2=0x7fe97c14f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.869 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.835+0000 7f6488800640 1 -- 172.21.15.145:0/3748979104 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64800cec50 con 0x7f64800d3c80 2024-05-09T04:33:58.869 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.835+0000 7f6488800640 1 -- 172.21.15.145:0/3748979104 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f64800cac70 con 0x7f64800d3c80 2024-05-09T04:33:58.869 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.835+0000 7f5aefe00640 1 -- 172.21.15.145:0/1180297608 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af0014620 con 0x7f5af80b2950 2024-05-09T04:33:58.869 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.835+0000 7f5aefe00640 1 -- 172.21.15.145:0/1180297608 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5af0019070 con 0x7f5af80b2950 2024-05-09T04:33:58.869 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.835+0000 7f5aefe00640 1 -- 172.21.15.145:0/1180297608 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af0014a00 con 0x7f5af80b2950 2024-05-09T04:33:58.869 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.835+0000 7fe97be00640 1 --2- 172.21.15.145:0/551572541 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe97c163910 0x7fe97c14f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.870 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.835+0000 7fe97be00640 1 -- 172.21.15.145:0/551572541 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe97c0d1290 msgr2=0x7fe97c0c6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.870 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.835+0000 7fe97be00640 1 --2- 172.21.15.145:0/551572541 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe97c0d1290 0x7fe97c0c6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.870 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.835+0000 7fe97be00640 1 -- 172.21.15.145:0/551572541 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe97c14ec70 con 0x7fe97c0c49d0 2024-05-09T04:33:58.870 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.833+0000 7f2094c00640 1 -- 172.21.15.145:0/1784680129 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f208401c460 con 0x7f209815f100 2024-05-09T04:33:58.870 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.834+0000 7f2094c00640 1 --2- 172.21.15.145:0/1784680129 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f20780763d0 0x7f2078078890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.870 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.835+0000 7f2094c00640 1 -- 172.21.15.145:0/1784680129 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2084097cf0 con 0x7f209815f100 2024-05-09T04:33:58.870 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.835+0000 7f5afd400640 1 -- 172.21.15.145:0/1180297608 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5af815c9c0 con 0x7f5af80b2950 2024-05-09T04:33:58.870 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.835+0000 7f5afd400640 1 -- 172.21.15.145:0/1180297608 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5af815cf80 con 0x7f5af80b2950 2024-05-09T04:33:58.871 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.836+0000 7f0e51a00640 1 Processor -- start 2024-05-09T04:33:58.871 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.836+0000 7f6488800640 1 -- 172.21.15.145:0/3748979104 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f64800cf170 con 0x7f64800d3c80 2024-05-09T04:33:58.871 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.836+0000 7f0e51a00640 1 -- start start 2024-05-09T04:33:58.872 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.836+0000 7fe97be00640 1 --2- 172.21.15.145:0/551572541 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe97c0c49d0 0x7fe97c1660d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe96c031000 tx=0x7fe96c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:58.872 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.836+0000 7f209ce00640 1 --2- 172.21.15.145:0/1784680129 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f20780763d0 0x7f2078078890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:58.872 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.836+0000 7f3294c00640 1 --2- 172.21.15.145:0/608230380 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3288056ab0 0x7f3288185310 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f32900515a0 tx=0x7f3290069fc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:58.872 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.837+0000 7fe973e00640 1 -- 172.21.15.145:0/551572541 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96c041020 con 0x7fe97c0c49d0 2024-05-09T04:33:58.872 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.837+0000 7fe973e00640 1 -- 172.21.15.145:0/551572541 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe96c0307a0 con 0x7fe97c0c49d0 2024-05-09T04:33:58.873 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.837+0000 7fe973e00640 1 -- 172.21.15.145:0/551572541 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96c037040 con 0x7fe97c0c49d0 2024-05-09T04:33:58.873 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.837+0000 7fe983000640 1 -- 172.21.15.145:0/551572541 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe97c166760 con 0x7fe97c0c49d0 2024-05-09T04:33:58.873 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.837+0000 7f5afd400640 1 -- 172.21.15.145:0/1180297608 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5ad0001030 con 0x7f5af80b2950 2024-05-09T04:33:58.873 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.837+0000 7fe983000640 1 -- 172.21.15.145:0/551572541 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe97c14ef30 con 0x7fe97c0c49d0 2024-05-09T04:33:58.874 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.837+0000 7f209ce00640 1 --2- 172.21.15.145:0/1784680129 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f20780763d0 0x7f2078078890 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f208c005fd0 tx=0x7f208c007e60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:58.874 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.838+0000 7f2094c00640 1 --2- 172.21.15.145:0/1784680129 >> [v2:172.21.15.145:6824/1845225555,v1:172.21.15.145:6825/1845225555] conn(0x7f207807e5f0 0x7f2078080a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.874 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.838+0000 7f2094c00640 1 -- 172.21.15.145:0/1784680129 --> [v2:172.21.15.145:6824/1845225555,v1:172.21.15.145:6825/1845225555] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f208400ee50 con 0x7f207807e5f0 2024-05-09T04:33:58.874 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.838+0000 7f2094c00640 1 -- 172.21.15.145:0/1784680129 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f20840980e0 con 0x7f209815f100 2024-05-09T04:33:58.874 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.838+0000 7fe983000640 1 -- 172.21.15.145:0/551572541 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe97c166c80 con 0x7fe97c0c49d0 2024-05-09T04:33:58.875 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.838+0000 7f209d800640 1 --2- 172.21.15.145:0/1784680129 >> [v2:172.21.15.145:6824/1845225555,v1:172.21.15.145:6825/1845225555] conn(0x7f207807e5f0 0x7f2078080a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:58.875 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.838+0000 7f328cc00640 1 -- 172.21.15.145:0/608230380 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329009a070 con 0x7f3288056ab0 2024-05-09T04:33:58.875 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.838+0000 7f328cc00640 1 -- 172.21.15.145:0/608230380 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3290074ac0 con 0x7f3288056ab0 2024-05-09T04:33:58.876 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.838+0000 7f328cc00640 1 -- 172.21.15.145:0/608230380 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329006c680 con 0x7f3288056ab0 2024-05-09T04:33:58.876 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.838+0000 7f3296000640 1 -- 172.21.15.145:0/608230380 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f328803bbd0 con 0x7f3288056ab0 2024-05-09T04:33:58.876 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.838+0000 7f0e51a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0e440078e0 0x7f0e44179d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.876 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.838+0000 7f0e51a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e4403e840 0x7f0e4417d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.876 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.838+0000 7f0e51a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0e440e30c0 0x7f0e440d84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.876 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.839+0000 7f0e51a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0e44054300 con 0x7f0e440078e0 2024-05-09T04:33:58.876 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.839+0000 7f0e51a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0e440e34a0 con 0x7f0e4403e840 2024-05-09T04:33:58.877 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.839+0000 7f0e51a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0e44054480 con 0x7f0e440e30c0 2024-05-09T04:33:58.877 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.839+0000 7f3296000640 1 -- 172.21.15.145:0/608230380 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f328803c190 con 0x7f3288056ab0 2024-05-09T04:33:58.877 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.839+0000 7f892fe00640 1 -- 172.21.15.145:0/2571431195 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f892801c020 con 0x7f89380715a0 2024-05-09T04:33:58.877 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.840+0000 7f0e51000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0e440e30c0 0x7f0e440d84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:58.878 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.840+0000 7f0e51000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0e440e30c0 0x7f0e440d84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.145:47430/0 (socket says 172.21.15.145:47430) 2024-05-09T04:33:58.878 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.840+0000 7f0e51000640 1 -- 172.21.15.145:0/789476549 learned_addr learned my addr 172.21.15.145:0/789476549 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:58.878 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.840+0000 7f647a000640 1 -- 172.21.15.145:0/3748979104 <== mon.2 v2:172.21.15.192:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f646c050430 con 0x7f64800d3c80 2024-05-09T04:33:58.878 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.840+0000 7f0e4b400640 1 --2- 172.21.15.145:0/789476549 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e4403e840 0x7f0e4417d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:58.878 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.840+0000 7f892fe00640 1 --2- 172.21.15.145:0/2571431195 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f89180762d0 0x7f8918078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.879 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.840+0000 7f892fe00640 1 -- 172.21.15.145:0/2571431195 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8928097b50 con 0x7f89380715a0 2024-05-09T04:33:58.879 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.840+0000 7f892fe00640 1 --2- 172.21.15.145:0/2571431195 >> [v2:172.21.15.145:6816/2168958993,v1:172.21.15.145:6817/2168958993] conn(0x7f891807e470 0x7f8918080890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.879 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.840+0000 7f892fe00640 1 -- 172.21.15.145:0/2571431195 --> [v2:172.21.15.145:6816/2168958993,v1:172.21.15.145:6817/2168958993] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8928014e50 con 0x7f891807e470 2024-05-09T04:33:58.879 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.840+0000 7f892fe00640 1 -- 172.21.15.145:0/2571431195 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f8928097f00 con 0x7f89380715a0 2024-05-09T04:33:58.879 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.841+0000 7f0e4be00640 1 --2- 172.21.15.145:0/789476549 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0e440078e0 0x7f0e44179d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:58.879 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.841+0000 7f209d800640 1 --2- 172.21.15.145:0/1784680129 >> [v2:172.21.15.145:6824/1845225555,v1:172.21.15.145:6825/1845225555] conn(0x7f207807e5f0 0x7f2078080a10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:58.880 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.841+0000 7f0e51000640 1 -- 172.21.15.145:0/789476549 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e4403e840 msgr2=0x7f0e4417d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.880 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.841+0000 7f0e51000640 1 --2- 172.21.15.145:0/789476549 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e4403e840 0x7f0e4417d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.880 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.841+0000 7f0e51000640 1 -- 172.21.15.145:0/789476549 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0e440078e0 msgr2=0x7f0e44179d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.880 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.841+0000 7f0e51000640 1 --2- 172.21.15.145:0/789476549 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0e440078e0 0x7f0e44179d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.880 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.841+0000 7f0e51000640 1 -- 172.21.15.145:0/789476549 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e4417d6b0 con 0x7f0e440e30c0 2024-05-09T04:33:58.880 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.841+0000 7f8937e00640 1 --2- 172.21.15.145:0/2571431195 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f89180762d0 0x7f8918078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:58.881 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.841+0000 7f3296000640 1 -- 172.21.15.145:0/608230380 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f328803c560 con 0x7f3288056ab0 2024-05-09T04:33:58.881 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.842+0000 7f647a000640 1 --2- 172.21.15.145:0/3748979104 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f6460076300 0x7f64600787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.881 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.842+0000 7f2094c00640 1 -- 172.21.15.145:0/1784680129 <== osd.1 v2:172.21.15.145:6824/1845225555 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f208400ee50 con 0x7f207807e5f0 2024-05-09T04:33:58.881 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.842+0000 7f647a000640 1 -- 172.21.15.145:0/3748979104 <== mon.2 v2:172.21.15.192:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f646c0b8c00 con 0x7f64800d3c80 2024-05-09T04:33:58.882 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.842+0000 7f8937e00640 1 --2- 172.21.15.145:0/2571431195 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f89180762d0 0x7f8918078790 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f892400b9f0 tx=0x7f8924030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:58.882 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.842+0000 7f0e51000640 1 --2- 172.21.15.145:0/789476549 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0e440e30c0 0x7f0e440d84f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c051a60 tx=0x7f0e4c06c270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:58.882 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.842+0000 7f647a000640 1 --2- 172.21.15.145:0/3748979104 >> [v2:172.21.15.192:6800/1328373268,v1:172.21.15.192:6801/1328373268] conn(0x7f646007e520 0x7f6460080940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.883 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.842+0000 7f647a000640 1 -- 172.21.15.145:0/3748979104 --> [v2:172.21.15.192:6800/1328373268,v1:172.21.15.192:6801/1328373268] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f646c032440 con 0x7f646007e520 2024-05-09T04:33:58.883 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.843+0000 7f647a000640 1 -- 172.21.15.145:0/3748979104 <== mon.2 v2:172.21.15.192:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f646c0b8ff0 con 0x7f64800d3c80 2024-05-09T04:33:58.883 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.843+0000 7fe973e00640 1 -- 172.21.15.145:0/551572541 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe96c044020 con 0x7fe97c0c49d0 2024-05-09T04:33:58.883 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.843+0000 7f5aefe00640 1 -- 172.21.15.145:0/1180297608 <== mon.2 v2:172.21.15.192:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5af0014ba0 con 0x7f5af80b2950 2024-05-09T04:33:58.883 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.843+0000 7f893ce00640 1 --2- 172.21.15.145:0/2571431195 >> [v2:172.21.15.145:6816/2168958993,v1:172.21.15.145:6817/2168958993] conn(0x7f891807e470 0x7f8918080890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:58.884 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.844+0000 7f0e3fe00640 1 -- 172.21.15.145:0/789476549 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c0b2050 con 0x7f0e440e30c0 2024-05-09T04:33:58.884 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.844+0000 7f0e3fe00640 1 -- 172.21.15.145:0/789476549 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e4c0b3040 con 0x7f0e440e30c0 2024-05-09T04:33:58.884 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.844+0000 7f0e3fe00640 1 -- 172.21.15.145:0/789476549 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c0adc90 con 0x7f0e440e30c0 2024-05-09T04:33:58.884 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.844+0000 7fe973e00640 1 --2- 172.21.15.145:0/551572541 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fe95c076300 0x7fe95c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.884 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.844+0000 7fe973e00640 1 -- 172.21.15.145:0/551572541 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe96c0b81c0 con 0x7fe97c0c49d0 2024-05-09T04:33:58.885 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.843+0000 7f6485e00640 1 --2- 172.21.15.145:0/3748979104 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f6460076300 0x7f64600787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:58.885 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.844+0000 7f0e51a00640 1 -- 172.21.15.145:0/789476549 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e44052150 con 0x7f0e440e30c0 2024-05-09T04:33:58.885 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.844+0000 7fe97b400640 1 --2- 172.21.15.145:0/551572541 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fe95c076300 0x7fe95c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:58.885 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.844+0000 7fe973e00640 1 --2- 172.21.15.145:0/551572541 >> [v2:172.21.15.192:6808/3368512157,v1:172.21.15.192:6809/3368512157] conn(0x7fe95c07e4a0 0x7fe95c0808c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.885 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.844+0000 7fe973e00640 1 -- 172.21.15.145:0/551572541 --> [v2:172.21.15.192:6808/3368512157,v1:172.21.15.192:6809/3368512157] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe96c031ec0 con 0x7fe95c07e4a0 2024-05-09T04:33:58.886 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.844+0000 7f6486800640 1 --2- 172.21.15.145:0/3748979104 >> [v2:172.21.15.192:6800/1328373268,v1:172.21.15.192:6801/1328373268] conn(0x7f646007e520 0x7f6460080940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:58.886 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.844+0000 7f6485e00640 1 --2- 172.21.15.145:0/3748979104 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f6460076300 0x7f64600787c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f648015d000 tx=0x7f6474009060 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:58.886 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.844+0000 7fe973e00640 1 -- 172.21.15.145:0/551572541 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fe96c0b8650 con 0x7fe97c0c49d0 2024-05-09T04:33:58.886 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.845+0000 7f0e51a00640 1 -- 172.21.15.145:0/789476549 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e44179280 con 0x7f0e440e30c0 2024-05-09T04:33:58.887 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.845+0000 7f328cc00640 1 -- 172.21.15.145:0/608230380 <== mon.2 v2:172.21.15.192:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f329009d070 con 0x7f3288056ab0 2024-05-09T04:33:58.887 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.845+0000 7fe981000640 1 --2- 172.21.15.145:0/551572541 >> [v2:172.21.15.192:6808/3368512157,v1:172.21.15.192:6809/3368512157] conn(0x7fe95c07e4a0 0x7fe95c0808c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:58.887 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.845+0000 7f5aefe00640 1 --2- 172.21.15.145:0/1180297608 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f5ad80762d0 0x7f5ad8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.887 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.845+0000 7f6486800640 1 --2- 172.21.15.145:0/3748979104 >> [v2:172.21.15.192:6800/1328373268,v1:172.21.15.192:6801/1328373268] conn(0x7f646007e520 0x7f6460080940 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:58.888 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.846+0000 7f5aefe00640 1 -- 172.21.15.145:0/1180297608 <== mon.2 v2:172.21.15.192:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5af0098970 con 0x7f5af80b2950 2024-05-09T04:33:58.888 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.846+0000 7f5af7400640 1 --2- 172.21.15.145:0/1180297608 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f5ad80762d0 0x7f5ad8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:58.888 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.846+0000 7f328cc00640 1 --2- 172.21.15.145:0/608230380 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f3270076320 0x7f32700787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.888 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.846+0000 7f328cc00640 1 -- 172.21.15.145:0/608230380 <== mon.2 v2:172.21.15.192:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3290112ff0 con 0x7f3288056ab0 2024-05-09T04:33:58.889 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.846+0000 7fe981000640 1 --2- 172.21.15.145:0/551572541 >> [v2:172.21.15.192:6808/3368512157,v1:172.21.15.192:6809/3368512157] conn(0x7fe95c07e4a0 0x7fe95c0808c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:58.889 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.846+0000 7f328cc00640 1 --2- 172.21.15.145:0/608230380 >> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] conn(0x7f327007e540 0x7f3270080960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.889 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.846+0000 7f328cc00640 1 -- 172.21.15.145:0/608230380 --> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3270081010 con 0x7f327007e540 2024-05-09T04:33:58.889 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.846+0000 7f5aefe00640 1 --2- 172.21.15.145:0/1180297608 >> [v2:172.21.15.192:6816/3117700995,v1:172.21.15.192:6818/3117700995] conn(0x7f5ad807e470 0x7f5ad8080890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.889 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.846+0000 7f5aefe00640 1 -- 172.21.15.145:0/1180297608 --> [v2:172.21.15.192:6816/3117700995,v1:172.21.15.192:6818/3117700995] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5af0015ec0 con 0x7f5ad807e470 2024-05-09T04:33:58.889 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.846+0000 7f5aefe00640 1 -- 172.21.15.145:0/1180297608 <== mon.2 v2:172.21.15.192:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f5af0098d20 con 0x7f5af80b2950 2024-05-09T04:33:58.890 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.846+0000 7f328cc00640 1 -- 172.21.15.145:0/608230380 <== mon.2 v2:172.21.15.192:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f3290113440 con 0x7f3288056ab0 2024-05-09T04:33:58.890 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.847+0000 7f328fe00640 1 --2- 172.21.15.145:0/608230380 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f3270076320 0x7f32700787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:58.890 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.847+0000 7fe97b400640 1 --2- 172.21.15.145:0/551572541 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fe95c076300 0x7fe95c0787c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe968000f60 tx=0x7fe9680079b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:58.890 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.847+0000 7f3295600640 1 --2- 172.21.15.145:0/608230380 >> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] conn(0x7f327007e540 0x7f3270080960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:58.891 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.847+0000 7f5af7400640 1 --2- 172.21.15.145:0/1180297608 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f5ad80762d0 0x7f5ad8078790 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4034000 tx=0x7f5ae402c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:58.891 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.847+0000 7f0e51a00640 1 -- 172.21.15.145:0/789476549 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0e44179940 con 0x7f0e440e30c0 2024-05-09T04:33:58.891 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.847+0000 7f893ce00640 1 --2- 172.21.15.145:0/2571431195 >> [v2:172.21.15.145:6816/2168958993,v1:172.21.15.145:6817/2168958993] conn(0x7f891807e470 0x7f8918080890 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:58.891 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.848+0000 7f647a000640 1 -- 172.21.15.145:0/3748979104 <== osd.5 v2:172.21.15.192:6800/1328373268 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f646c032440 con 0x7f646007e520 2024-05-09T04:33:58.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.848+0000 7f3295600640 1 --2- 172.21.15.145:0/608230380 >> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] conn(0x7f327007e540 0x7f3270080960 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:58.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.848+0000 7f328fe00640 1 --2- 172.21.15.145:0/608230380 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f3270076320 0x7f32700787e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3284034000 tx=0x7f328402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:58.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.848+0000 7f5afca00640 1 --2- 172.21.15.145:0/1180297608 >> [v2:172.21.15.192:6816/3117700995,v1:172.21.15.192:6818/3117700995] conn(0x7f5ad807e470 0x7f5ad8080890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:58.893 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.848+0000 7f0e3fe00640 1 -- 172.21.15.145:0/789476549 <== mon.2 v2:172.21.15.192:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0e4c0c0030 con 0x7f0e440e30c0 2024-05-09T04:33:58.893 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.848+0000 7f892fe00640 1 -- 172.21.15.145:0/2571431195 <== osd.2 v2:172.21.15.145:6816/2168958993 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8928014e50 con 0x7f891807e470 2024-05-09T04:33:58.893 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.849+0000 7f5afca00640 1 --2- 172.21.15.145:0/1180297608 >> [v2:172.21.15.192:6816/3117700995,v1:172.21.15.192:6818/3117700995] conn(0x7f5ad807e470 0x7f5ad8080890 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:58.893 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.850+0000 7f0e3fe00640 1 --2- 172.21.15.145:0/789476549 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f0e2c076320 0x7f0e2c0787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.895 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.850+0000 7f328cc00640 1 -- 172.21.15.145:0/608230380 <== osd.3 v2:172.21.15.145:6808/337009826 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3270081010 con 0x7f327007e540 2024-05-09T04:33:58.896 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.850+0000 7f0e3fe00640 1 -- 172.21.15.145:0/789476549 <== mon.2 v2:172.21.15.192:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0e4c133850 con 0x7f0e440e30c0 2024-05-09T04:33:58.896 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.850+0000 7f0e3fe00640 1 --2- 172.21.15.145:0/789476549 >> [v2:172.21.15.192:6817/1868011057,v1:172.21.15.192:6820/1868011057] conn(0x7f0e2c07e540 0x7f0e2c080960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:58.896 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.850+0000 7f0e3fe00640 1 -- 172.21.15.145:0/789476549 --> [v2:172.21.15.192:6817/1868011057,v1:172.21.15.192:6820/1868011057] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0e4c0bfe80 con 0x7f0e2c07e540 2024-05-09T04:33:58.896 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.851+0000 7f0e4be00640 1 --2- 172.21.15.145:0/789476549 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f0e2c076320 0x7f0e2c0787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:58.896 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.851+0000 7f0e3fe00640 1 -- 172.21.15.145:0/789476549 <== mon.2 v2:172.21.15.192:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f0e4c1382c0 con 0x7f0e440e30c0 2024-05-09T04:33:58.896 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.851+0000 7f0e4b400640 1 --2- 172.21.15.145:0/789476549 >> [v2:172.21.15.192:6817/1868011057,v1:172.21.15.192:6820/1868011057] conn(0x7f0e2c07e540 0x7f0e2c080960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:58.896 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.851+0000 7f0e4be00640 1 --2- 172.21.15.145:0/789476549 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f0e2c076320 0x7f0e2c0787e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0e40000f60 tx=0x7f0e40007d20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:58.896 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.852+0000 7f5aefe00640 1 -- 172.21.15.145:0/1180297608 <== osd.4 v2:172.21.15.192:6816/3117700995 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5af0015ec0 con 0x7f5ad807e470 2024-05-09T04:33:58.897 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.853+0000 7fe973e00640 1 -- 172.21.15.145:0/551572541 <== osd.7 v2:172.21.15.192:6808/3368512157 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe96c031ec0 con 0x7fe95c07e4a0 2024-05-09T04:33:58.897 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.856+0000 7f0e4b400640 1 --2- 172.21.15.145:0/789476549 >> [v2:172.21.15.192:6817/1868011057,v1:172.21.15.192:6820/1868011057] conn(0x7f0e2c07e540 0x7f0e2c080960 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:58.897 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.858+0000 7f8ae4200640 1 -- 172.21.15.145:0/1795159111 --> [v2:172.21.15.145:6800/3182572912,v1:172.21.15.145:6801/3182572912] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8ad4037fa0 con 0x7f8abc07e5c0 2024-05-09T04:33:58.898 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.860+0000 7f0e3fe00640 1 -- 172.21.15.145:0/789476549 <== osd.6 v2:172.21.15.192:6817/1868011057 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0e4c0bfe80 con 0x7f0e2c07e540 2024-05-09T04:33:58.898 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.863+0000 7f8ad9600640 1 -- 172.21.15.145:0/1795159111 <== osd.0 v2:172.21.15.145:6800/3182572912 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8ad4037fa0 con 0x7f8abc07e5c0 2024-05-09T04:33:58.899 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.864+0000 7f8ae4200640 1 -- 172.21.15.145:0/1795159111 >> [v2:172.21.15.145:6800/3182572912,v1:172.21.15.145:6801/3182572912] conn(0x7f8abc07e5c0 msgr2=0x7f8abc0809e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.901 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.864+0000 7f8ae4200640 1 --2- 172.21.15.145:0/1795159111 >> [v2:172.21.15.145:6800/3182572912,v1:172.21.15.145:6801/3182572912] conn(0x7f8abc07e5c0 0x7f8abc0809e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.901 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.865+0000 7f8ae4200640 1 -- 172.21.15.145:0/1795159111 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f8abc0763a0 msgr2=0x7f8abc078860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.902 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.865+0000 7f8ae4200640 1 --2- 172.21.15.145:0/1795159111 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f8abc0763a0 0x7f8abc078860 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8000c00 tx=0x7f8ac80079c0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.902 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.865+0000 7f8ae4200640 1 -- 172.21.15.145:0/1795159111 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8ad40e3150 msgr2=0x7f8ad417bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.902 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.865+0000 7f8ae4200640 1 --2- 172.21.15.145:0/1795159111 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8ad40e3150 0x7f8ad417bb30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8adc074fe0 tx=0x7f8adc069fc0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.902 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.865+0000 7f8ae2200640 1 -- 172.21.15.145:0/1795159111 reap_dead start 2024-05-09T04:33:58.902 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.865+0000 7f8ae4200640 1 -- 172.21.15.145:0/1795159111 shutdown_connections 2024-05-09T04:33:58.902 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.865+0000 7f8ae4200640 1 -- 172.21.15.145:0/1795159111 >> 172.21.15.145:0/1795159111 conn(0x7f8ad4058ed0 msgr2=0x7f8ad40392b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:58.902 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.866+0000 7f8ae4200640 1 -- 172.21.15.145:0/1795159111 shutdown_connections 2024-05-09T04:33:58.902 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.866+0000 7f8ae4200640 1 -- 172.21.15.145:0/1795159111 wait complete. 2024-05-09T04:33:58.902 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.874+0000 7f209e200640 1 -- 172.21.15.145:0/1784680129 --> [v2:172.21.15.145:6824/1845225555,v1:172.21.15.145:6825/1845225555] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2098076780 con 0x7f207807e5f0 2024-05-09T04:33:58.903 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.874+0000 7f2094c00640 1 -- 172.21.15.145:0/1784680129 <== osd.1 v2:172.21.15.145:6824/1845225555 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2098076780 con 0x7f207807e5f0 2024-05-09T04:33:58.903 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.875+0000 7f209e200640 1 -- 172.21.15.145:0/1784680129 >> [v2:172.21.15.145:6824/1845225555,v1:172.21.15.145:6825/1845225555] conn(0x7f207807e5f0 msgr2=0x7f2078080a10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.903 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.875+0000 7f209e200640 1 --2- 172.21.15.145:0/1784680129 >> [v2:172.21.15.145:6824/1845225555,v1:172.21.15.145:6825/1845225555] conn(0x7f207807e5f0 0x7f2078080a10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.903 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.875+0000 7f209e200640 1 -- 172.21.15.145:0/1784680129 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f20780763d0 msgr2=0x7f2078078890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.904 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.875+0000 7f209e200640 1 --2- 172.21.15.145:0/1784680129 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f20780763d0 0x7f2078078890 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f208c005fd0 tx=0x7f208c007e60 comp rx=0 tx=0).stop 2024-05-09T04:33:58.904 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.875+0000 7f209e200640 1 -- 172.21.15.145:0/1784680129 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f209815f100 msgr2=0x7f209814b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.904 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.875+0000 7f209e200640 1 --2- 172.21.15.145:0/1784680129 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f209815f100 0x7f209814b690 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f208400c430 tx=0x7f208400c8f0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.904 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.876+0000 7f209d800640 1 -- 172.21.15.145:0/1784680129 reap_dead start 2024-05-09T04:33:58.904 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.876+0000 7f209e200640 1 -- 172.21.15.145:0/1784680129 shutdown_connections 2024-05-09T04:33:58.904 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.876+0000 7f209e200640 1 -- 172.21.15.145:0/1784680129 >> 172.21.15.145:0/1784680129 conn(0x7f209809e2e0 msgr2=0x7f20980b94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:58.904 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.876+0000 7f209e200640 1 -- 172.21.15.145:0/1784680129 shutdown_connections 2024-05-09T04:33:58.904 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.876+0000 7f209e200640 1 -- 172.21.15.145:0/1784680129 wait complete. 2024-05-09T04:33:58.904 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.879+0000 7f6488800640 1 -- 172.21.15.145:0/3748979104 --> [v2:172.21.15.192:6800/1328373268,v1:172.21.15.192:6801/1328373268] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f64800694e0 con 0x7f646007e520 2024-05-09T04:33:58.904 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.880+0000 7f647a000640 1 -- 172.21.15.145:0/3748979104 <== osd.5 v2:172.21.15.192:6800/1328373268 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f64800694e0 con 0x7f646007e520 2024-05-09T04:33:58.905 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.882+0000 7f6488800640 1 -- 172.21.15.145:0/3748979104 >> [v2:172.21.15.192:6800/1328373268,v1:172.21.15.192:6801/1328373268] conn(0x7f646007e520 msgr2=0x7f6460080940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.905 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.882+0000 7f6488800640 1 --2- 172.21.15.145:0/3748979104 >> [v2:172.21.15.192:6800/1328373268,v1:172.21.15.192:6801/1328373268] conn(0x7f646007e520 0x7f6460080940 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.906 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.882+0000 7f6488800640 1 -- 172.21.15.145:0/3748979104 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f6460076300 msgr2=0x7f64600787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.906 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.882+0000 7f6488800640 1 --2- 172.21.15.145:0/3748979104 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f6460076300 0x7f64600787c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f648015d000 tx=0x7f6474009060 comp rx=0 tx=0).stop 2024-05-09T04:33:58.906 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.882+0000 7f6488800640 1 -- 172.21.15.145:0/3748979104 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f64800d3c80 msgr2=0x7f648015c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.906 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.882+0000 7f6488800640 1 --2- 172.21.15.145:0/3748979104 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f64800d3c80 0x7f648015c700 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f646c0090b0 tx=0x7f646c0046c0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.906 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.883+0000 7f6486800640 1 -- 172.21.15.145:0/3748979104 reap_dead start 2024-05-09T04:33:58.906 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.883+0000 7f6488800640 1 -- 172.21.15.145:0/3748979104 shutdown_connections 2024-05-09T04:33:58.906 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.883+0000 7f6488800640 1 -- 172.21.15.145:0/3748979104 >> 172.21.15.145:0/3748979104 conn(0x7f648009e4a0 msgr2=0x7f64800b4d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:58.906 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.883+0000 7f6488800640 1 -- 172.21.15.145:0/3748979104 shutdown_connections 2024-05-09T04:33:58.906 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.883+0000 7f6488800640 1 -- 172.21.15.145:0/3748979104 wait complete. 2024-05-09T04:33:58.906 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.882+0000 7f3296000640 1 -- 172.21.15.145:0/608230380 --> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3288050ed0 con 0x7f327007e540 2024-05-09T04:33:58.907 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.884+0000 7f893ee00640 1 -- 172.21.15.145:0/2571431195 --> [v2:172.21.15.145:6816/2168958993,v1:172.21.15.145:6817/2168958993] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8910002dd0 con 0x7f891807e470 2024-05-09T04:33:58.907 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.885+0000 7f328cc00640 1 -- 172.21.15.145:0/608230380 <== osd.3 v2:172.21.15.145:6808/337009826 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3288050ed0 con 0x7f327007e540 2024-05-09T04:33:58.907 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.885+0000 7f892fe00640 1 -- 172.21.15.145:0/2571431195 <== osd.2 v2:172.21.15.145:6816/2168958993 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8910002dd0 con 0x7f891807e470 2024-05-09T04:33:58.908 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.886+0000 7f3296000640 1 -- 172.21.15.145:0/608230380 >> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] conn(0x7f327007e540 msgr2=0x7f3270080960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.908 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.886+0000 7f3296000640 1 --2- 172.21.15.145:0/608230380 >> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] conn(0x7f327007e540 0x7f3270080960 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.908 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.886+0000 7f3296000640 1 -- 172.21.15.145:0/608230380 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f3270076320 msgr2=0x7f32700787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.909 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.886+0000 7f893ee00640 1 -- 172.21.15.145:0/2571431195 >> [v2:172.21.15.145:6816/2168958993,v1:172.21.15.145:6817/2168958993] conn(0x7f891807e470 msgr2=0x7f8918080890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.909 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.886+0000 7f893ee00640 1 --2- 172.21.15.145:0/2571431195 >> [v2:172.21.15.145:6816/2168958993,v1:172.21.15.145:6817/2168958993] conn(0x7f891807e470 0x7f8918080890 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.909 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.886+0000 7f893ee00640 1 -- 172.21.15.145:0/2571431195 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f89180762d0 msgr2=0x7f8918078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.909 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.886+0000 7f893ee00640 1 --2- 172.21.15.145:0/2571431195 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f89180762d0 0x7f8918078790 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f892400b9f0 tx=0x7f8924030000 comp rx=0 tx=0).stop 2024-05-09T04:33:58.909 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.887+0000 7f3296000640 1 --2- 172.21.15.145:0/608230380 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f3270076320 0x7f32700787e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3284034000 tx=0x7f328402e000 comp rx=0 tx=0).stop 2024-05-09T04:33:58.909 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.887+0000 7f5afd400640 1 -- 172.21.15.145:0/1180297608 --> [v2:172.21.15.192:6816/3117700995,v1:172.21.15.192:6818/3117700995] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5ad0002dd0 con 0x7f5ad807e470 2024-05-09T04:33:58.910 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.887+0000 7f893ee00640 1 -- 172.21.15.145:0/2571431195 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f89380715a0 msgr2=0x7f89380cc8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.910 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.887+0000 7f893ee00640 1 --2- 172.21.15.145:0/2571431195 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f89380715a0 0x7f89380cc8a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f892800f4e0 tx=0x7f892800f8e0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.910 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.887+0000 7f3296000640 1 -- 172.21.15.145:0/608230380 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3288056ab0 msgr2=0x7f3288185310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.910 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.887+0000 7f3296000640 1 --2- 172.21.15.145:0/608230380 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3288056ab0 0x7f3288185310 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f32900515a0 tx=0x7f3290069fc0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.911 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.887+0000 7f3295600640 1 -- 172.21.15.145:0/608230380 reap_dead start 2024-05-09T04:33:58.911 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.887+0000 7f3296000640 1 -- 172.21.15.145:0/608230380 shutdown_connections 2024-05-09T04:33:58.911 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.887+0000 7f893ce00640 1 -- 172.21.15.145:0/2571431195 reap_dead start 2024-05-09T04:33:58.911 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.887+0000 7f893ee00640 1 -- 172.21.15.145:0/2571431195 shutdown_connections 2024-05-09T04:33:58.911 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.887+0000 7f893ee00640 1 -- 172.21.15.145:0/2571431195 >> 172.21.15.145:0/2571431195 conn(0x7f89380d4fb0 msgr2=0x7f893806d630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:58.911 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.887+0000 7f3296000640 1 -- 172.21.15.145:0/608230380 >> 172.21.15.145:0/608230380 conn(0x7f3288058ee0 msgr2=0x7f3288038f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:58.912 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.887+0000 7f893ee00640 1 -- 172.21.15.145:0/2571431195 shutdown_connections 2024-05-09T04:33:58.912 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.887+0000 7f893ee00640 1 -- 172.21.15.145:0/2571431195 wait complete. 2024-05-09T04:33:58.912 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.887+0000 7fe983000640 1 -- 172.21.15.145:0/551572541 --> [v2:172.21.15.192:6808/3368512157,v1:172.21.15.192:6809/3368512157] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe97c14f170 con 0x7fe95c07e4a0 2024-05-09T04:33:58.912 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.887+0000 7f3296000640 1 -- 172.21.15.145:0/608230380 shutdown_connections 2024-05-09T04:33:58.913 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.888+0000 7f5aefe00640 1 -- 172.21.15.145:0/1180297608 <== osd.4 v2:172.21.15.192:6816/3117700995 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5ad0002dd0 con 0x7f5ad807e470 2024-05-09T04:33:58.913 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.888+0000 7f3296000640 1 -- 172.21.15.145:0/608230380 wait complete. 2024-05-09T04:33:58.913 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.888+0000 7fe973e00640 1 -- 172.21.15.145:0/551572541 <== osd.7 v2:172.21.15.192:6808/3368512157 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe974009db0 con 0x7fe95c07e4a0 2024-05-09T04:33:58.914 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.889+0000 7fe983000640 1 -- 172.21.15.145:0/551572541 >> [v2:172.21.15.192:6808/3368512157,v1:172.21.15.192:6809/3368512157] conn(0x7fe95c07e4a0 msgr2=0x7fe95c0808c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.914 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.889+0000 7fe983000640 1 --2- 172.21.15.145:0/551572541 >> [v2:172.21.15.192:6808/3368512157,v1:172.21.15.192:6809/3368512157] conn(0x7fe95c07e4a0 0x7fe95c0808c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.914 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.889+0000 7f5afd400640 1 -- 172.21.15.145:0/1180297608 >> [v2:172.21.15.192:6816/3117700995,v1:172.21.15.192:6818/3117700995] conn(0x7f5ad807e470 msgr2=0x7f5ad8080890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.914 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.889+0000 7f5afd400640 1 --2- 172.21.15.145:0/1180297608 >> [v2:172.21.15.192:6816/3117700995,v1:172.21.15.192:6818/3117700995] conn(0x7f5ad807e470 0x7f5ad8080890 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.915 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.890+0000 7fe983000640 1 -- 172.21.15.145:0/551572541 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fe95c076300 msgr2=0x7fe95c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.915 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.890+0000 7fe983000640 1 --2- 172.21.15.145:0/551572541 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fe95c076300 0x7fe95c0787c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe968000f60 tx=0x7fe9680079b0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.915 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.890+0000 7f5afd400640 1 -- 172.21.15.145:0/1180297608 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f5ad80762d0 msgr2=0x7f5ad8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.915 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.890+0000 7f5afd400640 1 --2- 172.21.15.145:0/1180297608 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f5ad80762d0 0x7f5ad8078790 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4034000 tx=0x7f5ae402c000 comp rx=0 tx=0).stop 2024-05-09T04:33:58.915 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.890+0000 7f5afd400640 1 -- 172.21.15.145:0/1180297608 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5af80b2950 msgr2=0x7f5af815d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.915 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.890+0000 7f5afd400640 1 --2- 172.21.15.145:0/1180297608 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5af80b2950 0x7f5af815d780 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5af0007e40 tx=0x7f5af000e4b0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.915 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.890+0000 7f5afca00640 1 -- 172.21.15.145:0/1180297608 reap_dead start 2024-05-09T04:33:58.915 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.890+0000 7f5afd400640 1 -- 172.21.15.145:0/1180297608 shutdown_connections 2024-05-09T04:33:58.915 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.891+0000 7f5afd400640 1 -- 172.21.15.145:0/1180297608 >> 172.21.15.145:0/1180297608 conn(0x7f5af80d5020 msgr2=0x7f5af80d5400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:58.916 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.890+0000 7fe983000640 1 -- 172.21.15.145:0/551572541 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe97c0c49d0 msgr2=0x7fe97c1660d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.916 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.890+0000 7fe983000640 1 --2- 172.21.15.145:0/551572541 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe97c0c49d0 0x7fe97c1660d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe96c031000 tx=0x7fe96c0046c0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.916 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.890+0000 7fe981000640 1 -- 172.21.15.145:0/551572541 reap_dead start 2024-05-09T04:33:58.916 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.890+0000 7fe983000640 1 -- 172.21.15.145:0/551572541 shutdown_connections 2024-05-09T04:33:58.916 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.890+0000 7fe983000640 1 -- 172.21.15.145:0/551572541 >> 172.21.15.145:0/551572541 conn(0x7fe97c0ce220 msgr2=0x7fe97c0b47c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:58.916 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.891+0000 7f5afd400640 1 -- 172.21.15.145:0/1180297608 shutdown_connections 2024-05-09T04:33:58.916 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.891+0000 7f5afd400640 1 -- 172.21.15.145:0/1180297608 wait complete. 2024-05-09T04:33:58.917 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.892+0000 7fe983000640 1 -- 172.21.15.145:0/551572541 shutdown_connections 2024-05-09T04:33:58.917 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.892+0000 7fe983000640 1 -- 172.21.15.145:0/551572541 wait complete. 2024-05-09T04:33:58.918 INFO:teuthology.orchestra.run.smithi145.stdout:55834574854 2024-05-09T04:33:58.918 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-09T04:33:58.918 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.894+0000 7f0e51a00640 1 -- 172.21.15.145:0/789476549 --> [v2:172.21.15.192:6817/1868011057,v1:172.21.15.192:6820/1868011057] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0e44179bb0 con 0x7f0e2c07e540 2024-05-09T04:33:58.918 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.896+0000 7f0e3fe00640 1 -- 172.21.15.145:0/789476549 <== osd.6 v2:172.21.15.192:6817/1868011057 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0e44179bb0 con 0x7f0e2c07e540 2024-05-09T04:33:58.919 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.897+0000 7f0e51a00640 1 -- 172.21.15.145:0/789476549 >> [v2:172.21.15.192:6817/1868011057,v1:172.21.15.192:6820/1868011057] conn(0x7f0e2c07e540 msgr2=0x7f0e2c080960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.920 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.897+0000 7f0e51a00640 1 --2- 172.21.15.145:0/789476549 >> [v2:172.21.15.192:6817/1868011057,v1:172.21.15.192:6820/1868011057] conn(0x7f0e2c07e540 0x7f0e2c080960 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:58.920 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.897+0000 7f0e51a00640 1 -- 172.21.15.145:0/789476549 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f0e2c076320 msgr2=0x7f0e2c0787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.920 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.897+0000 7f0e51a00640 1 --2- 172.21.15.145:0/789476549 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f0e2c076320 0x7f0e2c0787e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0e40000f60 tx=0x7f0e40007d20 comp rx=0 tx=0).stop 2024-05-09T04:33:58.920 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.897+0000 7f0e51a00640 1 -- 172.21.15.145:0/789476549 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0e440e30c0 msgr2=0x7f0e440d84f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:58.920 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.897+0000 7f0e51a00640 1 --2- 172.21.15.145:0/789476549 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0e440e30c0 0x7f0e440d84f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c051a60 tx=0x7f0e4c06c270 comp rx=0 tx=0).stop 2024-05-09T04:33:58.920 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.897+0000 7f0e51000640 1 -- 172.21.15.145:0/789476549 reap_dead start 2024-05-09T04:33:58.920 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.899+0000 7f0e51a00640 1 -- 172.21.15.145:0/789476549 shutdown_connections 2024-05-09T04:33:58.920 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.899+0000 7f0e51a00640 1 -- 172.21.15.145:0/789476549 >> 172.21.15.145:0/789476549 conn(0x7f0e440559e0 msgr2=0x7f0e44039bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:58.920 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.901+0000 7f0e51a00640 1 -- 172.21.15.145:0/789476549 shutdown_connections 2024-05-09T04:33:58.920 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:58.901+0000 7f0e51a00640 1 -- 172.21.15.145:0/789476549 wait complete. 2024-05-09T04:33:58.921 INFO:teuthology.orchestra.run.smithi145.stdout:55834574853 2024-05-09T04:33:58.921 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-09T04:33:58.922 INFO:teuthology.orchestra.run.smithi145.stdout:55834574853 2024-05-09T04:33:58.922 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-09T04:33:58.923 INFO:teuthology.orchestra.run.smithi145.stdout:55834574854 2024-05-09T04:33:58.923 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-09T04:33:58.924 INFO:teuthology.orchestra.run.smithi145.stdout:55834574856 2024-05-09T04:33:58.924 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-09T04:33:58.925 INFO:teuthology.orchestra.run.smithi145.stdout:55834574855 2024-05-09T04:33:58.925 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-09T04:33:58.926 INFO:teuthology.orchestra.run.smithi145.stdout:55834574854 2024-05-09T04:33:58.926 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-09T04:33:58.927 INFO:teuthology.orchestra.run.smithi145.stdout:55834574855 2024-05-09T04:33:58.928 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-09T04:33:59.250 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.250+0000 7f9132600640 1 Processor -- start 2024-05-09T04:33:59.250 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.250+0000 7f9132600640 1 -- start start 2024-05-09T04:33:59.251 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.251+0000 7f6903e00640 1 Processor -- start 2024-05-09T04:33:59.251 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.251+0000 7f6903e00640 1 -- start start 2024-05-09T04:33:59.252 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.252+0000 7f9132600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f912c06c6c0 0x7f912c06caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:59.252 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.252+0000 7f9132600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f912c06e0b0 0x7f912c06e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:59.252 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.252+0000 7f9132600640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f912c0d09e0 con 0x7f912c0cd640 2024-05-09T04:33:59.252 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.252+0000 7f9132600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f912c0d0550 con 0x7f912c06c6c0 2024-05-09T04:33:59.252 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.252+0000 7f9132600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f912c06ea50 con 0x7f912c06e0b0 2024-05-09T04:33:59.253 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.253+0000 7f6903e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f68fc15f5c0 0x7f68fc0d3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:59.253 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.253+0000 7f6903e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68fc0b6290 0x7f68fc0d3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:59.253 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.253+0000 7f6903e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68fc0d31d0 con 0x7f68fc0b4470 2024-05-09T04:33:59.253 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.253+0000 7f6903e00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f68fc0d5c40 con 0x7f68fc15f5c0 2024-05-09T04:33:59.253 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.253+0000 7f6903e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f68fc0d5f20 con 0x7f68fc0b6290 2024-05-09T04:33:59.253 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.254+0000 7f912b400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f912c06c6c0 0x7f912c06caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:59.254 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.254+0000 7f6900a00640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f68fc0b4470 0x7f68fc0cfc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.145:42524/0 (socket says 172.21.15.145:42524) 2024-05-09T04:33:59.254 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.254+0000 7f6901e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68fc0b6290 0x7f68fc0d3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:59.254 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.254+0000 7f6901e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68fc0b6290 0x7f68fc0d3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.145:47454/0 (socket says 172.21.15.145:47454) 2024-05-09T04:33:59.254 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.254+0000 7f6901e00640 1 -- 172.21.15.145:0/1747385889 learned_addr learned my addr 172.21.15.145:0/1747385889 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:59.255 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.254+0000 7f912aa00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f912c06e0b0 0x7f912c06e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:59.255 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.254+0000 7f912aa00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f912c06e0b0 0x7f912c06e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.145:47444/0 (socket says 172.21.15.145:47444) 2024-05-09T04:33:59.255 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.254+0000 7f912aa00640 1 -- 172.21.15.145:0/1633140354 learned_addr learned my addr 172.21.15.145:0/1633140354 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:59.255 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.254+0000 7f6901400640 1 --2- 172.21.15.145:0/1747385889 >> v2:172.21.15.145:3300/0 conn(0x7f68fc15f5c0 0x7f68fc0d3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:59.256 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.255+0000 7f912b400640 1 -- 172.21.15.145:0/1633140354 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f912c06e0b0 msgr2=0x7f912c06e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:59.256 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.255+0000 7f912b400640 1 --2- 172.21.15.145:0/1633140354 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f912c06e0b0 0x7f912c06e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.256 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.255+0000 7f912b400640 1 -- 172.21.15.145:0/1633140354 >> v1:172.21.15.192:6789/0 conn(0x7f912c0cd640 legacy=0x7f912c0cdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:59.256 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.255+0000 7f912b400640 1 -- 172.21.15.145:0/1633140354 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f912c0cf940 con 0x7f912c06c6c0 2024-05-09T04:33:59.256 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.256+0000 7f6901e00640 1 -- 172.21.15.145:0/1747385889 >> v2:172.21.15.145:3300/0 conn(0x7f68fc15f5c0 msgr2=0x7f68fc0d3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:59.256 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.256+0000 7f6901e00640 1 --2- 172.21.15.145:0/1747385889 >> v2:172.21.15.145:3300/0 conn(0x7f68fc15f5c0 0x7f68fc0d3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.256 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.256+0000 7f6901e00640 1 -- 172.21.15.145:0/1747385889 >> v1:172.21.15.192:6789/0 conn(0x7f68fc0b4470 legacy=0x7f68fc0cfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:59.256 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.256+0000 7f6901e00640 1 -- 172.21.15.145:0/1747385889 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68fc0b4a80 con 0x7f68fc0b6290 2024-05-09T04:33:59.257 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.256+0000 7f6901400640 1 --2- 172.21.15.145:0/1747385889 >> v2:172.21.15.145:3300/0 conn(0x7f68fc15f5c0 0x7f68fc0d3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:33:59.257 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.257+0000 7f912b400640 1 --2- 172.21.15.145:0/1633140354 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f912c06c6c0 0x7f912c06caa0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9124009d90 tx=0x7f912402a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70479e479ab41295 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:59.257 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.258+0000 7f6901e00640 1 --2- 172.21.15.145:0/1747385889 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68fc0b6290 0x7f68fc0d3ba0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f68ec009870 tx=0x7f68ec02a1f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=718a9615bcde6112 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:59.258 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.258+0000 7f912a000640 1 -- 172.21.15.145:0/1633140354 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9124031600 con 0x7f912c06c6c0 2024-05-09T04:33:59.258 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.258+0000 7f912a000640 1 -- 172.21.15.145:0/1633140354 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91240317a0 con 0x7f912c06c6c0 2024-05-09T04:33:59.258 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.258+0000 7f912a000640 1 -- 172.21.15.145:0/1633140354 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9124031b70 con 0x7f912c06c6c0 2024-05-09T04:33:59.258 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.259+0000 7f68fbe00640 1 -- 172.21.15.145:0/1747385889 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ec037070 con 0x7f68fc0b6290 2024-05-09T04:33:59.259 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.259+0000 7f9132600640 1 -- 172.21.15.145:0/1633140354 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f912c06c6c0 msgr2=0x7f912c06caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:59.259 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.259+0000 7f9132600640 1 --2- 172.21.15.145:0/1633140354 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f912c06c6c0 0x7f912c06caa0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9124009d90 tx=0x7f912402a5d0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.259 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.259+0000 7f68fbe00640 1 -- 172.21.15.145:0/1747385889 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68ec032440 con 0x7f68fc0b6290 2024-05-09T04:33:59.259 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.259+0000 7f68fbe00640 1 -- 172.21.15.145:0/1747385889 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ec032810 con 0x7f68fc0b6290 2024-05-09T04:33:59.260 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.259+0000 7f9132600640 1 -- 172.21.15.145:0/1633140354 shutdown_connections 2024-05-09T04:33:59.260 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.259+0000 7f9132600640 1 --2- 172.21.15.145:0/1633140354 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f912c06e0b0 0x7f912c06e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.260 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.259+0000 7f9132600640 1 --2- 172.21.15.145:0/1633140354 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f912c06c6c0 0x7f912c06caa0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.260 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.259+0000 7f9132600640 1 -- 172.21.15.145:0/1633140354 >> 172.21.15.145:0/1633140354 conn(0x7f912c0b2b40 msgr2=0x7f912c0b2f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:59.260 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.260+0000 7f6903e00640 1 -- 172.21.15.145:0/1747385889 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68fc0b6290 msgr2=0x7f68fc0d3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:59.260 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.260+0000 7f9132600640 1 -- 172.21.15.145:0/1633140354 shutdown_connections 2024-05-09T04:33:59.261 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.260+0000 7f383be00640 1 Processor -- start 2024-05-09T04:33:59.261 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.260+0000 7f6903e00640 1 --2- 172.21.15.145:0/1747385889 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68fc0b6290 0x7f68fc0d3ba0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f68ec009870 tx=0x7f68ec02a1f0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.261 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.261+0000 7f25f9e00640 1 Processor -- start 2024-05-09T04:33:59.261 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.260+0000 7f9132600640 1 -- 172.21.15.145:0/1633140354 wait complete. 2024-05-09T04:33:59.262 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.261+0000 7f25f9e00640 1 -- start start 2024-05-09T04:33:59.262 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.261+0000 7f383be00640 1 -- start start 2024-05-09T04:33:59.262 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.262+0000 7f383be00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f383415f5f0 0x7f38340d1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:59.262 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.262+0000 7f383be00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38340d2db0 0x7f38340d2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:59.263 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.262+0000 7f383be00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f38340b5930 0x7f38340b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:59.263 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.262+0000 7f383be00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f383415f9d0 con 0x7f383415f5f0 2024-05-09T04:33:59.263 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.262+0000 7f383be00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f38340b4b70 con 0x7f38340b5930 2024-05-09T04:33:59.263 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.262+0000 7f383be00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f38340b4e50 con 0x7f38340d2db0 2024-05-09T04:33:59.263 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.263+0000 7f9132600640 1 Processor -- start 2024-05-09T04:33:59.263 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.263+0000 7f3839400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f383415f5f0 0x7f38340d1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:59.264 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.263+0000 7f9132600640 1 -- start start 2024-05-09T04:33:59.264 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.263+0000 7f3838a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38340d2db0 0x7f38340d2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:59.264 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.263+0000 7f3838a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38340d2db0 0x7f38340d2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.145:47456/0 (socket says 172.21.15.145:47456) 2024-05-09T04:33:59.264 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.263+0000 7f3838a00640 1 -- 172.21.15.145:0/2503127004 learned_addr learned my addr 172.21.15.145:0/2503127004 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:59.265 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.264+0000 7f6903e00640 1 -- 172.21.15.145:0/1747385889 shutdown_connections 2024-05-09T04:33:59.265 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.264+0000 7f6903e00640 1 --2- 172.21.15.145:0/1747385889 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68fc0b6290 0x7f68fc0d3ba0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.265 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.264+0000 7f6903e00640 1 --2- 172.21.15.145:0/1747385889 >> v2:172.21.15.145:3300/0 conn(0x7f68fc15f5c0 0x7f68fc0d3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.265 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.264+0000 7f6903e00640 1 -- 172.21.15.145:0/1747385889 >> 172.21.15.145:0/1747385889 conn(0x7f68fc0d5080 msgr2=0x7f68fc0d5490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:59.265 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.264+0000 7f25f9e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f25ec0e7710 0x7f25ec04ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:59.265 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.264+0000 7f25f9e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f25ec0589c0 0x7f25ec053530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:59.265 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.264+0000 7f25f9e00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25ec057d90 con 0x7f25ec056ab0 2024-05-09T04:33:59.266 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.264+0000 7f25f9e00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f25ec006260 con 0x7f25ec0589c0 2024-05-09T04:33:59.266 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.264+0000 7f25f9e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f25ec006540 con 0x7f25ec0e7710 2024-05-09T04:33:59.266 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.264+0000 7f3839e00640 1 --2- 172.21.15.145:0/2503127004 >> v2:172.21.15.192:3300/0 conn(0x7f38340b5930 0x7f38340b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:59.266 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.264+0000 7f3839400640 1 -- 172.21.15.145:0/2503127004 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38340d2db0 msgr2=0x7f38340d2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:59.266 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.264+0000 7f3839400640 1 --2- 172.21.15.145:0/2503127004 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38340d2db0 0x7f38340d2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.266 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.264+0000 7f3839400640 1 -- 172.21.15.145:0/2503127004 >> v2:172.21.15.192:3300/0 conn(0x7f38340b5930 msgr2=0x7f38340b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:59.266 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.264+0000 7f3839400640 1 --2- 172.21.15.145:0/2503127004 >> v2:172.21.15.192:3300/0 conn(0x7f38340b5930 0x7f38340b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.266 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.264+0000 7f3839400640 1 -- 172.21.15.145:0/2503127004 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38340b5130 con 0x7f383415f5f0 2024-05-09T04:33:59.267 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.265+0000 7f25f8a00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f25ec0e7710 0x7f25ec04ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:59.267 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.265+0000 7f25f8a00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f25ec0e7710 0x7f25ec04ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.145:55348/0 (socket says 172.21.15.145:55348) 2024-05-09T04:33:59.267 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.265+0000 7f25f8a00640 1 -- 172.21.15.145:0/997043099 learned_addr learned my addr 172.21.15.145:0/997043099 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:59.267 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.265+0000 7f25f3e00640 1 --2- 172.21.15.145:0/997043099 >> v2:172.21.15.145:3300/0 conn(0x7f25ec0589c0 0x7f25ec053530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:59.267 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.265+0000 7f9132600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f912c06c6c0 0x7f912c14e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:59.268 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.265+0000 7f9132600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f912c06e0b0 0x7f912c09e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:59.268 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.265+0000 7f9132600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f912c0cd640 0x7f912c09e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:59.268 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.265+0000 7f9132600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f912c0d01e0 con 0x7f912c0cd640 2024-05-09T04:33:59.268 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.265+0000 7f9132600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f912c0cff00 con 0x7f912c06c6c0 2024-05-09T04:33:59.268 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.265+0000 7f9132600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f912c06cfe0 con 0x7f912c06e0b0 2024-05-09T04:33:59.268 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.266+0000 7f912be00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f912c0cd640 0x7f912c09e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:59.268 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.266+0000 7f912aa00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f912c06e0b0 0x7f912c09e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:59.268 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.266+0000 7f912be00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f912c0cd640 0x7f912c09e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:45174/0 (socket says 172.21.15.145:45174) 2024-05-09T04:33:59.268 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.266+0000 7f912be00640 1 -- 172.21.15.145:0/3524831578 learned_addr learned my addr 172.21.15.145:0/3524831578 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:59.268 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.265+0000 7f912b400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f912c06c6c0 0x7f912c14e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:59.268 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.266+0000 7f25f8a00640 1 -- 172.21.15.145:0/997043099 >> v2:172.21.15.145:3300/0 conn(0x7f25ec0589c0 msgr2=0x7f25ec053530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:59.269 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.266+0000 7f85c7400640 1 Processor -- start 2024-05-09T04:33:59.269 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.266+0000 7f25f8a00640 1 --2- 172.21.15.145:0/997043099 >> v2:172.21.15.145:3300/0 conn(0x7f25ec0589c0 0x7f25ec053530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.269 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.266+0000 7f85c7400640 1 -- start start 2024-05-09T04:33:59.269 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.266+0000 7f3839e00640 1 --2- 172.21.15.145:0/2503127004 >> v2:172.21.15.192:3300/0 conn(0x7f38340b5930 0x7f38340b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:33:59.270 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.266+0000 7f912aa00640 1 -- 172.21.15.145:0/3524831578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f912c06c6c0 msgr2=0x7f912c14e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:59.270 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.266+0000 7f912aa00640 1 --2- 172.21.15.145:0/3524831578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f912c06c6c0 0x7f912c14e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.270 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.266+0000 7f912aa00640 1 -- 172.21.15.145:0/3524831578 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f912c0cd640 msgr2=0x7f912c09e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:59.270 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.267+0000 7f912aa00640 1 --2- 172.21.15.145:0/3524831578 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f912c0cd640 0x7f912c09e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.270 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.267+0000 7f912aa00640 1 -- 172.21.15.145:0/3524831578 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f912c14f0a0 con 0x7f912c06e0b0 2024-05-09T04:33:59.270 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.267+0000 7f6903e00640 1 -- 172.21.15.145:0/1747385889 shutdown_connections 2024-05-09T04:33:59.271 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.267+0000 7f6903e00640 1 -- 172.21.15.145:0/1747385889 wait complete. 2024-05-09T04:33:59.271 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.267+0000 7f912be00640 1 --2- 172.21.15.145:0/3524831578 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f912c0cd640 0x7f912c09e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:33:59.271 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.267+0000 7f3839400640 1 --2- 172.21.15.145:0/2503127004 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f383415f5f0 0x7f38340d1bd0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f383409ec30 tx=0x7f382802a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c568176ad4159f43 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:59.271 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.266+0000 7f25f8a00640 1 -- 172.21.15.145:0/997043099 >> v1:172.21.15.145:6789/0 conn(0x7f25ec056ab0 legacy=0x7f25ec007020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:59.271 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.266+0000 7f25f8a00640 1 -- 172.21.15.145:0/997043099 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25ec006820 con 0x7f25ec0e7710 2024-05-09T04:33:59.272 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.268+0000 7f912aa00640 1 --2- 172.21.15.145:0/3524831578 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f912c06e0b0 0x7f912c09e060 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f911c00e880 tx=0x7f911c00ed40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:59.272 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.268+0000 7f25f8a00640 1 --2- 172.21.15.145:0/997043099 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25ec0e7710 0x7f25ec04ca30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f25ec00f7f0 tx=0x7f25e802a280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=595b546b109a8a2f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:59.272 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.268+0000 7f85c7400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f85c015f5f0 0x7f85c00d1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:59.272 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.268+0000 7f85c7400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f85c00d2db0 0x7f85c00d2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:59.272 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.268+0000 7f85c7400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85c00b5930 0x7f85c00b5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:59.272 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.268+0000 7f85c7400640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f85c015f9d0 con 0x7f85c00d2db0 2024-05-09T04:33:59.273 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.268+0000 7f85c7400640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f85c00b4b70 con 0x7f85c015f5f0 2024-05-09T04:33:59.273 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.268+0000 7f85c7400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f85c00b4e50 con 0x7f85c00b5930 2024-05-09T04:33:59.273 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.268+0000 7f3833e00640 1 -- 172.21.15.145:0/2503127004 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3828037070 con 0x7f383415f5f0 2024-05-09T04:33:59.273 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.268+0000 7f3833e00640 1 -- 172.21.15.145:0/2503127004 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3828032440 con 0x7f383415f5f0 2024-05-09T04:33:59.273 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.268+0000 7f3833e00640 1 -- 172.21.15.145:0/2503127004 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3828032850 con 0x7f383415f5f0 2024-05-09T04:33:59.274 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.268+0000 7f9123400640 1 -- 172.21.15.145:0/3524831578 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f911c017070 con 0x7f912c06e0b0 2024-05-09T04:33:59.274 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.268+0000 7f9123400640 1 -- 172.21.15.145:0/3524831578 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f911c00c940 con 0x7f912c06e0b0 2024-05-09T04:33:59.274 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.268+0000 7f9123400640 1 -- 172.21.15.145:0/3524831578 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f911c00cd10 con 0x7f912c06e0b0 2024-05-09T04:33:59.274 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.269+0000 7f85bfe00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f85c00d2db0 0x7f85c00d2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:59.274 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.269+0000 7f85c4a00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f85c015f5f0 0x7f85c00d1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:59.274 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.269+0000 7f85c5400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85c00b5930 0x7f85c00b5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:59.274 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.269+0000 7f85bfe00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f85c00d2db0 0x7f85c00d2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:45182/0 (socket says 172.21.15.145:45182) 2024-05-09T04:33:59.274 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.269+0000 7f85bfe00640 1 -- 172.21.15.145:0/2512527692 learned_addr learned my addr 172.21.15.145:0/2512527692 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:59.274 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.269+0000 7f85bfe00640 1 -- 172.21.15.145:0/2512527692 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85c00b5930 msgr2=0x7f85c00b5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:59.275 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.269+0000 7f85bfe00640 1 --2- 172.21.15.145:0/2512527692 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85c00b5930 0x7f85c00b5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.275 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.269+0000 7f85bfe00640 1 -- 172.21.15.145:0/2512527692 >> v2:172.21.15.192:3300/0 conn(0x7f85c015f5f0 msgr2=0x7f85c00d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:59.275 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.269+0000 7f85bfe00640 1 --2- 172.21.15.145:0/2512527692 >> v2:172.21.15.192:3300/0 conn(0x7f85c015f5f0 0x7f85c00d1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.275 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.269+0000 7f85bfe00640 1 -- 172.21.15.145:0/2512527692 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85c00b5130 con 0x7f85c00d2db0 2024-05-09T04:33:59.275 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.269+0000 7f383be00640 1 -- 172.21.15.145:0/2503127004 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f383415f5f0 msgr2=0x7f38340d1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:59.275 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.269+0000 7f383be00640 1 --2- 172.21.15.145:0/2503127004 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f383415f5f0 0x7f38340d1bd0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f383409ec30 tx=0x7f382802a130 comp rx=0 tx=0).stop 2024-05-09T04:33:59.275 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.269+0000 7f383be00640 1 -- 172.21.15.145:0/2503127004 shutdown_connections 2024-05-09T04:33:59.275 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.269+0000 7f383be00640 1 --2- 172.21.15.145:0/2503127004 >> v2:172.21.15.192:3300/0 conn(0x7f38340b5930 0x7f38340b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.275 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.269+0000 7f383be00640 1 --2- 172.21.15.145:0/2503127004 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38340d2db0 0x7f38340d2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.275 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.269+0000 7f383be00640 1 --2- 172.21.15.145:0/2503127004 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f383415f5f0 0x7f38340d1bd0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.276 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.269+0000 7f383be00640 1 -- 172.21.15.145:0/2503127004 >> 172.21.15.145:0/2503127004 conn(0x7f383409e520 msgr2=0x7f38340d51e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:59.276 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.269+0000 7f9132600640 1 -- 172.21.15.145:0/3524831578 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f912c14f380 con 0x7f912c06e0b0 2024-05-09T04:33:59.276 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.270+0000 7f9132600640 1 -- 172.21.15.145:0/3524831578 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f912c09ebc0 con 0x7f912c06e0b0 2024-05-09T04:33:59.276 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.270+0000 7f25f3400640 1 -- 172.21.15.145:0/997043099 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e8037070 con 0x7f25ec0e7710 2024-05-09T04:33:59.277 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.270+0000 7f383be00640 1 -- 172.21.15.145:0/2503127004 shutdown_connections 2024-05-09T04:33:59.277 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.270+0000 7f383be00640 1 -- 172.21.15.145:0/2503127004 wait complete. 2024-05-09T04:33:59.277 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.270+0000 7f25f3400640 1 -- 172.21.15.145:0/997043099 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25e8032440 con 0x7f25ec0e7710 2024-05-09T04:33:59.277 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.270+0000 7f25f3400640 1 -- 172.21.15.145:0/997043099 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e8032810 con 0x7f25ec0e7710 2024-05-09T04:33:59.277 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.271+0000 7f6903e00640 1 Processor -- start 2024-05-09T04:33:59.278 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.271+0000 7f25f9e00640 1 -- 172.21.15.145:0/997043099 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25ec0e7710 msgr2=0x7f25ec04ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:59.278 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.271+0000 7f25f9e00640 1 --2- 172.21.15.145:0/997043099 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25ec0e7710 0x7f25ec04ca30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f25ec00f7f0 tx=0x7f25e802a280 comp rx=0 tx=0).stop 2024-05-09T04:33:59.278 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.271+0000 7f85bfe00640 1 --2- 172.21.15.145:0/2512527692 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85c00d2db0 0x7f85c00d2110 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f85c00b7ce0 tx=0x7f85b002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a41f420cefd8744c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:59.278 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.271+0000 7f85bf400640 1 -- 172.21.15.145:0/2512527692 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b0037070 con 0x7f85c00d2db0 2024-05-09T04:33:59.278 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.271+0000 7f85bf400640 1 -- 172.21.15.145:0/2512527692 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85b0032440 con 0x7f85c00d2db0 2024-05-09T04:33:59.279 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.271+0000 7f6903e00640 1 -- start start 2024-05-09T04:33:59.279 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.271+0000 7f85bf400640 1 -- 172.21.15.145:0/2512527692 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b0032810 con 0x7f85c00d2db0 2024-05-09T04:33:59.279 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.272+0000 7f9f84c00640 1 Processor -- start 2024-05-09T04:33:59.280 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.272+0000 7f9f84c00640 1 -- start start 2024-05-09T04:33:59.280 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.273+0000 7f6903e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f68fc0b4470 0x7f68fc0b9c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:59.280 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.273+0000 7f6903e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68fc0b6290 0x7f68fc1f8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:59.280 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.273+0000 7f6903e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68fc15f5c0 0x7f68fc155890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:59.280 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.273+0000 7f6903e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f68fc0d2150 con 0x7f68fc0b4470 2024-05-09T04:33:59.280 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.273+0000 7f6903e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f68fc0d2e60 con 0x7f68fc15f5c0 2024-05-09T04:33:59.280 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.273+0000 7f6903e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f68fc0d22d0 con 0x7f68fc0b6290 2024-05-09T04:33:59.280 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.273+0000 7f25f9e00640 1 -- 172.21.15.145:0/997043099 shutdown_connections 2024-05-09T04:33:59.281 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.273+0000 7f85c7400640 1 -- 172.21.15.145:0/2512527692 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85c00d2db0 msgr2=0x7f85c00d2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:59.281 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.273+0000 7f383be00640 1 Processor -- start 2024-05-09T04:33:59.281 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.273+0000 7f85c7400640 1 --2- 172.21.15.145:0/2512527692 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85c00d2db0 0x7f85c00d2110 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f85c00b7ce0 tx=0x7f85b002a1e0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.282 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.273+0000 7fcd12800640 1 Processor -- start 2024-05-09T04:33:59.282 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.273+0000 7f25f9e00640 1 --2- 172.21.15.145:0/997043099 >> v2:172.21.15.145:3300/0 conn(0x7f25ec0589c0 0x7f25ec053530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.282 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.273+0000 7f25f9e00640 1 --2- 172.21.15.145:0/997043099 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25ec0e7710 0x7f25ec04ca30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.282 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.273+0000 7f25f9e00640 1 -- 172.21.15.145:0/997043099 >> 172.21.15.145:0/997043099 conn(0x7f25ec058ee0 msgr2=0x7f25ec0592c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:59.282 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.274+0000 7f9f84c00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f9f801629a0 0x7f9f8006d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:59.283 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.274+0000 7f9f84c00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f9f800b5540 0x7f9f8006dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:59.283 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.274+0000 7f6901e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68fc15f5c0 0x7f68fc155890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:59.283 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.274+0000 7f85c7400640 1 -- 172.21.15.145:0/2512527692 shutdown_connections 2024-05-09T04:33:59.283 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.274+0000 7f85c7400640 1 --2- 172.21.15.145:0/2512527692 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85c00b5930 0x7f85c00b5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.283 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.274+0000 7f85c7400640 1 --2- 172.21.15.145:0/2512527692 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85c00d2db0 0x7f85c00d2110 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.283 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.274+0000 7f85c7400640 1 --2- 172.21.15.145:0/2512527692 >> v2:172.21.15.192:3300/0 conn(0x7f85c015f5f0 0x7f85c00d1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.283 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.274+0000 7f85c7400640 1 -- 172.21.15.145:0/2512527692 >> 172.21.15.145:0/2512527692 conn(0x7f85c009e520 msgr2=0x7f85c00d51e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:59.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.274+0000 7f6901e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68fc15f5c0 0x7f68fc155890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.145:55372/0 (socket says 172.21.15.145:55372) 2024-05-09T04:33:59.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.274+0000 7f6901e00640 1 -- 172.21.15.145:0/2672212600 learned_addr learned my addr 172.21.15.145:0/2672212600 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:59.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.274+0000 7f9f84c00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f8006fd40 con 0x7f9f8006e220 2024-05-09T04:33:59.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.274+0000 7f9f84c00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f9f800cd580 con 0x7f9f800b5540 2024-05-09T04:33:59.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.274+0000 7f9f84c00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f9f800cd860 con 0x7f9f801629a0 2024-05-09T04:33:59.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.274+0000 7f9f7de00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f9f801629a0 0x7f9f8006d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:59.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.274+0000 7f383be00640 1 -- start start 2024-05-09T04:33:59.285 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.274+0000 7f9f7de00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f9f801629a0 0x7f9f8006d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.145:55384/0 (socket says 172.21.15.145:55384) 2024-05-09T04:33:59.285 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.274+0000 7f9f7de00640 1 -- 172.21.15.145:0/3596086964 learned_addr learned my addr 172.21.15.145:0/3596086964 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:59.285 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.274+0000 7f6901400640 1 --2- 172.21.15.145:0/2672212600 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f68fc0b4470 0x7f68fc0b9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:59.285 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.274+0000 7f6900a00640 1 --2- 172.21.15.145:0/2672212600 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68fc0b6290 0x7f68fc1f8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:59.286 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.274+0000 7f25f9e00640 1 -- 172.21.15.145:0/997043099 shutdown_connections 2024-05-09T04:33:59.286 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.274+0000 7f85c7400640 1 -- 172.21.15.145:0/2512527692 shutdown_connections 2024-05-09T04:33:59.286 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.274+0000 7f9f7d400640 1 --2- 172.21.15.145:0/3596086964 >> v2:172.21.15.145:3300/0 conn(0x7f9f800b5540 0x7f9f8006dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:59.286 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.275+0000 7f85c7400640 1 -- 172.21.15.145:0/2512527692 wait complete. 2024-05-09T04:33:59.287 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.275+0000 7f9f7d400640 1 -- 172.21.15.145:0/3596086964 >> v2:172.21.15.192:3300/0 conn(0x7f9f801629a0 msgr2=0x7f9f8006d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:59.287 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.275+0000 7f9f7d400640 1 --2- 172.21.15.145:0/3596086964 >> v2:172.21.15.192:3300/0 conn(0x7f9f801629a0 0x7f9f8006d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.287 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.275+0000 7f9f7d400640 1 -- 172.21.15.145:0/3596086964 >> v1:172.21.15.145:6789/0 conn(0x7f9f8006e220 legacy=0x7f9f8009ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:59.287 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.275+0000 7f383be00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38340b5930 0x7f3834161dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:59.288 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.276+0000 7f5940200640 1 Processor -- start 2024-05-09T04:33:59.288 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.275+0000 7f383be00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38340d2db0 0x7f38340b87e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:59.288 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.276+0000 7f383be00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f383415f5f0 0x7f3834159290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:59.288 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.276+0000 7f383be00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f38340d3c90 con 0x7f38340b5930 2024-05-09T04:33:59.289 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.276+0000 7f6901e00640 1 -- 172.21.15.145:0/2672212600 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68fc0b6290 msgr2=0x7f68fc1f8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:59.289 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.276+0000 7f6901e00640 1 --2- 172.21.15.145:0/2672212600 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68fc0b6290 0x7f68fc1f8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.289 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.276+0000 7f6901e00640 1 -- 172.21.15.145:0/2672212600 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f68fc0b4470 msgr2=0x7f68fc0b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:59.289 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.276+0000 7f383be00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f38340d4040 con 0x7f383415f5f0 2024-05-09T04:33:59.289 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.276+0000 7f383be00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f38340cc350 con 0x7f38340d2db0 2024-05-09T04:33:59.290 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.276+0000 7f6901e00640 1 --2- 172.21.15.145:0/2672212600 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f68fc0b4470 0x7f68fc0b9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.290 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.276+0000 7f6901e00640 1 -- 172.21.15.145:0/2672212600 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68fc0ba2b0 con 0x7f68fc15f5c0 2024-05-09T04:33:59.290 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.275+0000 7f25f9e00640 1 -- 172.21.15.145:0/997043099 wait complete. 2024-05-09T04:33:59.290 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.276+0000 7f6901400640 1 --2- 172.21.15.145:0/2672212600 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f68fc0b4470 0x7f68fc0b9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:33:59.291 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.276+0000 7f3839400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38340b5930 0x7f3834161dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:59.291 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.277+0000 7f3838a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38340d2db0 0x7f38340b87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:59.291 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.277+0000 7f3838a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38340d2db0 0x7f38340b87e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.145:47486/0 (socket says 172.21.15.145:47486) 2024-05-09T04:33:59.291 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.277+0000 7f3838a00640 1 -- 172.21.15.145:0/612557606 learned_addr learned my addr 172.21.15.145:0/612557606 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:59.291 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.275+0000 7f9f7d400640 1 -- 172.21.15.145:0/3596086964 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f800d1db0 con 0x7f9f800b5540 2024-05-09T04:33:59.292 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.277+0000 7f6901e00640 1 --2- 172.21.15.145:0/2672212600 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68fc15f5c0 0x7f68fc155890 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f68ec009870 tx=0x7f68ec0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:59.292 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.273+0000 7fcd12800640 1 -- start start 2024-05-09T04:33:59.292 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.275+0000 7fcd12800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcd040e30c0 0x7fcd04057d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:59.292 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.277+0000 7fcd11400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcd040e30c0 0x7fcd04057d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:59.292 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.277+0000 7fcd11400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcd040e30c0 0x7fcd04057d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.145:47480/0 (socket says 172.21.15.145:47480) 2024-05-09T04:33:59.292 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.277+0000 7fcd11400640 1 -- 172.21.15.145:0/2536147687 learned_addr learned my addr 172.21.15.145:0/2536147687 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:59.293 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.277+0000 7f3839e00640 1 --2- 172.21.15.145:0/612557606 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f383415f5f0 0x7f3834159290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:59.293 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.277+0000 7f3839400640 1 -- 172.21.15.145:0/612557606 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38340d2db0 msgr2=0x7f38340b87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:59.293 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.277+0000 7f3839400640 1 --2- 172.21.15.145:0/612557606 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38340d2db0 0x7f38340b87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.293 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.277+0000 7f9f7d400640 1 --2- 172.21.15.145:0/3596086964 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f800b5540 0x7f9f8006dce0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9f80074070 tx=0x7f9f6c02a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91ea4ea093f93d1c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:59.293 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.277+0000 7f3839400640 1 -- 172.21.15.145:0/612557606 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f383415f5f0 msgr2=0x7f3834159290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:59.294 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.277+0000 7f3839400640 1 --2- 172.21.15.145:0/612557606 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f383415f5f0 0x7f3834159290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.294 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.277+0000 7f3839400640 1 -- 172.21.15.145:0/612557606 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38340ba9f0 con 0x7f38340b5930 2024-05-09T04:33:59.294 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.277+0000 7fcd12800640 1 --2- 172.21.15.145:0/2536147687 >> v2:172.21.15.145:3300/0 conn(0x7fcd0403e840 0x7fcd04053bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:59.294 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.277+0000 7f25f9e00640 1 Processor -- start 2024-05-09T04:33:59.294 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.278+0000 7f68f9600640 1 -- 172.21.15.145:0/2672212600 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ec037040 con 0x7f68fc15f5c0 2024-05-09T04:33:59.295 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.278+0000 7f9f7ca00640 1 -- 172.21.15.145:0/3596086964 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c037070 con 0x7f9f800b5540 2024-05-09T04:33:59.295 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.278+0000 7f68f9600640 1 -- 172.21.15.145:0/2672212600 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68ec004230 con 0x7f68fc15f5c0 2024-05-09T04:33:59.295 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.278+0000 7f68f9600640 1 -- 172.21.15.145:0/2672212600 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ec003610 con 0x7f68fc15f5c0 2024-05-09T04:33:59.296 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.278+0000 7fcd12800640 1 -- 172.21.15.145:0/2536147687 --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd040582d0 con 0x7fcd04054130 2024-05-09T04:33:59.296 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.278+0000 7fcd12800640 1 -- 172.21.15.145:0/2536147687 --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fcd04058570 con 0x7fcd0403e840 2024-05-09T04:33:59.296 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.276+0000 7f5940200640 1 -- start start 2024-05-09T04:33:59.296 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.278+0000 7fcd12800640 1 -- 172.21.15.145:0/2536147687 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fcd04058850 con 0x7fcd040e30c0 2024-05-09T04:33:59.297 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.278+0000 7f5940200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f593814cdc0 0x7f5938164f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:59.297 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.278+0000 7f5940200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f5938164af0 0x7f59380b47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:59.297 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.278+0000 7f5940200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f593806edb0 con 0x7f593814d1a0 2024-05-09T04:33:59.297 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.278+0000 7f5940200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f593806f020 con 0x7f5938164af0 2024-05-09T04:33:59.297 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.278+0000 7f5940200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f593806f300 con 0x7f593814cdc0 2024-05-09T04:33:59.297 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.278+0000 7f3839400640 1 --2- 172.21.15.145:0/612557606 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38340b5930 0x7f3834161dd0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3828031000 tx=0x7f38280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:59.298 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.273+0000 7f9132600640 1 -- 172.21.15.145:0/3524831578 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f912c06fa70 con 0x7f912c06e0b0 2024-05-09T04:33:59.298 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.275+0000 7f9123400640 1 -- 172.21.15.145:0/3524831578 <== mon.2 v2:172.21.15.192:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f911c01a070 con 0x7f912c06e0b0 2024-05-09T04:33:59.298 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.277+0000 7f9123400640 1 --2- 172.21.15.145:0/3524831578 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f910c0762d0 0x7f910c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:59.298 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.277+0000 7f9123400640 1 -- 172.21.15.145:0/3524831578 <== mon.2 v2:172.21.15.192:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f911c0978a0 con 0x7f912c06e0b0 2024-05-09T04:33:59.298 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.278+0000 7f912b400640 1 --2- 172.21.15.145:0/3524831578 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f910c0762d0 0x7f910c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:59.298 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.278+0000 7f6903e00640 1 -- 172.21.15.145:0/2672212600 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68fc150c00 con 0x7f68fc15f5c0 2024-05-09T04:33:59.298 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.279+0000 7f9f7ca00640 1 -- 172.21.15.145:0/3596086964 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f6c02ade0 con 0x7f9f800b5540 2024-05-09T04:33:59.298 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.279+0000 7f9f7ca00640 1 -- 172.21.15.145:0/3596086964 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c037070 con 0x7f9f800b5540 2024-05-09T04:33:59.299 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.278+0000 7fcd10a00640 1 --2- 172.21.15.145:0/2536147687 >> v2:172.21.15.145:3300/0 conn(0x7fcd0403e840 0x7fcd04053bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:59.299 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.279+0000 7f593d800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f593814cdc0 0x7f5938164f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:59.299 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.279+0000 7f593d800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f593814cdc0 0x7f5938164f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.145:47496/0 (socket says 172.21.15.145:47496) 2024-05-09T04:33:59.299 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.279+0000 7f593ce00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f5938164af0 0x7f59380b47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:59.299 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.279+0000 7f593d800640 1 -- 172.21.15.145:0/3682848119 learned_addr learned my addr 172.21.15.145:0/3682848119 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:59.299 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.279+0000 7f912b400640 1 --2- 172.21.15.145:0/3524831578 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f910c0762d0 0x7f910c078790 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9124034000 tx=0x7f912402f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:59.300 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.279+0000 7f3831600640 1 -- 172.21.15.145:0/612557606 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3828037050 con 0x7f38340b5930 2024-05-09T04:33:59.300 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.280+0000 7f5937e00640 1 -- 172.21.15.145:0/3682848119 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 841429326 0 0) 0x7f593806edb0 con 0x7f593814d1a0 2024-05-09T04:33:59.300 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.278+0000 7f25f9e00640 1 -- start start 2024-05-09T04:33:59.300 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.280+0000 7f25f9e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25ec056ab0 0x7f25ec0526f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:59.300 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.280+0000 7f25f9e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f25ec0589c0 0x7f25ec051780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:59.300 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.280+0000 7f25f9e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f25ec0e7710 0x7f25ec0e7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:59.301 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.280+0000 7f25f9e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f25ec0579a0 con 0x7f25ec0e7710 2024-05-09T04:33:59.301 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.280+0000 7f25f9e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f25ec04cf70 con 0x7f25ec056ab0 2024-05-09T04:33:59.301 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.280+0000 7f25f9e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f25ec057b20 con 0x7f25ec0589c0 2024-05-09T04:33:59.301 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.280+0000 7f383be00640 1 -- 172.21.15.145:0/612557606 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38340cc5f0 con 0x7f38340b5930 2024-05-09T04:33:59.301 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.280+0000 7f85c7400640 1 Processor -- start 2024-05-09T04:33:59.301 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.280+0000 7f6903e00640 1 -- 172.21.15.145:0/2672212600 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68fc1511c0 con 0x7f68fc15f5c0 2024-05-09T04:33:59.302 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.280+0000 7f5937e00640 1 -- 172.21.15.145:0/3682848119 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5920003520 con 0x7f593814d1a0 2024-05-09T04:33:59.302 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.280+0000 7f85c7400640 1 -- start start 2024-05-09T04:33:59.302 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.281+0000 7f9f84c00640 1 -- 172.21.15.145:0/3596086964 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f800b5540 msgr2=0x7f9f8006dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:59.302 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.281+0000 7f9f84c00640 1 --2- 172.21.15.145:0/3596086964 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f800b5540 0x7f9f8006dce0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9f80074070 tx=0x7f9f6c02a0b0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.303 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.280+0000 7f383be00640 1 -- 172.21.15.145:0/612557606 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38340bacd0 con 0x7f38340b5930 2024-05-09T04:33:59.303 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.281+0000 7f25f9400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f25ec0e7710 0x7f25ec0e7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:59.303 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.280+0000 7f25f8a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25ec056ab0 0x7f25ec0526f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:59.303 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.281+0000 7f25f9400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f25ec0e7710 0x7f25ec0e7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:45220/0 (socket says 172.21.15.145:45220) 2024-05-09T04:33:59.303 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.281+0000 7f25f9400640 1 -- 172.21.15.145:0/793631504 learned_addr learned my addr 172.21.15.145:0/793631504 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:59.304 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.281+0000 7f9f84c00640 1 -- 172.21.15.145:0/3596086964 shutdown_connections 2024-05-09T04:33:59.304 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.281+0000 7f9f84c00640 1 --2- 172.21.15.145:0/3596086964 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f800b5540 0x7f9f8006dce0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.304 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.281+0000 7f9f84c00640 1 --2- 172.21.15.145:0/3596086964 >> v2:172.21.15.192:3300/0 conn(0x7f9f801629a0 0x7f9f8006d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.304 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.281+0000 7f9f84c00640 1 -- 172.21.15.145:0/3596086964 >> 172.21.15.145:0/3596086964 conn(0x7f9f800b4a20 msgr2=0x7f9f800b4e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:59.304 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.280+0000 7fcd10a00640 1 -- 172.21.15.145:0/2536147687 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcd040e30c0 msgr2=0x7fcd04057d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:59.304 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.280+0000 7fcd10a00640 1 --2- 172.21.15.145:0/2536147687 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcd040e30c0 0x7fcd04057d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.304 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.280+0000 7fcd10a00640 1 -- 172.21.15.145:0/2536147687 >> v1:172.21.15.145:6789/0 conn(0x7fcd04054130 legacy=0x7fcd040e6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:59.304 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.280+0000 7fcd10a00640 1 -- 172.21.15.145:0/2536147687 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd04058b30 con 0x7fcd0403e840 2024-05-09T04:33:59.305 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.281+0000 7f593ce00640 1 -- 172.21.15.145:0/3682848119 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f593814cdc0 msgr2=0x7f5938164f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:33:59.305 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.281+0000 7f593ce00640 1 --2- 172.21.15.145:0/3682848119 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f593814cdc0 0x7f5938164f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.305 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.281+0000 7f593ce00640 1 -- 172.21.15.145:0/3682848119 >> v1:172.21.15.192:6789/0 conn(0x7f593814d1a0 legacy=0x7f59380cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:59.305 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.281+0000 7f593ce00640 1 -- 172.21.15.145:0/3682848119 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f593806f5e0 con 0x7f5938164af0 2024-05-09T04:33:59.305 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.281+0000 7f9f84c00640 1 -- 172.21.15.145:0/3596086964 shutdown_connections 2024-05-09T04:33:59.305 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.281+0000 7f9f84c00640 1 -- 172.21.15.145:0/3596086964 wait complete. 2024-05-09T04:33:59.306 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.282+0000 7f6903e00640 1 -- 172.21.15.145:0/2672212600 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68fc076780 con 0x7f68fc15f5c0 2024-05-09T04:33:59.306 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.282+0000 7fcd10a00640 1 --2- 172.21.15.145:0/2536147687 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcd0403e840 0x7fcd04053bf0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fccf80097a0 tx=0x7fccf802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f9d1f61ca7c0379 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:59.306 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.281+0000 7f593d800640 1 --2- 172.21.15.145:0/3682848119 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f593814cdc0 0x7f5938164f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:33:59.310 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.281+0000 7f3831600640 1 -- 172.21.15.145:0/612557606 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3828043070 con 0x7f38340b5930 2024-05-09T04:33:59.310 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.281+0000 7f3831600640 1 -- 172.21.15.145:0/612557606 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3828037050 con 0x7f38340b5930 2024-05-09T04:33:59.310 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.282+0000 7f25f3e00640 1 --2- 172.21.15.145:0/793631504 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f25ec0589c0 0x7f25ec051780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:59.310 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.282+0000 7f85c7400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85c00b5930 0x7f85c01536f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:59.310 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.282+0000 7f593ce00640 1 --2- 172.21.15.145:0/3682848119 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5938164af0 0x7f59380b47f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f592c0097a0 tx=0x7f592c02a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fb937f89330122d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:59.311 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.282+0000 7f85c7400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85c00d2db0 0x7f85c0154800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:59.311 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.282+0000 7f85c7400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85c015f5f0 0x7f85c00b8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:59.311 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.283+0000 7f5937e00640 1 -- 172.21.15.145:0/3682848119 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c037070 con 0x7f5938164af0 2024-05-09T04:33:59.311 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.283+0000 7f85c4a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85c00b5930 0x7f85c01536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:59.312 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.283+0000 7f5937e00640 1 -- 172.21.15.145:0/3682848119 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f592c032440 con 0x7f5938164af0 2024-05-09T04:33:59.312 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.283+0000 7f5937e00640 1 -- 172.21.15.145:0/3682848119 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c032850 con 0x7f5938164af0 2024-05-09T04:33:59.312 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.283+0000 7f25f9400640 1 -- 172.21.15.145:0/793631504 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f25ec0589c0 msgr2=0x7f25ec051780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:59.312 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.283+0000 7f25f9400640 1 --2- 172.21.15.145:0/793631504 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f25ec0589c0 0x7f25ec051780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.312 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.283+0000 7f25f9400640 1 -- 172.21.15.145:0/793631504 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25ec056ab0 msgr2=0x7f25ec0526f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:59.312 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.283+0000 7f25f9400640 1 --2- 172.21.15.145:0/793631504 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25ec056ab0 0x7f25ec0526f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.312 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.283+0000 7f25f9400640 1 -- 172.21.15.145:0/793631504 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25ec03e4f0 con 0x7f25ec0e7710 2024-05-09T04:33:59.313 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.283+0000 7f383be00640 1 -- 172.21.15.145:0/612557606 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3810005290 con 0x7f38340b5930 2024-05-09T04:33:59.313 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.283+0000 7f85c4a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85c00b5930 0x7f85c01536f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:45236/0 (socket says 172.21.15.145:45236) 2024-05-09T04:33:59.313 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.283+0000 7f85c7400640 1 -- 172.21.15.145:0/462099658 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f85c00d3c90 con 0x7f85c00b5930 2024-05-09T04:33:59.313 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.283+0000 7f85c4a00640 1 -- 172.21.15.145:0/462099658 learned_addr learned my addr 172.21.15.145:0/462099658 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:59.313 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.283+0000 7f85c7400640 1 -- 172.21.15.145:0/462099658 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f85c00d4040 con 0x7f85c015f5f0 2024-05-09T04:33:59.313 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.283+0000 7f85c7400640 1 -- 172.21.15.145:0/462099658 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f85c0153c30 con 0x7f85c00d2db0 2024-05-09T04:33:59.313 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.284+0000 7fcd0be00640 1 -- 172.21.15.145:0/2536147687 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf8037070 con 0x7fcd0403e840 2024-05-09T04:33:59.314 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.283+0000 7f85bfe00640 1 --2- 172.21.15.145:0/462099658 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85c00d2db0 0x7f85c0154800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:59.314 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.284+0000 7f85bfe00640 1 -- 172.21.15.145:0/462099658 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85c015f5f0 msgr2=0x7f85c00b8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:59.314 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.284+0000 7f85bfe00640 1 --2- 172.21.15.145:0/462099658 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85c015f5f0 0x7f85c00b8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.314 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.284+0000 7f85bfe00640 1 -- 172.21.15.145:0/462099658 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85c00b5930 msgr2=0x7f85c01536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:59.314 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.284+0000 7f85bfe00640 1 --2- 172.21.15.145:0/462099658 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85c00b5930 0x7f85c01536f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.314 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.284+0000 7f85bfe00640 1 -- 172.21.15.145:0/462099658 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85c0154d40 con 0x7f85c00d2db0 2024-05-09T04:33:59.314 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.284+0000 7f85c4a00640 1 --2- 172.21.15.145:0/462099658 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85c00b5930 0x7f85c01536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:33:59.315 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.284+0000 7f5940200640 1 -- 172.21.15.145:0/3682848119 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5938164af0 msgr2=0x7f59380b47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:59.315 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.284+0000 7f5940200640 1 --2- 172.21.15.145:0/3682848119 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5938164af0 0x7f59380b47f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f592c0097a0 tx=0x7f592c02a1a0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.315 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.285+0000 7f68f9600640 1 -- 172.21.15.145:0/2672212600 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f68ec0037b0 con 0x7f68fc15f5c0 2024-05-09T04:33:59.315 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.285+0000 7f85bfe00640 1 --2- 172.21.15.145:0/462099658 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85c00d2db0 0x7f85c0154800 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f85b0031000 tx=0x7f85b00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:59.315 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.285+0000 7f9f84c00640 1 Processor -- start 2024-05-09T04:33:59.316 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.285+0000 7f5940200640 1 -- 172.21.15.145:0/3682848119 shutdown_connections 2024-05-09T04:33:59.316 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.285+0000 7f5940200640 1 --2- 172.21.15.145:0/3682848119 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5938164af0 0x7f59380b47f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.316 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.285+0000 7f5940200640 1 --2- 172.21.15.145:0/3682848119 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f593814cdc0 0x7f5938164f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.316 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.285+0000 7f5940200640 1 -- 172.21.15.145:0/3682848119 >> 172.21.15.145:0/3682848119 conn(0x7f593806c8d0 msgr2=0x7f593806cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:59.316 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.285+0000 7f9f84c00640 1 -- start start 2024-05-09T04:33:59.317 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.285+0000 7f5940200640 1 -- 172.21.15.145:0/3682848119 shutdown_connections 2024-05-09T04:33:59.317 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.285+0000 7f25f9400640 1 --2- 172.21.15.145:0/793631504 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f25ec0e7710 0x7f25ec0e7b80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f25f4074fe0 tx=0x7f25f4069fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:59.317 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.285+0000 7f3831600640 1 -- 172.21.15.145:0/612557606 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f382803c8d0 con 0x7f38340b5930 2024-05-09T04:33:59.317 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.286+0000 7f3831600640 1 --2- 172.21.15.145:0/612557606 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f38140764d0 0x7f3814078990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:59.317 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.286+0000 7f68f9600640 1 --2- 172.21.15.145:0/2672212600 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f68dc0763a0 0x7f68dc078860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:59.318 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.286+0000 7f3831600640 1 -- 172.21.15.145:0/612557606 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f38280b8980 con 0x7f38340b5930 2024-05-09T04:33:59.318 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.287+0000 7f6901400640 1 --2- 172.21.15.145:0/2672212600 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f68dc0763a0 0x7f68dc078860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:59.318 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.287+0000 7f3838a00640 1 --2- 172.21.15.145:0/612557606 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f38140764d0 0x7f3814078990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:59.319 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.288+0000 7f68f9600640 1 -- 172.21.15.145:0/2672212600 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f68ec0b7fe0 con 0x7f68fc15f5c0 2024-05-09T04:33:59.319 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.286+0000 7f9f84c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f8006e220 0x7f9f801513b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:59.319 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.287+0000 7f9f84c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f800b5540 0x7f9f800ca000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:59.319 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.287+0000 7f9f84c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f801629a0 0x7f9f800cbb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:59.319 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.287+0000 7f9f7e800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f801629a0 0x7f9f800cbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:59.319 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.287+0000 7f9f7e800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f801629a0 0x7f9f800cbb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:45244/0 (socket says 172.21.15.145:45244) 2024-05-09T04:33:59.319 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.287+0000 7f9f7e800640 1 -- 172.21.15.145:0/1638358900 learned_addr learned my addr 172.21.15.145:0/1638358900 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:59.319 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.287+0000 7f9f84c00640 1 -- 172.21.15.145:0/1638358900 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9f80162d80 con 0x7f9f801629a0 2024-05-09T04:33:59.320 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.287+0000 7f9f84c00640 1 -- 172.21.15.145:0/1638358900 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9f800d2370 con 0x7f9f8006e220 2024-05-09T04:33:59.320 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.287+0000 7f9f84c00640 1 -- 172.21.15.145:0/1638358900 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9f800ced00 con 0x7f9f800b5540 2024-05-09T04:33:59.320 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.287+0000 7f9f7d400640 1 --2- 172.21.15.145:0/1638358900 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f800b5540 0x7f9f800ca000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:59.320 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.287+0000 7f9f7de00640 1 --2- 172.21.15.145:0/1638358900 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f8006e220 0x7f9f801513b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:59.320 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.288+0000 7f3838a00640 1 --2- 172.21.15.145:0/612557606 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f38140764d0 0x7f3814078990 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3834162b50 tx=0x7f3820008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:59.320 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.285+0000 7f5940200640 1 -- 172.21.15.145:0/3682848119 wait complete. 2024-05-09T04:33:59.320 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.288+0000 7f5940200640 1 Processor -- start 2024-05-09T04:33:59.321 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.288+0000 7f9f7e800640 1 -- 172.21.15.145:0/1638358900 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f800b5540 msgr2=0x7f9f800ca000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:59.321 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.288+0000 7f5940200640 1 -- start start 2024-05-09T04:33:59.321 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.288+0000 7f9f7e800640 1 --2- 172.21.15.145:0/1638358900 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f800b5540 0x7f9f800ca000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.322 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.288+0000 7f9f7e800640 1 -- 172.21.15.145:0/1638358900 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f8006e220 msgr2=0x7f9f801513b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:59.322 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.288+0000 7f9f7e800640 1 --2- 172.21.15.145:0/1638358900 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f8006e220 0x7f9f801513b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.322 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.288+0000 7f9f7e800640 1 -- 172.21.15.145:0/1638358900 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f801518f0 con 0x7f9f801629a0 2024-05-09T04:33:59.322 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.288+0000 7f9f7de00640 1 --2- 172.21.15.145:0/1638358900 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f8006e220 0x7f9f801513b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:33:59.322 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.288+0000 7f9f7d400640 1 --2- 172.21.15.145:0/1638358900 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f800b5540 0x7f9f800ca000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:33:59.322 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.288+0000 7f6901400640 1 --2- 172.21.15.145:0/2672212600 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f68dc0763a0 0x7f68dc078860 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f68fc073b30 tx=0x7f68f400b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:59.322 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.289+0000 7f9f7e800640 1 --2- 172.21.15.145:0/1638358900 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f801629a0 0x7f9f800cbb60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9f7001d400 tx=0x7f9f7001d8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:59.323 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.290+0000 7f5940200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f593814cdc0 0x7f593813a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:59.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.290+0000 7f5940200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f593814d1a0 0x7f5938160b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:59.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.290+0000 7f593d800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f593814cdc0 0x7f593813a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:59.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.290+0000 7f5940200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5938164af0 0x7f593813b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:59.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.290+0000 7f593d800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f593814cdc0 0x7f593813a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:45254/0 (socket says 172.21.15.145:45254) 2024-05-09T04:33:59.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.290+0000 7f593d800640 1 -- 172.21.15.145:0/468768286 learned_addr learned my addr 172.21.15.145:0/468768286 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:59.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.290+0000 7f5940200640 1 -- 172.21.15.145:0/468768286 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f59381655f0 con 0x7f593814cdc0 2024-05-09T04:33:59.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.290+0000 7f5940200640 1 -- 172.21.15.145:0/468768286 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5938165470 con 0x7f5938164af0 2024-05-09T04:33:59.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.290+0000 7f5940200640 1 -- 172.21.15.145:0/468768286 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f59380b4d30 con 0x7f593814d1a0 2024-05-09T04:33:59.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.290+0000 7f593ce00640 1 --2- 172.21.15.145:0/468768286 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f593814d1a0 0x7f5938160b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:59.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.291+0000 7f593e200640 1 --2- 172.21.15.145:0/468768286 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5938164af0 0x7f593813b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:59.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.291+0000 7f593d800640 1 -- 172.21.15.145:0/468768286 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f593814d1a0 msgr2=0x7f5938160b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:59.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.291+0000 7f593d800640 1 --2- 172.21.15.145:0/468768286 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f593814d1a0 0x7f5938160b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.291+0000 7f593d800640 1 -- 172.21.15.145:0/468768286 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5938164af0 msgr2=0x7f593813b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:59.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.291+0000 7f593d800640 1 --2- 172.21.15.145:0/468768286 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5938164af0 0x7f593813b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.291+0000 7f593d800640 1 -- 172.21.15.145:0/468768286 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5938074300 con 0x7f593814cdc0 2024-05-09T04:33:59.325 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.291+0000 7f9f76000640 1 -- 172.21.15.145:0/1638358900 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f70028070 con 0x7f9f801629a0 2024-05-09T04:33:59.325 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.291+0000 7f9f76000640 1 -- 172.21.15.145:0/1638358900 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f700050d0 con 0x7f9f801629a0 2024-05-09T04:33:59.325 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.291+0000 7f9f84c00640 1 -- 172.21.15.145:0/1638358900 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f800cca70 con 0x7f9f801629a0 2024-05-09T04:33:59.325 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.291+0000 7f9f76000640 1 -- 172.21.15.145:0/1638358900 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f700035f0 con 0x7f9f801629a0 2024-05-09T04:33:59.325 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.291+0000 7f9f84c00640 1 -- 172.21.15.145:0/1638358900 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f800c6f60 con 0x7f9f801629a0 2024-05-09T04:33:59.325 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.291+0000 7fcd0be00640 1 -- 172.21.15.145:0/2536147687 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccf8032440 con 0x7fcd0403e840 2024-05-09T04:33:59.326 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.291+0000 7fcd0be00640 1 -- 172.21.15.145:0/2536147687 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccf8032850 con 0x7fcd0403e840 2024-05-09T04:33:59.326 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.291+0000 7f593d800640 1 --2- 172.21.15.145:0/468768286 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f593814cdc0 0x7f593813a470 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f593000efc0 tx=0x7f593000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:59.326 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.292+0000 7f25f0c00640 1 -- 172.21.15.145:0/793631504 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f406ddf0 con 0x7f25ec0e7710 2024-05-09T04:33:59.327 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.292+0000 7f25f0c00640 1 -- 172.21.15.145:0/793631504 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25f40741c0 con 0x7f25ec0e7710 2024-05-09T04:33:59.327 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.292+0000 7f25f0c00640 1 -- 172.21.15.145:0/793631504 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f4074670 con 0x7f25ec0e7710 2024-05-09T04:33:59.327 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.292+0000 7f25f9e00640 1 -- 172.21.15.145:0/793631504 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25ec052cd0 con 0x7f25ec0e7710 2024-05-09T04:33:59.327 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.292+0000 7f25f9e00640 1 -- 172.21.15.145:0/793631504 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25ec050f80 con 0x7f25ec0e7710 2024-05-09T04:33:59.327 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.292+0000 7fcd12800640 1 -- 172.21.15.145:0/2536147687 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcd0403e840 msgr2=0x7fcd04053bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:59.327 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.292+0000 7fcd12800640 1 --2- 172.21.15.145:0/2536147687 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcd0403e840 0x7fcd04053bf0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fccf80097a0 tx=0x7fccf802a250 comp rx=0 tx=0).stop 2024-05-09T04:33:59.327 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.292+0000 7f3831600640 1 -- 172.21.15.145:0/612557606 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f38280823e0 con 0x7f38340b5930 2024-05-09T04:33:59.328 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.292+0000 7fcd12800640 1 -- 172.21.15.145:0/2536147687 shutdown_connections 2024-05-09T04:33:59.328 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.292+0000 7fcd12800640 1 --2- 172.21.15.145:0/2536147687 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcd0403e840 0x7fcd04053bf0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.328 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.292+0000 7fcd12800640 1 --2- 172.21.15.145:0/2536147687 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcd040e30c0 0x7fcd04057d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.328 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.292+0000 7fcd12800640 1 -- 172.21.15.145:0/2536147687 >> 172.21.15.145:0/2536147687 conn(0x7fcd040559e0 msgr2=0x7fcd04055df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:59.328 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.293+0000 7fcd12800640 1 -- 172.21.15.145:0/2536147687 shutdown_connections 2024-05-09T04:33:59.328 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.293+0000 7fcd12800640 1 -- 172.21.15.145:0/2536147687 wait complete. 2024-05-09T04:33:59.329 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.293+0000 7f25f9e00640 1 -- 172.21.15.145:0/793631504 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25ec005390 con 0x7f25ec0e7710 2024-05-09T04:33:59.329 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.294+0000 7f5935600640 1 -- 172.21.15.145:0/468768286 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5930014480 con 0x7f593814cdc0 2024-05-09T04:33:59.329 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.294+0000 7f5935600640 1 -- 172.21.15.145:0/468768286 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5930019070 con 0x7f593814cdc0 2024-05-09T04:33:59.329 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.294+0000 7f5935600640 1 -- 172.21.15.145:0/468768286 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5930014820 con 0x7f593814cdc0 2024-05-09T04:33:59.329 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.294+0000 7f9f84c00640 1 -- 172.21.15.145:0/1638358900 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f800694e0 con 0x7f9f801629a0 2024-05-09T04:33:59.330 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.294+0000 7f5940200640 1 -- 172.21.15.145:0/468768286 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f593813ab50 con 0x7f593814cdc0 2024-05-09T04:33:59.330 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.294+0000 7f5940200640 1 -- 172.21.15.145:0/468768286 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5938072850 con 0x7f593814cdc0 2024-05-09T04:33:59.330 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.295+0000 7fcd12800640 1 Processor -- start 2024-05-09T04:33:59.330 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.295+0000 7f9f76000640 1 -- 172.21.15.145:0/1638358900 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9f7002b070 con 0x7f9f801629a0 2024-05-09T04:33:59.331 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.296+0000 7f5940200640 1 -- 172.21.15.145:0/468768286 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f593806a310 con 0x7f593814cdc0 2024-05-09T04:33:59.331 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.296+0000 7fcd12800640 1 -- start start 2024-05-09T04:33:59.331 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.296+0000 7f25f0c00640 1 -- 172.21.15.145:0/793631504 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f25f4074810 con 0x7f25ec0e7710 2024-05-09T04:33:59.332 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.297+0000 7fcd12800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcd0403e840 0x7fcd040e7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:59.332 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.297+0000 7fcd12800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcd04054130 0x7fcd040509f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:59.332 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.297+0000 7fcd12800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd040e30c0 0x7fcd040d7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:59.332 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.297+0000 7fcd12800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fcd040e34a0 con 0x7fcd0403e840 2024-05-09T04:33:59.332 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.297+0000 7fcd12800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0403bfc0 con 0x7fcd040e30c0 2024-05-09T04:33:59.332 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.297+0000 7fcd12800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fcd040078e0 con 0x7fcd04054130 2024-05-09T04:33:59.332 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.297+0000 7fcd11400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcd0403e840 0x7fcd040e7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:59.332 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.297+0000 7f25f0c00640 1 --2- 172.21.15.145:0/793631504 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f25d4076350 0x7f25d4078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:59.332 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.297+0000 7f25f0c00640 1 -- 172.21.15.145:0/793631504 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f25f4113280 con 0x7f25ec0e7710 2024-05-09T04:33:59.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.297+0000 7fcd11400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcd0403e840 0x7fcd040e7380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:45258/0 (socket says 172.21.15.145:45258) 2024-05-09T04:33:59.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.297+0000 7fcd11400640 1 -- 172.21.15.145:0/3990378403 learned_addr learned my addr 172.21.15.145:0/3990378403 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:33:59.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.298+0000 7fcd10a00640 1 --2- 172.21.15.145:0/3990378403 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcd04054130 0x7fcd040509f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:59.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.298+0000 7fcd11e00640 1 --2- 172.21.15.145:0/3990378403 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd040e30c0 0x7fcd040d7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:59.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.298+0000 7fcd11400640 1 -- 172.21.15.145:0/3990378403 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcd04054130 msgr2=0x7fcd040509f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:59.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.298+0000 7f25f8a00640 1 --2- 172.21.15.145:0/793631504 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f25d4076350 0x7f25d4078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:59.334 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.298+0000 7fcd11400640 1 --2- 172.21.15.145:0/3990378403 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcd04054130 0x7fcd040509f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.334 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.298+0000 7fcd11400640 1 -- 172.21.15.145:0/3990378403 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd040e30c0 msgr2=0x7fcd040d7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:59.334 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.298+0000 7fcd11400640 1 --2- 172.21.15.145:0/3990378403 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd040e30c0 0x7fcd040d7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.334 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.298+0000 7fcd11400640 1 -- 172.21.15.145:0/3990378403 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd0404ffa0 con 0x7fcd0403e840 2024-05-09T04:33:59.334 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.298+0000 7f9f76000640 1 --2- 172.21.15.145:0/1638358900 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f9f600763d0 0x7f9f60078890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:59.334 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.299+0000 7f9f76000640 1 -- 172.21.15.145:0/1638358900 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9f700a7750 con 0x7f9f801629a0 2024-05-09T04:33:59.335 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.299+0000 7f9f7de00640 1 --2- 172.21.15.145:0/1638358900 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f9f600763d0 0x7f9f60078890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:59.335 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.299+0000 7fcd11400640 1 --2- 172.21.15.145:0/3990378403 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcd0403e840 0x7fcd040e7380 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcd0000e7e0 tx=0x7fcd0000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:59.335 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.300+0000 7f9f7de00640 1 --2- 172.21.15.145:0/1638358900 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f9f600763d0 0x7f9f60078890 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9f78009c70 tx=0x7f9f78009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:59.335 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.300+0000 7f25f8a00640 1 --2- 172.21.15.145:0/793631504 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f25d4076350 0x7f25d4078810 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f25e8034000 tx=0x7f25e802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:59.336 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.302+0000 7fcd09600640 1 -- 172.21.15.145:0/3990378403 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd00017070 con 0x7fcd0403e840 2024-05-09T04:33:59.336 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.302+0000 7f25f0c00640 1 -- 172.21.15.145:0/793631504 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f25f409b030 con 0x7f25ec0e7710 2024-05-09T04:33:59.336 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.302+0000 7fcd12800640 1 -- 172.21.15.145:0/3990378403 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd040e79b0 con 0x7fcd0403e840 2024-05-09T04:33:59.336 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.302+0000 7fcd12800640 1 -- 172.21.15.145:0/3990378403 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd04050520 con 0x7fcd0403e840 2024-05-09T04:33:59.336 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.303+0000 7fcd12800640 1 -- 172.21.15.145:0/3990378403 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd040057e0 con 0x7fcd0403e840 2024-05-09T04:33:59.337 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.303+0000 7f5935600640 1 -- 172.21.15.145:0/468768286 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5930005d10 con 0x7f593814cdc0 2024-05-09T04:33:59.337 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.305+0000 7f5935600640 1 --2- 172.21.15.145:0/468768286 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f591c076320 0x7f591c0787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:59.337 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.305+0000 7f593ce00640 1 --2- 172.21.15.145:0/468768286 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f591c076320 0x7f591c0787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:59.337 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.306+0000 7f593ce00640 1 --2- 172.21.15.145:0/468768286 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f591c076320 0x7f591c0787e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f592c034000 tx=0x7f592c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:59.338 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.307+0000 7fcd09600640 1 -- 172.21.15.145:0/3990378403 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd00005990 con 0x7fcd0403e840 2024-05-09T04:33:59.338 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.307+0000 7fcd09600640 1 -- 172.21.15.145:0/3990378403 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd00021060 con 0x7fcd0403e840 2024-05-09T04:33:59.338 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.307+0000 7f5935600640 1 -- 172.21.15.145:0/468768286 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f593005db30 con 0x7f593814cdc0 2024-05-09T04:33:59.338 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.307+0000 7f9f76000640 1 -- 172.21.15.145:0/1638358900 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f9f70060990 con 0x7f9f801629a0 2024-05-09T04:33:59.338 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.308+0000 7f9123400640 1 -- 172.21.15.145:0/3524831578 <== mon.2 v2:172.21.15.192:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f912c06fa70 con 0x7f912c06e0b0 2024-05-09T04:33:59.339 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.308+0000 7f85bcc00640 1 -- 172.21.15.145:0/462099658 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b0041020 con 0x7f85c00d2db0 2024-05-09T04:33:59.339 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.308+0000 7f85bcc00640 1 -- 172.21.15.145:0/462099658 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85b003d6e0 con 0x7f85c00d2db0 2024-05-09T04:33:59.339 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.308+0000 7f85bcc00640 1 -- 172.21.15.145:0/462099658 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b0037040 con 0x7f85c00d2db0 2024-05-09T04:33:59.339 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.308+0000 7f85c7400640 1 -- 172.21.15.145:0/462099658 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85c00cf340 con 0x7f85c00d2db0 2024-05-09T04:33:59.340 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.312+0000 7f85c7400640 1 -- 172.21.15.145:0/462099658 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f85c00cc7e0 con 0x7f85c00d2db0 2024-05-09T04:33:59.340 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.313+0000 7f85c7400640 1 -- 172.21.15.145:0/462099658 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85c006a310 con 0x7f85c00d2db0 2024-05-09T04:33:59.340 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.315+0000 7f85bcc00640 1 -- 172.21.15.145:0/462099658 <== mon.2 v2:172.21.15.192:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f85b0044020 con 0x7f85c00d2db0 2024-05-09T04:33:59.340 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.316+0000 7f68f9600640 1 -- 172.21.15.145:0/2672212600 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f68ec081360 con 0x7f68fc15f5c0 2024-05-09T04:33:59.340 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.316+0000 7f85bcc00640 1 --2- 172.21.15.145:0/462099658 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f85a0076320 0x7f85a00787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:59.340 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.317+0000 7f85bcc00640 1 -- 172.21.15.145:0/462099658 <== mon.2 v2:172.21.15.192:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f85b00b7dd0 con 0x7f85c00d2db0 2024-05-09T04:33:59.341 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.317+0000 7f85c4a00640 1 --2- 172.21.15.145:0/462099658 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f85a0076320 0x7f85a00787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:59.341 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.318+0000 7f5935600640 1 -- 172.21.15.145:0/468768286 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5930010040 con 0x7f593814cdc0 2024-05-09T04:33:59.341 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.319+0000 7f85c4a00640 1 --2- 172.21.15.145:0/462099658 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f85a0076320 0x7f85a00787e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f85b8009a40 tx=0x7f85b800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:59.342 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.322+0000 7fcd09600640 1 -- 172.21.15.145:0/3990378403 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fcd0001a070 con 0x7fcd0403e840 2024-05-09T04:33:59.342 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.324+0000 7fcd09600640 1 --2- 172.21.15.145:0/3990378403 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fccec0762d0 0x7fccec078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:33:59.342 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.325+0000 7fcd09600640 1 -- 172.21.15.145:0/3990378403 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fcd00097640 con 0x7fcd0403e840 2024-05-09T04:33:59.342 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.326+0000 7fcd10a00640 1 --2- 172.21.15.145:0/3990378403 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fccec0762d0 0x7fccec078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:33:59.342 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.327+0000 7fcd10a00640 1 --2- 172.21.15.145:0/3990378403 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fccec0762d0 0x7fccec078790 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fccf8034000 tx=0x7fccf802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:33:59.342 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.327+0000 7fcd09600640 1 -- 172.21.15.145:0/3990378403 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fcd00060f80 con 0x7fcd0403e840 2024-05-09T04:33:59.342 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.330+0000 7f85bcc00640 1 -- 172.21.15.145:0/462099658 <== mon.2 v2:172.21.15.192:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f85b004b030 con 0x7f85c00d2db0 2024-05-09T04:33:59.735 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.735+0000 7f383be00640 1 -- 172.21.15.145:0/612557606 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f3810005850 con 0x7f38340b5930 2024-05-09T04:33:59.738 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.736+0000 7f3831600640 1 -- 172.21.15.145:0/612557606 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3828081ce0 con 0x7f38340b5930 2024-05-09T04:33:59.738 INFO:teuthology.orchestra.run.smithi145.stdout:55834574852 2024-05-09T04:33:59.740 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.740+0000 7f383be00640 1 -- 172.21.15.145:0/612557606 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f38140764d0 msgr2=0x7f3814078990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:59.740 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.740+0000 7f383be00640 1 --2- 172.21.15.145:0/612557606 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f38140764d0 0x7f3814078990 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3834162b50 tx=0x7f3820008040 comp rx=0 tx=0).stop 2024-05-09T04:33:59.740 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.740+0000 7f383be00640 1 -- 172.21.15.145:0/612557606 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38340b5930 msgr2=0x7f3834161dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:59.740 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.740+0000 7f383be00640 1 --2- 172.21.15.145:0/612557606 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38340b5930 0x7f3834161dd0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3828031000 tx=0x7f38280046c0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.741 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.741+0000 7f383be00640 1 -- 172.21.15.145:0/612557606 shutdown_connections 2024-05-09T04:33:59.741 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.741+0000 7f383be00640 1 --2- 172.21.15.145:0/612557606 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f38140764d0 0x7f3814078990 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.741 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.741+0000 7f383be00640 1 --2- 172.21.15.145:0/612557606 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f383415f5f0 0x7f3834159290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.741 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.741+0000 7f383be00640 1 --2- 172.21.15.145:0/612557606 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38340d2db0 0x7f38340b87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.741 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.741+0000 7f383be00640 1 --2- 172.21.15.145:0/612557606 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38340b5930 0x7f3834161dd0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.741 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.741+0000 7f383be00640 1 -- 172.21.15.145:0/612557606 >> 172.21.15.145:0/612557606 conn(0x7f383409e520 msgr2=0x7f383406e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:59.741 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.741+0000 7f383be00640 1 -- 172.21.15.145:0/612557606 shutdown_connections 2024-05-09T04:33:59.741 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.741+0000 7f383be00640 1 -- 172.21.15.145:0/612557606 wait complete. 2024-05-09T04:33:59.748 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.748+0000 7f9132600640 1 -- 172.21.15.145:0/3524831578 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f912c160a20 con 0x7f912c06e0b0 2024-05-09T04:33:59.749 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.749+0000 7f9123400640 1 -- 172.21.15.145:0/3524831578 <== mon.2 v2:172.21.15.192:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f911c060ba0 con 0x7f912c06e0b0 2024-05-09T04:33:59.752 INFO:teuthology.orchestra.run.smithi145.stdout:55834574853 2024-05-09T04:33:59.753 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.753+0000 7f9132600640 1 -- 172.21.15.145:0/3524831578 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f910c0762d0 msgr2=0x7f910c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:59.753 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.754+0000 7f9132600640 1 --2- 172.21.15.145:0/3524831578 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f910c0762d0 0x7f910c078790 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9124034000 tx=0x7f912402f000 comp rx=0 tx=0).stop 2024-05-09T04:33:59.753 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.754+0000 7f9132600640 1 -- 172.21.15.145:0/3524831578 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f912c06e0b0 msgr2=0x7f912c09e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:59.753 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.754+0000 7f9132600640 1 --2- 172.21.15.145:0/3524831578 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f912c06e0b0 0x7f912c09e060 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f911c00e880 tx=0x7f911c00ed40 comp rx=0 tx=0).stop 2024-05-09T04:33:59.754 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.754+0000 7f25f9e00640 1 -- 172.21.15.145:0/793631504 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f25ec0531f0 con 0x7f25ec0e7710 2024-05-09T04:33:59.755 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.755+0000 7f9132600640 1 -- 172.21.15.145:0/3524831578 shutdown_connections 2024-05-09T04:33:59.755 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.755+0000 7f9132600640 1 --2- 172.21.15.145:0/3524831578 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f910c0762d0 0x7f910c078790 secure :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9124034000 tx=0x7f912402f000 comp rx=0 tx=0).stop 2024-05-09T04:33:59.755 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.755+0000 7f9132600640 1 --2- 172.21.15.145:0/3524831578 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f912c0cd640 0x7f912c09e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.755 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.755+0000 7f9132600640 1 --2- 172.21.15.145:0/3524831578 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f912c06e0b0 0x7f912c09e060 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.755 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.755+0000 7f9132600640 1 --2- 172.21.15.145:0/3524831578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f912c06c6c0 0x7f912c14e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.755 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.755+0000 7f9132600640 1 -- 172.21.15.145:0/3524831578 >> 172.21.15.145:0/3524831578 conn(0x7f912c0b2b40 msgr2=0x7f912c0b1870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:59.755 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.755+0000 7f25f0c00640 1 -- 172.21.15.145:0/793631504 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f25f4090030 con 0x7f25ec0e7710 2024-05-09T04:33:59.758 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.756+0000 7f9132600640 1 -- 172.21.15.145:0/3524831578 shutdown_connections 2024-05-09T04:33:59.758 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.756+0000 7f9132600640 1 -- 172.21.15.145:0/3524831578 wait complete. 2024-05-09T04:33:59.758 INFO:teuthology.orchestra.run.smithi145.stdout:55834574853 2024-05-09T04:33:59.760 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.760+0000 7f25f9e00640 1 -- 172.21.15.145:0/793631504 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f25d4076350 msgr2=0x7f25d4078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:59.761 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.760+0000 7f25f9e00640 1 --2- 172.21.15.145:0/793631504 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f25d4076350 0x7f25d4078810 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f25e8034000 tx=0x7f25e802e000 comp rx=0 tx=0).stop 2024-05-09T04:33:59.761 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.760+0000 7f25f9e00640 1 -- 172.21.15.145:0/793631504 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f25ec0e7710 msgr2=0x7f25ec0e7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:59.761 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.760+0000 7f25f9e00640 1 --2- 172.21.15.145:0/793631504 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f25ec0e7710 0x7f25ec0e7b80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f25f4074fe0 tx=0x7f25f4069fc0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.762 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.762+0000 7f25f9e00640 1 -- 172.21.15.145:0/793631504 shutdown_connections 2024-05-09T04:33:59.762 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.762+0000 7f25f9e00640 1 --2- 172.21.15.145:0/793631504 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f25d4076350 0x7f25d4078810 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.762 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.762+0000 7f25f9e00640 1 --2- 172.21.15.145:0/793631504 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f25ec0e7710 0x7f25ec0e7b80 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.762 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.762+0000 7f25f9e00640 1 --2- 172.21.15.145:0/793631504 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f25ec0589c0 0x7f25ec051780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.762 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.762+0000 7f25f9e00640 1 --2- 172.21.15.145:0/793631504 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25ec056ab0 0x7f25ec0526f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.762 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.762+0000 7f25f9e00640 1 -- 172.21.15.145:0/793631504 >> 172.21.15.145:0/793631504 conn(0x7f25ec058ee0 msgr2=0x7f25ec007020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:59.762 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.762+0000 7f25f9e00640 1 -- 172.21.15.145:0/793631504 shutdown_connections 2024-05-09T04:33:59.763 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.763+0000 7f25f9e00640 1 -- 172.21.15.145:0/793631504 wait complete. 2024-05-09T04:33:59.763 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.764+0000 7f9f84c00640 1 -- 172.21.15.145:0/1638358900 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f9f800b5210 con 0x7f9f801629a0 2024-05-09T04:33:59.764 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.765+0000 7f9f76000640 1 -- 172.21.15.145:0/1638358900 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9f801518f0 con 0x7f9f801629a0 2024-05-09T04:33:59.768 INFO:teuthology.orchestra.run.smithi145.stdout:55834574854 2024-05-09T04:33:59.769 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.5 2024-05-09T04:33:59.770 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.769+0000 7f9f84c00640 1 -- 172.21.15.145:0/1638358900 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f9f600763d0 msgr2=0x7f9f60078890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:59.770 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.769+0000 7f9f84c00640 1 --2- 172.21.15.145:0/1638358900 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f9f600763d0 0x7f9f60078890 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9f78009c70 tx=0x7f9f78009040 comp rx=0 tx=0).stop 2024-05-09T04:33:59.770 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.769+0000 7f9f84c00640 1 -- 172.21.15.145:0/1638358900 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f801629a0 msgr2=0x7f9f800cbb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:59.770 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.769+0000 7f9f84c00640 1 --2- 172.21.15.145:0/1638358900 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f801629a0 0x7f9f800cbb60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9f7001d400 tx=0x7f9f7001d8c0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.770 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.770+0000 7f9f84c00640 1 -- 172.21.15.145:0/1638358900 shutdown_connections 2024-05-09T04:33:59.770 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.770+0000 7f9f84c00640 1 --2- 172.21.15.145:0/1638358900 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f9f600763d0 0x7f9f60078890 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.770 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.770+0000 7f9f84c00640 1 --2- 172.21.15.145:0/1638358900 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9f801629a0 0x7f9f800cbb60 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.770 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.770+0000 7f9f84c00640 1 --2- 172.21.15.145:0/1638358900 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f800b5540 0x7f9f800ca000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.770 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.770+0000 7f9f84c00640 1 --2- 172.21.15.145:0/1638358900 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f8006e220 0x7f9f801513b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.770 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.770+0000 7f9f84c00640 1 -- 172.21.15.145:0/1638358900 >> 172.21.15.145:0/1638358900 conn(0x7f9f800b4a20 msgr2=0x7f9f8009ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:59.771 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.770+0000 7f9f84c00640 1 -- 172.21.15.145:0/1638358900 shutdown_connections 2024-05-09T04:33:59.771 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.770+0000 7f9f84c00640 1 -- 172.21.15.145:0/1638358900 wait complete. 2024-05-09T04:33:59.771 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.771+0000 7f6903e00640 1 -- 172.21.15.145:0/2672212600 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f68fc152ae0 con 0x7f68fc15f5c0 2024-05-09T04:33:59.771 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.772+0000 7f68f9600640 1 -- 172.21.15.145:0/2672212600 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f68ec085660 con 0x7f68fc15f5c0 2024-05-09T04:33:59.773 INFO:teuthology.orchestra.run.smithi145.stdout:55834574852 2024-05-09T04:33:59.778 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.777+0000 7f6903e00640 1 -- 172.21.15.145:0/2672212600 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f68dc0763a0 msgr2=0x7f68dc078860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:59.778 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.777+0000 7f6903e00640 1 --2- 172.21.15.145:0/2672212600 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f68dc0763a0 0x7f68dc078860 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f68fc073b30 tx=0x7f68f400b040 comp rx=0 tx=0).stop 2024-05-09T04:33:59.778 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.777+0000 7f6903e00640 1 -- 172.21.15.145:0/2672212600 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68fc15f5c0 msgr2=0x7f68fc155890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:59.778 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.777+0000 7f6903e00640 1 --2- 172.21.15.145:0/2672212600 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68fc15f5c0 0x7f68fc155890 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f68ec009870 tx=0x7f68ec0046c0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.778 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.778+0000 7f6903e00640 1 -- 172.21.15.145:0/2672212600 shutdown_connections 2024-05-09T04:33:59.778 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.778+0000 7f6903e00640 1 --2- 172.21.15.145:0/2672212600 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f68dc0763a0 0x7f68dc078860 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.779 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.778+0000 7f6903e00640 1 --2- 172.21.15.145:0/2672212600 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68fc15f5c0 0x7f68fc155890 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.779 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.778+0000 7f6903e00640 1 --2- 172.21.15.145:0/2672212600 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68fc0b6290 0x7f68fc1f8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.779 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.778+0000 7f6903e00640 1 --2- 172.21.15.145:0/2672212600 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f68fc0b4470 0x7f68fc0b9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.779 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.778+0000 7f6903e00640 1 -- 172.21.15.145:0/2672212600 >> 172.21.15.145:0/2672212600 conn(0x7f68fc0d5080 msgr2=0x7f68fc0d0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:59.779 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.778+0000 7f6903e00640 1 -- 172.21.15.145:0/2672212600 shutdown_connections 2024-05-09T04:33:59.779 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.778+0000 7f6903e00640 1 -- 172.21.15.145:0/2672212600 wait complete. 2024-05-09T04:33:59.779 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.779+0000 7f5940200640 1 -- 172.21.15.145:0/468768286 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f593806d490 con 0x7f593814cdc0 2024-05-09T04:33:59.780 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.780+0000 7f5935600640 1 -- 172.21.15.145:0/468768286 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5930061fa0 con 0x7f593814cdc0 2024-05-09T04:33:59.780 INFO:teuthology.orchestra.run.smithi145.stdout:55834574854 2024-05-09T04:33:59.780 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.780+0000 7fcd12800640 1 -- 172.21.15.145:0/3990378403 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fcd040db100 con 0x7fcd0403e840 2024-05-09T04:33:59.782 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.782+0000 7fcd09600640 1 -- 172.21.15.145:0/3990378403 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fcd00060880 con 0x7fcd0403e840 2024-05-09T04:33:59.784 INFO:teuthology.orchestra.run.smithi145.stdout:55834574853 2024-05-09T04:33:59.785 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.784+0000 7f5940200640 1 -- 172.21.15.145:0/468768286 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f591c076320 msgr2=0x7f591c0787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:59.785 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.784+0000 7f5940200640 1 --2- 172.21.15.145:0/468768286 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f591c076320 0x7f591c0787e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f592c034000 tx=0x7f592c02e000 comp rx=0 tx=0).stop 2024-05-09T04:33:59.785 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.784+0000 7f5940200640 1 -- 172.21.15.145:0/468768286 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f593814cdc0 msgr2=0x7f593813a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:59.785 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.784+0000 7f5940200640 1 --2- 172.21.15.145:0/468768286 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f593814cdc0 0x7f593813a470 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f593000efc0 tx=0x7f593000c490 comp rx=0 tx=0).stop 2024-05-09T04:33:59.785 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.784+0000 7f85c7400640 1 -- 172.21.15.145:0/462099658 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f85c00cbe90 con 0x7f85c00d2db0 2024-05-09T04:33:59.786 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.0 2024-05-09T04:33:59.786 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.785+0000 7f85bcc00640 1 -- 172.21.15.145:0/462099658 <== mon.2 v2:172.21.15.192:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f85b00818a0 con 0x7f85c00d2db0 2024-05-09T04:33:59.786 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.785+0000 7fcd12800640 1 -- 172.21.15.145:0/3990378403 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fccec0762d0 msgr2=0x7fccec078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:59.787 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.785+0000 7fcd12800640 1 --2- 172.21.15.145:0/3990378403 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fccec0762d0 0x7fccec078790 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fccf8034000 tx=0x7fccf802e000 comp rx=0 tx=0).stop 2024-05-09T04:33:59.787 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.785+0000 7fcd12800640 1 -- 172.21.15.145:0/3990378403 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcd0403e840 msgr2=0x7fcd040e7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:59.787 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.785+0000 7fcd12800640 1 --2- 172.21.15.145:0/3990378403 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcd0403e840 0x7fcd040e7380 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcd0000e7e0 tx=0x7fcd0000eca0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.787 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.785+0000 7f5940200640 1 -- 172.21.15.145:0/468768286 shutdown_connections 2024-05-09T04:33:59.787 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.785+0000 7f5940200640 1 --2- 172.21.15.145:0/468768286 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f591c076320 0x7f591c0787e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.787 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.785+0000 7f5940200640 1 --2- 172.21.15.145:0/468768286 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5938164af0 0x7f593813b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.787 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.785+0000 7f5940200640 1 --2- 172.21.15.145:0/468768286 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f593814d1a0 0x7f5938160b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.788 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.785+0000 7f5940200640 1 --2- 172.21.15.145:0/468768286 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f593814cdc0 0x7f593813a470 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.788 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.785+0000 7f5940200640 1 -- 172.21.15.145:0/468768286 >> 172.21.15.145:0/468768286 conn(0x7f593806c8d0 msgr2=0x7f593813e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:59.788 INFO:teuthology.orchestra.run.smithi145.stdout:55834574855 2024-05-09T04:33:59.788 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.785+0000 7f5940200640 1 -- 172.21.15.145:0/468768286 shutdown_connections 2024-05-09T04:33:59.788 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.785+0000 7fcd12800640 1 -- 172.21.15.145:0/3990378403 shutdown_connections 2024-05-09T04:33:59.789 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.785+0000 7fcd12800640 1 --2- 172.21.15.145:0/3990378403 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fccec0762d0 0x7fccec078790 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.789 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.785+0000 7fcd12800640 1 --2- 172.21.15.145:0/3990378403 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcd040e30c0 0x7fcd040d7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.789 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.785+0000 7f5940200640 1 -- 172.21.15.145:0/468768286 wait complete. 2024-05-09T04:33:59.789 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.785+0000 7fcd12800640 1 --2- 172.21.15.145:0/3990378403 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcd04054130 0x7fcd040509f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.789 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.785+0000 7fcd12800640 1 --2- 172.21.15.145:0/3990378403 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcd0403e840 0x7fcd040e7380 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.790 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.785+0000 7fcd12800640 1 -- 172.21.15.145:0/3990378403 >> 172.21.15.145:0/3990378403 conn(0x7fcd040559e0 msgr2=0x7fcd040068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:59.790 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.785+0000 7fcd12800640 1 -- 172.21.15.145:0/3990378403 shutdown_connections 2024-05-09T04:33:59.790 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.786+0000 7fcd12800640 1 -- 172.21.15.145:0/3990378403 wait complete. 2024-05-09T04:33:59.790 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.788+0000 7f85c7400640 1 -- 172.21.15.145:0/462099658 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f85a0076320 msgr2=0x7f85a00787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:59.790 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.788+0000 7f85c7400640 1 --2- 172.21.15.145:0/462099658 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f85a0076320 0x7f85a00787e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f85b8009a40 tx=0x7f85b800b040 comp rx=0 tx=0).stop 2024-05-09T04:33:59.791 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.788+0000 7f85c7400640 1 -- 172.21.15.145:0/462099658 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85c00d2db0 msgr2=0x7f85c0154800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:33:59.791 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.788+0000 7f85c7400640 1 --2- 172.21.15.145:0/462099658 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85c00d2db0 0x7f85c0154800 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f85b0031000 tx=0x7f85b00046c0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.791 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.788+0000 7f85c7400640 1 -- 172.21.15.145:0/462099658 shutdown_connections 2024-05-09T04:33:59.791 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.788+0000 7f85c7400640 1 --2- 172.21.15.145:0/462099658 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f85a0076320 0x7f85a00787e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.791 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.788+0000 7f85c7400640 1 --2- 172.21.15.145:0/462099658 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85c015f5f0 0x7f85c00b8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.791 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.788+0000 7f85c7400640 1 --2- 172.21.15.145:0/462099658 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85c00d2db0 0x7f85c0154800 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.791 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.788+0000 7f85c7400640 1 --2- 172.21.15.145:0/462099658 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f85c00b5930 0x7f85c01536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:33:59.791 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.788+0000 7f85c7400640 1 -- 172.21.15.145:0/462099658 >> 172.21.15.145:0/462099658 conn(0x7f85c009e520 msgr2=0x7f85c00b6cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:33:59.792 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.3 2024-05-09T04:33:59.792 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.789+0000 7f85c7400640 1 -- 172.21.15.145:0/462099658 shutdown_connections 2024-05-09T04:33:59.792 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:33:59.789+0000 7f85c7400640 1 -- 172.21.15.145:0/462099658 wait complete. 2024-05-09T04:33:59.793 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.2 2024-05-09T04:33:59.808 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.4 2024-05-09T04:33:59.808 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.1 2024-05-09T04:33:59.810 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.7 2024-05-09T04:33:59.812 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.6 2024-05-09T04:34:00.770 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-09T04:34:00.786 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-09T04:34:00.792 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-09T04:34:00.794 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-09T04:34:00.808 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-09T04:34:00.809 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-09T04:34:00.811 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-09T04:34:00.813 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-09T04:34:01.051 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.051+0000 7f698bc00640 1 Processor -- start 2024-05-09T04:34:01.051 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.051+0000 7f698bc00640 1 -- start start 2024-05-09T04:34:01.054 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.054+0000 7f698bc00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f697c04fa80 0x7f697c04fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:01.054 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.054+0000 7f698bc00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f697c0df410 0x7f697c0382b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:01.055 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.055+0000 7f6989200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f697c04fa80 0x7f697c04fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:01.055 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.055+0000 7f698bc00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f697c0387f0 0x7f697c038c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:01.055 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.055+0000 7f6989200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f697c04fa80 0x7f697c04fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.145:49838/0 (socket says 172.21.15.145:49838) 2024-05-09T04:34:01.055 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.055+0000 7f6989200640 1 -- 172.21.15.145:0/1002225004 learned_addr learned my addr 172.21.15.145:0/1002225004 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:01.055 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.055+0000 7f698bc00640 1 -- 172.21.15.145:0/1002225004 --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f697c006890 con 0x7f697c0df410 2024-05-09T04:34:01.055 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.055+0000 7f698bc00640 1 -- 172.21.15.145:0/1002225004 --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f697c0503a0 con 0x7f697c04fa80 2024-05-09T04:34:01.056 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.055+0000 7f698bc00640 1 -- 172.21.15.145:0/1002225004 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f697c006ae0 con 0x7f697c0387f0 2024-05-09T04:34:01.056 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.056+0000 7f6989c00640 1 --2- 172.21.15.145:0/1002225004 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f697c0387f0 0x7f697c038c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:01.056 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.056+0000 7f6983e00640 1 --2- 172.21.15.145:0/1002225004 >> v2:172.21.15.145:3300/0 conn(0x7f697c0df410 0x7f697c0382b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:01.057 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.057+0000 7f6989c00640 1 -- 172.21.15.145:0/1002225004 >> v2:172.21.15.192:3300/0 conn(0x7f697c04fa80 msgr2=0x7f697c04fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:01.057 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.057+0000 7f6989c00640 1 --2- 172.21.15.145:0/1002225004 >> v2:172.21.15.192:3300/0 conn(0x7f697c04fa80 0x7f697c04fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.057 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.057+0000 7f6989c00640 1 -- 172.21.15.145:0/1002225004 >> v2:172.21.15.145:3300/0 conn(0x7f697c0df410 msgr2=0x7f697c0382b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:01.057 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.057+0000 7f6989c00640 1 --2- 172.21.15.145:0/1002225004 >> v2:172.21.15.145:3300/0 conn(0x7f697c0df410 0x7f697c0382b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.058 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.057+0000 7f6983e00640 1 --2- 172.21.15.145:0/1002225004 >> v2:172.21.15.145:3300/0 conn(0x7f697c0df410 0x7f697c0382b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:34:01.058 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.057+0000 7f6989c00640 1 -- 172.21.15.145:0/1002225004 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f697c006dc0 con 0x7f697c0387f0 2024-05-09T04:34:01.059 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.059+0000 7f6989c00640 1 --2- 172.21.15.145:0/1002225004 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f697c0387f0 0x7f697c038c80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f698406abf0 tx=0x7f698406c270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4df97573b1376477 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:01.059 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.060+0000 7f6983400640 1 -- 172.21.15.145:0/1002225004 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69840b2070 con 0x7f697c0387f0 2024-05-09T04:34:01.060 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.060+0000 7f6983400640 1 -- 172.21.15.145:0/1002225004 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69840a6e50 con 0x7f697c0387f0 2024-05-09T04:34:01.061 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.062+0000 7f698bc00640 1 -- 172.21.15.145:0/1002225004 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f697c0387f0 msgr2=0x7f697c038c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:01.061 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.062+0000 7f698bc00640 1 --2- 172.21.15.145:0/1002225004 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f697c0387f0 0x7f697c038c80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f698406abf0 tx=0x7f698406c270 comp rx=0 tx=0).stop 2024-05-09T04:34:01.062 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.062+0000 7f698bc00640 1 -- 172.21.15.145:0/1002225004 shutdown_connections 2024-05-09T04:34:01.062 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.062+0000 7f698bc00640 1 --2- 172.21.15.145:0/1002225004 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f697c0387f0 0x7f697c038c80 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.062 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.062+0000 7f698bc00640 1 --2- 172.21.15.145:0/1002225004 >> v2:172.21.15.145:3300/0 conn(0x7f697c0df410 0x7f697c0382b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.062 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.062+0000 7f698bc00640 1 --2- 172.21.15.145:0/1002225004 >> v2:172.21.15.192:3300/0 conn(0x7f697c04fa80 0x7f697c04fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.063 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.062+0000 7f698bc00640 1 -- 172.21.15.145:0/1002225004 >> 172.21.15.145:0/1002225004 conn(0x7f697c009fa0 msgr2=0x7f697c00a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:01.063 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.063+0000 7f698bc00640 1 -- 172.21.15.145:0/1002225004 shutdown_connections 2024-05-09T04:34:01.064 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.063+0000 7f698bc00640 1 -- 172.21.15.145:0/1002225004 wait complete. 2024-05-09T04:34:01.066 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.066+0000 7f698bc00640 1 Processor -- start 2024-05-09T04:34:01.066 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.067+0000 7f698bc00640 1 -- start start 2024-05-09T04:34:01.068 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.069+0000 7f698bc00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f697c0387f0 0x7f697c0e1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:01.069 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.070+0000 7f698bc00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f697c04fa80 0x7f697c0cff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:01.070 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.070+0000 7f698bc00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f697c0df410 0x7f697c039990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:01.070 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.070+0000 7f698bc00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f697c052b30 con 0x7f697c0387f0 2024-05-09T04:34:01.070 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.071+0000 7f698bc00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f697c007380 con 0x7f697c0df410 2024-05-09T04:34:01.071 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.071+0000 7f6983e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f697c04fa80 0x7f697c0cff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:01.071 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.071+0000 7f6983e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f697c04fa80 0x7f697c0cff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.145:42552/0 (socket says 172.21.15.145:42552) 2024-05-09T04:34:01.071 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.071+0000 7f6983e00640 1 -- 172.21.15.145:0/3835493073 learned_addr learned my addr 172.21.15.145:0/3835493073 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:01.071 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.071+0000 7f6989c00640 1 --2- 172.21.15.145:0/3835493073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f697c0df410 0x7f697c039990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:01.071 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.071+0000 7f698bc00640 1 -- 172.21.15.145:0/3835493073 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f697c052cb0 con 0x7f697c04fa80 2024-05-09T04:34:01.071 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.071+0000 7f6989200640 1 --2- 172.21.15.145:0/3835493073 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f697c0387f0 0x7f697c0e1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:01.072 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.072+0000 7f6983e00640 1 -- 172.21.15.145:0/3835493073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f697c0df410 msgr2=0x7f697c039990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:01.072 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.072+0000 7f6983e00640 1 --2- 172.21.15.145:0/3835493073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f697c0df410 0x7f697c039990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.072 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.072+0000 7f6983e00640 1 -- 172.21.15.145:0/3835493073 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f697c0387f0 msgr2=0x7f697c0e1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:01.072 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.072+0000 7f6983e00640 1 --2- 172.21.15.145:0/3835493073 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f697c0387f0 0x7f697c0e1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.073 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.072+0000 7f6983e00640 1 -- 172.21.15.145:0/3835493073 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f697c0d0440 con 0x7f697c04fa80 2024-05-09T04:34:01.073 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.073+0000 7f6989200640 1 --2- 172.21.15.145:0/3835493073 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f697c0387f0 0x7f697c0e1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:34:01.073 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.073+0000 7f6983e00640 1 --2- 172.21.15.145:0/3835493073 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f697c04fa80 0x7f697c0cff00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f697000c7a0 tx=0x7f697000cc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:01.074 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.074+0000 7f6980c00640 1 -- 172.21.15.145:0/3835493073 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697001e070 con 0x7f697c04fa80 2024-05-09T04:34:01.074 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.074+0000 7f6980c00640 1 -- 172.21.15.145:0/3835493073 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6970007a30 con 0x7f697c04fa80 2024-05-09T04:34:01.074 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.074+0000 7f6980c00640 1 -- 172.21.15.145:0/3835493073 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6970007e00 con 0x7f697c04fa80 2024-05-09T04:34:01.074 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.074+0000 7f698bc00640 1 -- 172.21.15.145:0/3835493073 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f697c0490e0 con 0x7f697c04fa80 2024-05-09T04:34:01.074 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.074+0000 7f698bc00640 1 -- 172.21.15.145:0/3835493073 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f697c0496a0 con 0x7f697c04fa80 2024-05-09T04:34:01.075 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.076+0000 7f698bc00640 1 -- 172.21.15.145:0/3835493073 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f697c0089c0 con 0x7f697c04fa80 2024-05-09T04:34:01.082 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.080+0000 7f6980c00640 1 -- 172.21.15.145:0/3835493073 <== mon.2 v2:172.21.15.192:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6970021070 con 0x7f697c04fa80 2024-05-09T04:34:01.082 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.082+0000 7f6980c00640 1 --2- 172.21.15.145:0/3835493073 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f69640762d0 0x7f6964078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:01.082 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.082+0000 7f6980c00640 1 -- 172.21.15.145:0/3835493073 <== mon.2 v2:172.21.15.192:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f697009bac0 con 0x7f697c04fa80 2024-05-09T04:34:01.083 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.083+0000 7f6989200640 1 --2- 172.21.15.145:0/3835493073 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f69640762d0 0x7f6964078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:01.084 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.084+0000 7f6989200640 1 --2- 172.21.15.145:0/3835493073 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f69640762d0 0x7f6964078790 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f697800a520 tx=0x7f6978008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:01.094 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.094+0000 7f6980c00640 1 -- 172.21.15.145:0/3835493073 <== mon.2 v2:172.21.15.192:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f697c0d0440 con 0x7f697c04fa80 2024-05-09T04:34:01.100 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.099+0000 7faaab800640 1 Processor -- start 2024-05-09T04:34:01.100 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.099+0000 7faaab800640 1 -- start start 2024-05-09T04:34:01.101 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.101+0000 7faaab800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7faaa415b370 0x7faaa406f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:01.102 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.102+0000 7faaab800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faaa4073420 0x7faaa40cf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:01.102 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.102+0000 7faaab800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faaa40cedf0 con 0x7faaa415b750 2024-05-09T04:34:01.102 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.102+0000 7faaab800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7faaa406ff80 con 0x7faaa415b370 2024-05-09T04:34:01.102 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.102+0000 7faaab800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7faaa4070260 con 0x7faaa4073420 2024-05-09T04:34:01.103 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.103+0000 7faaa8e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7faaa415b370 0x7faaa406f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:01.103 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.103+0000 7faaa3e00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7faaa415b750 0x7faaa4072d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:37050/0 (socket says 172.21.15.145:37050) 2024-05-09T04:34:01.103 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.103+0000 7faaa3e00640 1 -- 172.21.15.145:0/3962096242 learned_addr learned my addr 172.21.15.145:0/3962096242 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:34:01.103 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.103+0000 7faaa9800640 1 --2- 172.21.15.145:0/3962096242 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faaa4073420 0x7faaa40cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:01.103 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.103+0000 7faaa3400640 1 -- 172.21.15.145:0/3962096242 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 203224709 0 0) 0x7faaa40cedf0 con 0x7faaa415b750 2024-05-09T04:34:01.103 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.103+0000 7faaa3400640 1 -- 172.21.15.145:0/3962096242 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa88003420 con 0x7faaa415b750 2024-05-09T04:34:01.105 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.104+0000 7faaa8e00640 1 -- 172.21.15.145:0/3962096242 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faaa4073420 msgr2=0x7faaa40cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:01.105 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.104+0000 7faaa8e00640 1 --2- 172.21.15.145:0/3962096242 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faaa4073420 0x7faaa40cf580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.105 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.104+0000 7faaa8e00640 1 -- 172.21.15.145:0/3962096242 >> v1:172.21.15.145:6789/0 conn(0x7faaa415b750 legacy=0x7faaa4072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:01.105 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.104+0000 7faaa8e00640 1 -- 172.21.15.145:0/3962096242 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaa40cf060 con 0x7faaa415b370 2024-05-09T04:34:01.106 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.106+0000 7faaa8e00640 1 --2- 172.21.15.145:0/3962096242 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faaa415b370 0x7faaa406f720 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7faa940097a0 tx=0x7faa9402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1f1fa7fc324acb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:01.107 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.107+0000 7faaa3400640 1 -- 172.21.15.145:0/3962096242 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa94037070 con 0x7faaa415b370 2024-05-09T04:34:01.107 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.107+0000 7faaa3400640 1 -- 172.21.15.145:0/3962096242 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa94032440 con 0x7faaa415b370 2024-05-09T04:34:01.107 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.107+0000 7faaa3400640 1 -- 172.21.15.145:0/3962096242 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa94032830 con 0x7faaa415b370 2024-05-09T04:34:01.109 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.109+0000 7faaab800640 1 -- 172.21.15.145:0/3962096242 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faaa415b370 msgr2=0x7faaa406f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:01.109 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.109+0000 7faaab800640 1 --2- 172.21.15.145:0/3962096242 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faaa415b370 0x7faaa406f720 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7faa940097a0 tx=0x7faa9402a1c0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.109 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.109+0000 7faaab800640 1 -- 172.21.15.145:0/3962096242 shutdown_connections 2024-05-09T04:34:01.109 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.109+0000 7faaab800640 1 --2- 172.21.15.145:0/3962096242 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faaa4073420 0x7faaa40cf580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.109 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.109+0000 7faaab800640 1 --2- 172.21.15.145:0/3962096242 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faaa415b370 0x7faaa406f720 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.109 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.109+0000 7faaab800640 1 -- 172.21.15.145:0/3962096242 >> 172.21.15.145:0/3962096242 conn(0x7faaa40b2c00 msgr2=0x7faaa40b2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:01.109 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.109+0000 7faaab800640 1 -- 172.21.15.145:0/3962096242 shutdown_connections 2024-05-09T04:34:01.109 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.109+0000 7faaab800640 1 -- 172.21.15.145:0/3962096242 wait complete. 2024-05-09T04:34:01.110 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.109+0000 7f3d25e00640 1 Processor -- start 2024-05-09T04:34:01.110 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.108+0000 7f0eb8a00640 1 Processor -- start 2024-05-09T04:34:01.110 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.108+0000 7f0eb8a00640 1 -- start start 2024-05-09T04:34:01.110 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.109+0000 7f3d25e00640 1 -- start start 2024-05-09T04:34:01.111 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.110+0000 7f0eb8a00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f0eb41558a0 0x7f0eb406c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:01.111 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.111+0000 7f0eb8a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0eb4070230 0x7f0eb406fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:01.111 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.111+0000 7f0eb8a00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0eb415ef30 con 0x7f0eb406ced0 2024-05-09T04:34:01.111 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.111+0000 7f0eb8a00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f0eb415fb40 con 0x7f0eb41558a0 2024-05-09T04:34:01.112 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.111+0000 7f0eb8a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0eb415f210 con 0x7f0eb4070230 2024-05-09T04:34:01.112 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.111+0000 7f0eb3400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f0eb41558a0 0x7f0eb406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:01.112 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.111+0000 7f3d25e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f3d200d2120 0x7f3d200d58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:01.112 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.111+0000 7f3d25e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f3d200b5fe0 0x7f3d200d0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:01.112 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.111+0000 7f3d25e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d200d3780 con 0x7f3d20162c00 2024-05-09T04:34:01.112 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.111+0000 7f3d25e00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f3d200d5ec0 con 0x7f3d200b5fe0 2024-05-09T04:34:01.112 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.111+0000 7f3d25e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f3d200d3a00 con 0x7f3d200d2120 2024-05-09T04:34:01.112 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.112+0000 7faaab800640 1 Processor -- start 2024-05-09T04:34:01.113 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.112+0000 7f3d1ea00640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f3d20162c00 0x7f3d200d1a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.145:49628/0 (socket says 172.21.15.145:49628) 2024-05-09T04:34:01.113 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.112+0000 7faaab800640 1 -- start start 2024-05-09T04:34:01.113 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.112+0000 7f3d1ea00640 1 -- 172.21.15.145:0/1741417959 learned_addr learned my addr 172.21.15.145:0/1741417959 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:34:01.113 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.112+0000 7f3d1f400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f3d200b5fe0 0x7f3d200d0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:01.114 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.112+0000 7f0eb3400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f0eb41558a0 0x7f0eb406c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.145:49856/0 (socket says 172.21.15.145:49856) 2024-05-09T04:34:01.114 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.112+0000 7f0eb3400640 1 -- 172.21.15.145:0/3372146387 learned_addr learned my addr 172.21.15.145:0/3372146387 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:01.114 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.112+0000 7f0eb3e00640 1 --2- 172.21.15.145:0/3372146387 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0eb4070230 0x7f0eb406fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:01.114 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.112+0000 7f0eb3400640 1 -- 172.21.15.145:0/3372146387 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0eb4070230 msgr2=0x7f0eb406fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:01.114 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.112+0000 7f0eb3400640 1 --2- 172.21.15.145:0/3372146387 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0eb4070230 0x7f0eb406fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.114 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.112+0000 7f0eb3400640 1 -- 172.21.15.145:0/3372146387 >> v1:172.21.15.145:6789/0 conn(0x7f0eb406ced0 legacy=0x7f0eb406f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:01.114 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.112+0000 7f0eb3400640 1 -- 172.21.15.145:0/3372146387 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eb415f4f0 con 0x7f0eb41558a0 2024-05-09T04:34:01.114 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.114+0000 7f3d1f400640 1 -- 172.21.15.145:0/1741417959 >> v2:172.21.15.192:3300/0 conn(0x7f3d200d2120 msgr2=0x7f3d200d58e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T04:34:01.115 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.114+0000 7faaab800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faaa4073420 0x7faaa4152e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:01.115 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.114+0000 7f3d1f400640 1 --2- 172.21.15.145:0/1741417959 >> v2:172.21.15.192:3300/0 conn(0x7f3d200d2120 0x7f3d200d58e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.115 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.114+0000 7f3d1f400640 1 -- 172.21.15.145:0/1741417959 >> v1:172.21.15.192:6789/0 conn(0x7f3d20162c00 legacy=0x7f3d200d1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:01.115 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.114+0000 7f3d1f400640 1 -- 172.21.15.145:0/1741417959 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d200d3ce0 con 0x7f3d200b5fe0 2024-05-09T04:34:01.115 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.115+0000 7f0eb3400640 1 --2- 172.21.15.145:0/3372146387 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0eb41558a0 0x7f0eb406c990 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0ea40098a0 tx=0x7f0ea4032a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2828b91cf28005e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:01.115 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.115+0000 7f0eb2000640 1 -- 172.21.15.145:0/3372146387 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea403b070 con 0x7f0eb41558a0 2024-05-09T04:34:01.116 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.115+0000 7faaab800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faaa415b370 0x7faaa41e4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:01.116 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.115+0000 7faaab800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faaa415b750 0x7faaa4155ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:01.116 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.115+0000 7faaab800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7faaa40b36f0 con 0x7faaa4073420 2024-05-09T04:34:01.116 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.115+0000 7faaab800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7faaa4070540 con 0x7faaa415b750 2024-05-09T04:34:01.116 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.115+0000 7faaab800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7faaa40b3870 con 0x7faaa415b370 2024-05-09T04:34:01.116 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.115+0000 7faaa8e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faaa4073420 0x7faaa4152e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:01.116 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.116+0000 7faaa8e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faaa4073420 0x7faaa4152e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:37708/0 (socket says 172.21.15.145:37708) 2024-05-09T04:34:01.116 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.116+0000 7faaa8e00640 1 -- 172.21.15.145:0/2512685809 learned_addr learned my addr 172.21.15.145:0/2512685809 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:01.117 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.115+0000 7f0eb2000640 1 -- 172.21.15.145:0/3372146387 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ea4037610 con 0x7f0eb41558a0 2024-05-09T04:34:01.117 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.115+0000 7f0eb2000640 1 -- 172.21.15.145:0/3372146387 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea40379e0 con 0x7f0eb41558a0 2024-05-09T04:34:01.117 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.116+0000 7f0eb8a00640 1 -- 172.21.15.145:0/3372146387 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0eb41558a0 msgr2=0x7f0eb406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:01.117 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.116+0000 7f0eb8a00640 1 --2- 172.21.15.145:0/3372146387 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0eb41558a0 0x7f0eb406c990 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0ea40098a0 tx=0x7f0ea4032a00 comp rx=0 tx=0).stop 2024-05-09T04:34:01.117 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.116+0000 7faaa9800640 1 --2- 172.21.15.145:0/2512685809 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faaa415b750 0x7faaa4155ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:01.117 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.116+0000 7f3d1f400640 1 --2- 172.21.15.145:0/1741417959 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d200b5fe0 0x7f3d200d0030 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3d200d17b0 tx=0x7f3d1802a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eb4891a425a9f7f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:01.118 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.116+0000 7f0eb8a00640 1 -- 172.21.15.145:0/3372146387 shutdown_connections 2024-05-09T04:34:01.118 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.116+0000 7f0eb8a00640 1 --2- 172.21.15.145:0/3372146387 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0eb4070230 0x7f0eb406fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.118 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.116+0000 7f0eb8a00640 1 --2- 172.21.15.145:0/3372146387 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0eb41558a0 0x7f0eb406c990 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.118 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.116+0000 7f0eb8a00640 1 -- 172.21.15.145:0/3372146387 >> 172.21.15.145:0/3372146387 conn(0x7f0eb40b2d80 msgr2=0x7f0eb40b3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:01.118 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.116+0000 7f3d1d600640 1 -- 172.21.15.145:0/1741417959 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d18037070 con 0x7f3d200b5fe0 2024-05-09T04:34:01.119 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.116+0000 7faaa8e00640 1 -- 172.21.15.145:0/2512685809 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faaa415b370 msgr2=0x7faaa41e4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:01.119 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.116+0000 7faaa8e00640 1 --2- 172.21.15.145:0/2512685809 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faaa415b370 0x7faaa41e4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.119 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.116+0000 7faaa8e00640 1 -- 172.21.15.145:0/2512685809 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faaa415b750 msgr2=0x7faaa4155ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:01.119 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.116+0000 7faaa8e00640 1 --2- 172.21.15.145:0/2512685809 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faaa415b750 0x7faaa4155ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.119 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.116+0000 7faaa8e00640 1 -- 172.21.15.145:0/2512685809 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaa4155020 con 0x7faaa4073420 2024-05-09T04:34:01.119 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.117+0000 7f3d1d600640 1 -- 172.21.15.145:0/1741417959 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d18032440 con 0x7f3d200b5fe0 2024-05-09T04:34:01.119 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.117+0000 7f3d1d600640 1 -- 172.21.15.145:0/1741417959 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d18032810 con 0x7f3d200b5fe0 2024-05-09T04:34:01.119 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.117+0000 7f0eb8a00640 1 -- 172.21.15.145:0/3372146387 shutdown_connections 2024-05-09T04:34:01.120 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.117+0000 7faaa8e00640 1 --2- 172.21.15.145:0/2512685809 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faaa4073420 0x7faaa4152e30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7faa940097a0 tx=0x7faa940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:01.120 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.117+0000 7f0eb8a00640 1 -- 172.21.15.145:0/3372146387 wait complete. 2024-05-09T04:34:01.120 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.118+0000 7faaa0c00640 1 -- 172.21.15.145:0/2512685809 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa94037040 con 0x7faaa4073420 2024-05-09T04:34:01.120 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.118+0000 7f3d25e00640 1 -- 172.21.15.145:0/1741417959 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d200b5fe0 msgr2=0x7f3d200d0030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:01.121 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.118+0000 7f4a6a800640 1 Processor -- start 2024-05-09T04:34:01.121 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.118+0000 7f4a6a800640 1 -- start start 2024-05-09T04:34:01.121 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.118+0000 7f3d25e00640 1 --2- 172.21.15.145:0/1741417959 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d200b5fe0 0x7f3d200d0030 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3d200d17b0 tx=0x7f3d1802a130 comp rx=0 tx=0).stop 2024-05-09T04:34:01.122 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.119+0000 7f3d25e00640 1 -- 172.21.15.145:0/1741417959 shutdown_connections 2024-05-09T04:34:01.122 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.119+0000 7f3d25e00640 1 --2- 172.21.15.145:0/1741417959 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d200b5fe0 0x7f3d200d0030 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.122 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.119+0000 7f3d25e00640 1 --2- 172.21.15.145:0/1741417959 >> v2:172.21.15.192:3300/0 conn(0x7f3d200d2120 0x7f3d200d58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.122 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.119+0000 7f3d25e00640 1 -- 172.21.15.145:0/1741417959 >> 172.21.15.145:0/1741417959 conn(0x7f3d2009e510 msgr2=0x7f3d200d51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:01.122 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.119+0000 7f3d25e00640 1 -- 172.21.15.145:0/1741417959 shutdown_connections 2024-05-09T04:34:01.122 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.118+0000 7faaa0c00640 1 -- 172.21.15.145:0/2512685809 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa94004230 con 0x7faaa4073420 2024-05-09T04:34:01.122 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.120+0000 7faaa0c00640 1 -- 172.21.15.145:0/2512685809 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa94003610 con 0x7faaa4073420 2024-05-09T04:34:01.122 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.119+0000 7f3d25e00640 1 -- 172.21.15.145:0/1741417959 wait complete. 2024-05-09T04:34:01.123 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.120+0000 7f0eb8a00640 1 Processor -- start 2024-05-09T04:34:01.123 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.120+0000 7faaab800640 1 -- 172.21.15.145:0/2512685809 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaa41e56e0 con 0x7faaa4073420 2024-05-09T04:34:01.123 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.120+0000 7faaab800640 1 -- 172.21.15.145:0/2512685809 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaa41552e0 con 0x7faaa4073420 2024-05-09T04:34:01.123 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.121+0000 7f4a6a800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f4a640d38a0 0x7f4a640b36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:01.123 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.121+0000 7f4a68a00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f4a640c6460 0x7f4a640afb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:37056/0 (socket says 172.21.15.145:37056) 2024-05-09T04:34:01.124 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.121+0000 7f6e2d800640 1 Processor -- start 2024-05-09T04:34:01.124 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.121+0000 7f4a68a00640 1 -- 172.21.15.145:0/1950908719 learned_addr learned my addr 172.21.15.145:0/1950908719 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:34:01.124 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.122+0000 7f6e2d800640 1 -- start start 2024-05-09T04:34:01.124 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.122+0000 7f4a6a800640 1 -- 172.21.15.145:0/1950908719 --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a64163b50 con 0x7f4a640c6460 2024-05-09T04:34:01.125 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.122+0000 7f3d25e00640 1 Processor -- start 2024-05-09T04:34:01.125 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.122+0000 7f3d25e00640 1 -- start start 2024-05-09T04:34:01.125 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.123+0000 7f4a69e00640 1 --2- 172.21.15.145:0/1950908719 >> v2:172.21.15.192:3300/0 conn(0x7f4a640d38a0 0x7f4a640b36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:01.125 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.122+0000 7f4a63e00640 1 -- 172.21.15.145:0/1950908719 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3290026326 0 0) 0x7f4a64163b50 con 0x7f4a640c6460 2024-05-09T04:34:01.126 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.123+0000 7f4a6a800640 1 -- 172.21.15.145:0/1950908719 --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a640b2930 con 0x7f4a64163770 2024-05-09T04:34:01.126 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.123+0000 7f4a6a800640 1 -- 172.21.15.145:0/1950908719 --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f4a640b26e0 con 0x7f4a640d38a0 2024-05-09T04:34:01.126 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.123+0000 7f6e2d800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6e2815f5f0 0x7f6e280d1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:01.126 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.123+0000 7f6e2d800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f6e280d2db0 0x7f6e280d2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:01.126 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.123+0000 7f6e2d800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e280b4b70 con 0x7f6e280b5930 2024-05-09T04:34:01.126 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.123+0000 7f6e2d800640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f6e2815f9d0 con 0x7f6e280d2db0 2024-05-09T04:34:01.126 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.123+0000 7f6e2d800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f6e280b4e50 con 0x7f6e2815f5f0 2024-05-09T04:34:01.127 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.123+0000 7f99e2000640 1 Processor -- start 2024-05-09T04:34:01.127 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.123+0000 7f99e2000640 1 -- start start 2024-05-09T04:34:01.127 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.123+0000 7f0eb8a00640 1 -- start start 2024-05-09T04:34:01.128 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.124+0000 7f3d25e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d200b5fe0 0x7f3d200ce3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:01.128 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.124+0000 7f3d25e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3d200d2120 0x7f3d200b7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:01.128 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.124+0000 7f3d25e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d20162c00 0x7f3d200b9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:01.128 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.124+0000 7f3d25e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3d200b6670 con 0x7f3d20162c00 2024-05-09T04:34:01.128 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.124+0000 7f3d25e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3d200b4950 con 0x7f3d200b5fe0 2024-05-09T04:34:01.128 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.124+0000 7f3d25e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f3d200b8610 con 0x7f3d200d2120 2024-05-09T04:34:01.129 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.124+0000 7f6e26a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6e2815f5f0 0x7f6e280d1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:01.129 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.124+0000 7f6e26a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6e2815f5f0 0x7f6e280d1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.145:42590/0 (socket says 172.21.15.145:42590) 2024-05-09T04:34:01.129 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.124+0000 7f4a63e00640 1 -- 172.21.15.145:0/1950908719 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4a4c003520 con 0x7f4a640c6460 2024-05-09T04:34:01.129 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.124+0000 7f4a63e00640 1 -- 172.21.15.145:0/1950908719 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4125875927 0 0) 0x7f4a640b2930 con 0x7f4a64163770 2024-05-09T04:34:01.130 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.124+0000 7f6e26a00640 1 -- 172.21.15.145:0/835750837 learned_addr learned my addr 172.21.15.145:0/835750837 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:01.130 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.124+0000 7f4a63e00640 1 -- 172.21.15.145:0/1950908719 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4a64163b50 con 0x7f4a64163770 2024-05-09T04:34:01.130 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.124+0000 7f8a3a000640 1 Processor -- start 2024-05-09T04:34:01.130 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.124+0000 7faaab800640 1 -- 172.21.15.145:0/2512685809 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaa4075c10 con 0x7faaa4073420 2024-05-09T04:34:01.131 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.125+0000 7f3d1e000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3d200d2120 0x7f3d200b7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:01.131 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.125+0000 7f3d1e000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3d200d2120 0x7f3d200b7d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.145:42594/0 (socket says 172.21.15.145:42594) 2024-05-09T04:34:01.131 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.125+0000 7f3d1e000640 1 -- 172.21.15.145:0/3291087378 learned_addr learned my addr 172.21.15.145:0/3291087378 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:01.131 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.125+0000 7f3d1f400640 1 --2- 172.21.15.145:0/3291087378 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d20162c00 0x7f3d200b9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:01.131 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.125+0000 7f3d1ea00640 1 --2- 172.21.15.145:0/3291087378 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d200b5fe0 0x7f3d200ce3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:01.132 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.125+0000 7f6e26a00640 1 -- 172.21.15.145:0/835750837 >> v2:172.21.15.145:3300/0 conn(0x7f6e280d2db0 msgr2=0x7f6e280d2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:01.132 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.125+0000 7f6e26a00640 1 --2- 172.21.15.145:0/835750837 >> v2:172.21.15.145:3300/0 conn(0x7f6e280d2db0 0x7f6e280d2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.132 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.125+0000 7f99e2000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99dc1602b0 0x7f99dc0d04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:01.132 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.126+0000 7f99e2000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f99dc162e00 0x7f99dc073180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:01.132 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.125+0000 7f4a63e00640 1 -- 172.21.15.145:0/1950908719 <== mon.1 v1:172.21.15.192:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1344622793 0 0) 0x7f4a64163b50 con 0x7f4a64163770 2024-05-09T04:34:01.132 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.125+0000 7f4a63e00640 1 -- 172.21.15.145:0/1950908719 >> v2:172.21.15.192:3300/0 conn(0x7f4a640d38a0 msgr2=0x7f4a640b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:01.132 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.125+0000 7f4a63e00640 1 --2- 172.21.15.145:0/1950908719 >> v2:172.21.15.192:3300/0 conn(0x7f4a640d38a0 0x7f4a640b36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.132 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.125+0000 7f4a63e00640 1 -- 172.21.15.145:0/1950908719 >> v1:172.21.15.145:6789/0 conn(0x7f4a640c6460 legacy=0x7f4a640afb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:01.133 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.125+0000 7f4a63e00640 1 -- 172.21.15.145:0/1950908719 --> v1:172.21.15.192:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a640b2c10 con 0x7f4a64163770 2024-05-09T04:34:01.133 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.125+0000 7f4a63e00640 1 -- 172.21.15.145:0/1950908719 <== mon.1 v1:172.21.15.192:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3474413261 0 0) 0x7f4a580016d0 con 0x7f4a64163770 2024-05-09T04:34:01.133 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.125+0000 7f4a63e00640 1 -- 172.21.15.145:0/1950908719 >> v1:172.21.15.192:6789/0 conn(0x7f4a64163770 legacy=0x7f4a640c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:01.133 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.125+0000 7f4a63e00640 1 --2- 172.21.15.145:0/1950908719 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4a4c004c90 0x7f4a4c005090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:01.133 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.125+0000 7f4a63e00640 1 --2- 172.21.15.145:0/1950908719 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4a4c0037a0 0x7f4a4c003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:01.133 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.125+0000 7f4a63e00640 1 --2- 172.21.15.145:0/1950908719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a4c0041d0 0x7f4a4c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:01.133 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.125+0000 7f4a63e00640 1 -- 172.21.15.145:0/1950908719 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4a640b26e0 con 0x7f4a4c0037a0 2024-05-09T04:34:01.133 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.125+0000 7f4a63e00640 1 -- 172.21.15.145:0/1950908719 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4c00c190 con 0x7f4a4c0041d0 2024-05-09T04:34:01.133 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.125+0000 7f4a63e00640 1 -- 172.21.15.145:0/1950908719 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f4a4c00c390 con 0x7f4a4c004c90 2024-05-09T04:34:01.133 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.125+0000 7f6e26a00640 1 -- 172.21.15.145:0/835750837 >> v1:172.21.15.145:6789/0 conn(0x7f6e280b5930 legacy=0x7f6e280b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:01.133 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.125+0000 7f6e26a00640 1 -- 172.21.15.145:0/835750837 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e280b5130 con 0x7f6e2815f5f0 2024-05-09T04:34:01.134 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.126+0000 7f99e2000640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99dc163570 con 0x7f99dc15eec0 2024-05-09T04:34:01.134 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.126+0000 7f99e2000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f99dc160710 con 0x7f99dc162e00 2024-05-09T04:34:01.134 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.126+0000 7f99e2000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f99dc163850 con 0x7f99dc1602b0 2024-05-09T04:34:01.134 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.125+0000 7f0eb8a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0eb406ced0 0x7f0eb4153cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:01.134 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.126+0000 7f0eb8a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0eb4070230 0x7f0eb415b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:01.134 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.126+0000 7f0eb8a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0eb41558a0 0x7f0eb415ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:01.134 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.126+0000 7f0eb8a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb406dec0 con 0x7f0eb41558a0 2024-05-09T04:34:01.134 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.126+0000 7f0eb8a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0eb4155c80 con 0x7f0eb406ced0 2024-05-09T04:34:01.134 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.126+0000 7f0eb8a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0eb406e040 con 0x7f0eb4070230 2024-05-09T04:34:01.135 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.126+0000 7f4a69e00640 1 --2- 172.21.15.145:0/1950908719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a4c0041d0 0x7f4a4c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:01.135 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.126+0000 7f0eb3400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0eb406ced0 0x7f0eb4153cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:01.135 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.126+0000 7f0eb3400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0eb406ced0 0x7f0eb4153cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.145:49920/0 (socket says 172.21.15.145:49920) 2024-05-09T04:34:01.135 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.126+0000 7f0eb3400640 1 -- 172.21.15.145:0/1989200502 learned_addr learned my addr 172.21.15.145:0/1989200502 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:01.136 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.127+0000 7faaa0c00640 1 -- 172.21.15.145:0/2512685809 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7faa9402ad10 con 0x7faaa4073420 2024-05-09T04:34:01.136 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.126+0000 7f8a3a000640 1 -- start start 2024-05-09T04:34:01.136 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.127+0000 7f4a69e00640 1 -- 172.21.15.145:0/1950908719 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4a4c004c90 msgr2=0x7f4a4c005090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:01.136 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.127+0000 7f4a69e00640 1 --2- 172.21.15.145:0/1950908719 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4a4c004c90 0x7f4a4c005090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.136 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.126+0000 7f3d1e000640 1 -- 172.21.15.145:0/3291087378 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d200b5fe0 msgr2=0x7f3d200ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:01.137 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.127+0000 7f4a69e00640 1 -- 172.21.15.145:0/1950908719 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4a4c0037a0 msgr2=0x7f4a4c003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:01.137 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.127+0000 7f4a69e00640 1 --2- 172.21.15.145:0/1950908719 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4a4c0037a0 0x7f4a4c003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.137 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.126+0000 7f3d1e000640 1 --2- 172.21.15.145:0/3291087378 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d200b5fe0 0x7f3d200ce3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.137 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.126+0000 7f3d1e000640 1 -- 172.21.15.145:0/3291087378 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d20162c00 msgr2=0x7f3d200b9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:01.137 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.127+0000 7f4a69e00640 1 -- 172.21.15.145:0/1950908719 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a640b2c10 con 0x7f4a4c0041d0 2024-05-09T04:34:01.138 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.126+0000 7f3d1e000640 1 --2- 172.21.15.145:0/3291087378 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d20162c00 0x7f3d200b9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.138 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.126+0000 7f3d1e000640 1 -- 172.21.15.145:0/3291087378 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d200b88f0 con 0x7f3d200d2120 2024-05-09T04:34:01.138 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.127+0000 7f3d1e000640 1 --2- 172.21.15.145:0/3291087378 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3d200d2120 0x7f3d200b7d30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3d0800b4a0 tx=0x7f3d0800b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:01.138 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.127+0000 7f0eb2a00640 1 --2- 172.21.15.145:0/1989200502 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0eb4070230 0x7f0eb415b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:01.138 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.127+0000 7f0eb3400640 1 -- 172.21.15.145:0/1989200502 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0eb4070230 msgr2=0x7f0eb415b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:01.138 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.127+0000 7f0eb3400640 1 --2- 172.21.15.145:0/1989200502 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0eb4070230 0x7f0eb415b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.138 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.127+0000 7f0eb3400640 1 -- 172.21.15.145:0/1989200502 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0eb41558a0 msgr2=0x7f0eb415ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:01.139 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.127+0000 7f0eb3400640 1 --2- 172.21.15.145:0/1989200502 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0eb41558a0 0x7f0eb415ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.139 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.128+0000 7f0eb3400640 1 -- 172.21.15.145:0/1989200502 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eb415be50 con 0x7f0eb406ced0 2024-05-09T04:34:01.139 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.128+0000 7f4a69e00640 1 -- 172.21.15.145:0/1950908719 reap_dead start 2024-05-09T04:34:01.139 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.128+0000 7f8a3a000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f8a2c0e3160 0x7f8a2c0e7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:01.139 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.128+0000 7faaa0c00640 1 --2- 172.21.15.145:0/2512685809 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7faa840763d0 0x7faa84078890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:01.140 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.128+0000 7f8a3a000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f8a2c03c0a0 0x7f8a2c058050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:01.140 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.128+0000 7f6e26a00640 1 --2- 172.21.15.145:0/835750837 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6e2815f5f0 0x7f6e280d1bd0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6e18009870 tx=0x7f6e1802a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a111a67d174ec98a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:01.140 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.128+0000 7f0eb3400640 1 --2- 172.21.15.145:0/1989200502 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0eb406ced0 0x7f0eb4153cb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4000c00 tx=0x7f0ea40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:01.140 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.128+0000 7f8a3a000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8a2c053cb0 0x7f8a2c054140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:01.141 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.128+0000 7f4a69e00640 1 --2- 172.21.15.145:0/1950908719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a4c0041d0 0x7f4a4c00b9c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c009fc0 tx=0x7f4a4c005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d3f541c1c59a7dd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:01.141 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.129+0000 7f8a3a000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f8a2c058590 con 0x7f8a2c0e3160 2024-05-09T04:34:01.141 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.128+0000 7faaa0c00640 1 -- 172.21.15.145:0/2512685809 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7faa940b7ff0 con 0x7faaa4073420 2024-05-09T04:34:01.141 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.129+0000 7f0eab400640 1 -- 172.21.15.145:0/1989200502 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea403b040 con 0x7f0eb406ced0 2024-05-09T04:34:01.141 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.129+0000 7f0eab400640 1 -- 172.21.15.145:0/1989200502 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ea4040a90 con 0x7f0eb406ced0 2024-05-09T04:34:01.141 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.129+0000 7f0eab400640 1 -- 172.21.15.145:0/1989200502 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea404a3f0 con 0x7f0eb406ced0 2024-05-09T04:34:01.142 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.129+0000 7f0eb8a00640 1 -- 172.21.15.145:0/1989200502 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0eb415c0f0 con 0x7f0eb406ced0 2024-05-09T04:34:01.142 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.129+0000 7f8a38c00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f8a2c0e3160 0x7f8a2c0e7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:01.142 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.129+0000 7f8a38c00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f8a2c0e3160 0x7f8a2c0e7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:37764/0 (socket says 172.21.15.145:37764) 2024-05-09T04:34:01.142 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.129+0000 7f8a38c00640 1 -- 172.21.15.145:0/796977844 learned_addr learned my addr 172.21.15.145:0/796977844 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:01.142 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.129+0000 7f8a3a000640 1 -- 172.21.15.145:0/796977844 --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f8a2c054680 con 0x7f8a2c03c0a0 2024-05-09T04:34:01.142 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.129+0000 7f8a3a000640 1 -- 172.21.15.145:0/796977844 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8a2c058810 con 0x7f8a2c053cb0 2024-05-09T04:34:01.142 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.129+0000 7f99e0c00640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f99dc15eec0 0x7f99dc06e5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.145:49642/0 (socket says 172.21.15.145:49642) 2024-05-09T04:34:01.143 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.129+0000 7f6e25600640 1 -- 172.21.15.145:0/835750837 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e18037070 con 0x7f6e2815f5f0 2024-05-09T04:34:01.143 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.129+0000 7f6e25600640 1 -- 172.21.15.145:0/835750837 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e18032440 con 0x7f6e2815f5f0 2024-05-09T04:34:01.143 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.129+0000 7f6e25600640 1 -- 172.21.15.145:0/835750837 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e18032810 con 0x7f6e2815f5f0 2024-05-09T04:34:01.143 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.128+0000 7f3d17400640 1 -- 172.21.15.145:0/3291087378 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d08014070 con 0x7f3d200d2120 2024-05-09T04:34:01.143 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.129+0000 7f0eb8a00640 1 -- 172.21.15.145:0/1989200502 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0eb415c6b0 con 0x7f0eb406ced0 2024-05-09T04:34:01.144 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.128+0000 7f3d17400640 1 -- 172.21.15.145:0/3291087378 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d08010610 con 0x7f3d200d2120 2024-05-09T04:34:01.144 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.128+0000 7f3d17400640 1 -- 172.21.15.145:0/3291087378 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d08010a00 con 0x7f3d200d2120 2024-05-09T04:34:01.144 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.128+0000 7f3d25e00640 1 -- 172.21.15.145:0/3291087378 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d200b8bd0 con 0x7f3d200d2120 2024-05-09T04:34:01.144 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.128+0000 7f3d25e00640 1 -- 172.21.15.145:0/3291087378 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d200b9190 con 0x7f3d200d2120 2024-05-09T04:34:01.144 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.129+0000 7f99e0c00640 1 -- 172.21.15.145:0/3158932120 learned_addr learned my addr 172.21.15.145:0/3158932120 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:34:01.144 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.129+0000 7f99dbe00640 1 --2- 172.21.15.145:0/3158932120 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99dc1602b0 0x7f99dc0d04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:01.144 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.129+0000 7f8a39600640 1 --2- 172.21.15.145:0/796977844 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8a2c053cb0 0x7f8a2c054140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:01.145 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.129+0000 7f8a33e00640 1 --2- 172.21.15.145:0/796977844 >> v2:172.21.15.192:3300/0 conn(0x7f8a2c03c0a0 0x7f8a2c058050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:01.145 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.129+0000 7f4a63e00640 1 -- 172.21.15.145:0/1950908719 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5c035020 con 0x7f4a4c0041d0 2024-05-09T04:34:01.145 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.130+0000 7f8a38c00640 1 -- 172.21.15.145:0/796977844 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8a2c053cb0 msgr2=0x7f8a2c054140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:01.145 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.130+0000 7f8a38c00640 1 --2- 172.21.15.145:0/796977844 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8a2c053cb0 0x7f8a2c054140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.145 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.130+0000 7f8a38c00640 1 -- 172.21.15.145:0/796977844 >> v2:172.21.15.192:3300/0 conn(0x7f8a2c03c0a0 msgr2=0x7f8a2c058050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:01.145 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.130+0000 7f8a38c00640 1 --2- 172.21.15.145:0/796977844 >> v2:172.21.15.192:3300/0 conn(0x7f8a2c03c0a0 0x7f8a2c058050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.145 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.130+0000 7f8a38c00640 1 -- 172.21.15.145:0/796977844 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a2c058af0 con 0x7f8a2c0e3160 2024-05-09T04:34:01.146 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.130+0000 7f99e1600640 1 --2- 172.21.15.145:0/3158932120 >> v2:172.21.15.145:3300/0 conn(0x7f99dc162e00 0x7f99dc073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:01.146 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.130+0000 7f99e1600640 1 -- 172.21.15.145:0/3158932120 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99dc1602b0 msgr2=0x7f99dc0d04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:01.146 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.130+0000 7f99e1600640 1 --2- 172.21.15.145:0/3158932120 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99dc1602b0 0x7f99dc0d04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.146 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.130+0000 7f99e1600640 1 -- 172.21.15.145:0/3158932120 >> v1:172.21.15.192:6789/0 conn(0x7f99dc15eec0 legacy=0x7f99dc06e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:01.146 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.130+0000 7f99e1600640 1 -- 172.21.15.145:0/3158932120 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99dc163b30 con 0x7f99dc162e00 2024-05-09T04:34:01.146 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.130+0000 7f6e2d800640 1 -- 172.21.15.145:0/835750837 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6e2815f5f0 msgr2=0x7f6e280d1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:01.146 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.130+0000 7f6e2d800640 1 --2- 172.21.15.145:0/835750837 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6e2815f5f0 0x7f6e280d1bd0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6e18009870 tx=0x7f6e1802a250 comp rx=0 tx=0).stop 2024-05-09T04:34:01.146 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.130+0000 7f6e2d800640 1 -- 172.21.15.145:0/835750837 shutdown_connections 2024-05-09T04:34:01.147 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.130+0000 7f6e2d800640 1 --2- 172.21.15.145:0/835750837 >> v2:172.21.15.145:3300/0 conn(0x7f6e280d2db0 0x7f6e280d2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.147 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.130+0000 7f6e2d800640 1 --2- 172.21.15.145:0/835750837 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6e2815f5f0 0x7f6e280d1bd0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.147 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.130+0000 7f6e2d800640 1 -- 172.21.15.145:0/835750837 >> 172.21.15.145:0/835750837 conn(0x7f6e2809e520 msgr2=0x7f6e280d51e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:01.147 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.131+0000 7f0eb8a00640 1 -- 172.21.15.145:0/1989200502 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0eb4076780 con 0x7f0eb406ced0 2024-05-09T04:34:01.147 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.131+0000 7f8a38c00640 1 --2- 172.21.15.145:0/796977844 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a2c0e3160 0x7f8a2c0e7780 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c053700 tx=0x7f8a2802a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c805f1595c3bac63 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:01.148 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.131+0000 7f4a63e00640 1 -- 172.21.15.145:0/1950908719 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a5c003bd0 con 0x7f4a4c0041d0 2024-05-09T04:34:01.148 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.132+0000 7f4a6a800640 1 -- 172.21.15.145:0/1950908719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a4c0041d0 msgr2=0x7f4a4c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:01.148 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.132+0000 7f4a6a800640 1 --2- 172.21.15.145:0/1950908719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a4c0041d0 0x7f4a4c00b9c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c009fc0 tx=0x7f4a4c005830 comp rx=0 tx=0).stop 2024-05-09T04:34:01.148 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.132+0000 7f4a6a800640 1 -- 172.21.15.145:0/1950908719 shutdown_connections 2024-05-09T04:34:01.148 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.132+0000 7f4a6a800640 1 --2- 172.21.15.145:0/1950908719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a4c0041d0 0x7f4a4c00b9c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.148 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.132+0000 7f4a6a800640 1 -- 172.21.15.145:0/1950908719 >> 172.21.15.145:0/1950908719 conn(0x7f4a640d2d80 msgr2=0x7f4a640d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:01.148 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.132+0000 7f99e1600640 1 --2- 172.21.15.145:0/3158932120 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99dc162e00 0x7f99dc073180 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f99cc0097a0 tx=0x7f99cc02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7a47e78424701e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:01.148 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.132+0000 7f3d25e00640 1 -- 172.21.15.145:0/3291087378 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d200b8390 con 0x7f3d200d2120 2024-05-09T04:34:01.149 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.132+0000 7f4a6a800640 1 -- 172.21.15.145:0/1950908719 shutdown_connections 2024-05-09T04:34:01.149 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.132+0000 7faaa3e00640 1 --2- 172.21.15.145:0/2512685809 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7faa840763d0 0x7faa84078890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:01.149 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.132+0000 7f4a6a800640 1 -- 172.21.15.145:0/1950908719 wait complete. 2024-05-09T04:34:01.149 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.130+0000 7f6e2d800640 1 -- 172.21.15.145:0/835750837 shutdown_connections 2024-05-09T04:34:01.149 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.131+0000 7f6e2d800640 1 -- 172.21.15.145:0/835750837 wait complete. 2024-05-09T04:34:01.149 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.133+0000 7f6e2d800640 1 Processor -- start 2024-05-09T04:34:01.150 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.133+0000 7faaa3e00640 1 --2- 172.21.15.145:0/2512685809 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7faa840763d0 0x7faa84078890 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7faa9c0015a0 tx=0x7faa9c0015f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:01.150 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.134+0000 7f6e2d800640 1 -- start start 2024-05-09T04:34:01.150 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.134+0000 7f4a6a800640 1 Processor -- start 2024-05-09T04:34:01.150 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.134+0000 7f0eab400640 1 -- 172.21.15.145:0/1989200502 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0ea4051060 con 0x7f0eb406ced0 2024-05-09T04:34:01.151 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.135+0000 7f6e2d800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6e280b5930 0x7f6e280b7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:01.151 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.135+0000 7f6e2d800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e280d2db0 0x7f6e280cecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:01.151 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.135+0000 7f6e2d800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6e2815f5f0 0x7f6e2814d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:01.151 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.136+0000 7f6e26a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6e280b5930 0x7f6e280b7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:01.151 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.136+0000 7f6e26a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6e280b5930 0x7f6e280b7e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.145:49942/0 (socket says 172.21.15.145:49942) 2024-05-09T04:34:01.151 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.136+0000 7f6e26a00640 1 -- 172.21.15.145:0/2050724596 learned_addr learned my addr 172.21.15.145:0/2050724596 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:01.152 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.136+0000 7f0eab400640 1 --2- 172.21.15.145:0/1989200502 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f0e94076300 0x7f0e940787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:01.152 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.136+0000 7f6e26000640 1 --2- 172.21.15.145:0/2050724596 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e280d2db0 0x7f6e280cecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:01.152 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.136+0000 7f6e2d800640 1 -- 172.21.15.145:0/2050724596 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6e280d3c90 con 0x7f6e280d2db0 2024-05-09T04:34:01.152 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.136+0000 7f6e2d800640 1 -- 172.21.15.145:0/2050724596 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6e280d4040 con 0x7f6e280b5930 2024-05-09T04:34:01.152 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.136+0000 7f6e2d800640 1 -- 172.21.15.145:0/2050724596 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f6e280cf230 con 0x7f6e2815f5f0 2024-05-09T04:34:01.152 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.136+0000 7f0eab400640 1 -- 172.21.15.145:0/1989200502 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0ea40c0700 con 0x7f0eb406ced0 2024-05-09T04:34:01.153 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.136+0000 7f8a33400640 1 -- 172.21.15.145:0/796977844 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a28037070 con 0x7f8a2c0e3160 2024-05-09T04:34:01.153 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.136+0000 7f6e27400640 1 --2- 172.21.15.145:0/2050724596 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6e2815f5f0 0x7f6e2814d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:01.153 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.137+0000 7f0eb2a00640 1 --2- 172.21.15.145:0/1989200502 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f0e94076300 0x7f0e940787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:01.154 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.137+0000 7f99db400640 1 -- 172.21.15.145:0/3158932120 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99cc037070 con 0x7f99dc162e00 2024-05-09T04:34:01.154 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.137+0000 7f99db400640 1 -- 172.21.15.145:0/3158932120 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99cc032440 con 0x7f99dc162e00 2024-05-09T04:34:01.154 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.137+0000 7f99db400640 1 -- 172.21.15.145:0/3158932120 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99cc037070 con 0x7f99dc162e00 2024-05-09T04:34:01.154 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.137+0000 7f8a33400640 1 -- 172.21.15.145:0/796977844 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a28032440 con 0x7f8a2c0e3160 2024-05-09T04:34:01.154 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.137+0000 7f8a33400640 1 -- 172.21.15.145:0/796977844 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a28037070 con 0x7f8a2c0e3160 2024-05-09T04:34:01.154 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.137+0000 7faaa0c00640 1 -- 172.21.15.145:0/2512685809 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7faa940818b0 con 0x7faaa4073420 2024-05-09T04:34:01.155 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.137+0000 7f6e26a00640 1 -- 172.21.15.145:0/2050724596 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6e2815f5f0 msgr2=0x7f6e2814d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:01.155 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.137+0000 7f6e26a00640 1 --2- 172.21.15.145:0/2050724596 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6e2815f5f0 0x7f6e2814d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.155 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.137+0000 7f4a6a800640 1 -- start start 2024-05-09T04:34:01.155 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.137+0000 7f6e26a00640 1 -- 172.21.15.145:0/2050724596 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e280d2db0 msgr2=0x7f6e280cecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:01.155 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.138+0000 7f6e26a00640 1 --2- 172.21.15.145:0/2050724596 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e280d2db0 0x7f6e280cecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.155 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.138+0000 7f6e26a00640 1 -- 172.21.15.145:0/2050724596 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e280b8490 con 0x7f6e280b5930 2024-05-09T04:34:01.156 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.138+0000 7f99e2000640 1 -- 172.21.15.145:0/3158932120 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99dc162e00 msgr2=0x7f99dc073180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:01.156 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.138+0000 7f99e2000640 1 --2- 172.21.15.145:0/3158932120 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99dc162e00 0x7f99dc073180 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f99cc0097a0 tx=0x7f99cc02a1c0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.156 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.138+0000 7f8a3a000640 1 -- 172.21.15.145:0/796977844 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a2c0e3160 msgr2=0x7f8a2c0e7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:01.156 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.138+0000 7f8a3a000640 1 --2- 172.21.15.145:0/796977844 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a2c0e3160 0x7f8a2c0e7780 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c053700 tx=0x7f8a2802a1d0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.157 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.138+0000 7f6e26000640 1 --2- 172.21.15.145:0/2050724596 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e280d2db0 0x7f6e280cecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:34:01.157 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.138+0000 7f99e2000640 1 -- 172.21.15.145:0/3158932120 shutdown_connections 2024-05-09T04:34:01.157 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.138+0000 7f99e2000640 1 --2- 172.21.15.145:0/3158932120 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99dc162e00 0x7f99dc073180 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.157 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.138+0000 7f99e2000640 1 --2- 172.21.15.145:0/3158932120 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99dc1602b0 0x7f99dc0d04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.157 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.138+0000 7f99e2000640 1 -- 172.21.15.145:0/3158932120 >> 172.21.15.145:0/3158932120 conn(0x7f99dc0b2960 msgr2=0x7f99dc0b2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:01.157 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.138+0000 7f0eb2a00640 1 --2- 172.21.15.145:0/1989200502 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f0e94076300 0x7f0e940787c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0eb41f44a0 tx=0x7f0ea0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:01.158 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.138+0000 7f8a3a000640 1 -- 172.21.15.145:0/796977844 shutdown_connections 2024-05-09T04:34:01.158 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.138+0000 7f8a3a000640 1 --2- 172.21.15.145:0/796977844 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8a2c053cb0 0x7f8a2c054140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.158 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.138+0000 7f8a3a000640 1 --2- 172.21.15.145:0/796977844 >> v2:172.21.15.192:3300/0 conn(0x7f8a2c03c0a0 0x7f8a2c058050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.158 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.138+0000 7f8a3a000640 1 --2- 172.21.15.145:0/796977844 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a2c0e3160 0x7f8a2c0e7780 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.158 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.138+0000 7f8a3a000640 1 -- 172.21.15.145:0/796977844 >> 172.21.15.145:0/796977844 conn(0x7f8a2c058ee0 msgr2=0x7f8a2c0592c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:01.158 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.138+0000 7f6e26a00640 1 --2- 172.21.15.145:0/2050724596 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6e280b5930 0x7f6e280b7e00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6e180090b0 tx=0x7f6e180046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:01.159 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.138+0000 7f99e2000640 1 -- 172.21.15.145:0/3158932120 shutdown_connections 2024-05-09T04:34:01.159 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.138+0000 7f99e2000640 1 -- 172.21.15.145:0/3158932120 wait complete. 2024-05-09T04:34:01.159 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.139+0000 7f8a3a000640 1 -- 172.21.15.145:0/796977844 shutdown_connections 2024-05-09T04:34:01.159 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.135+0000 7f3d17400640 1 -- 172.21.15.145:0/3291087378 <== mon.2 v2:172.21.15.192:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3d08017070 con 0x7f3d200d2120 2024-05-09T04:34:01.159 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.137+0000 7f3d17400640 1 --2- 172.21.15.145:0/3291087378 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f3cfc076320 0x7f3cfc0787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:01.159 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.139+0000 7f3d17400640 1 -- 172.21.15.145:0/3291087378 <== mon.2 v2:172.21.15.192:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3d0805c8c0 con 0x7f3d200d2120 2024-05-09T04:34:01.159 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.139+0000 7f4a6a800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4a4c0041d0 0x7f4a641f95b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:01.160 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.139+0000 7f4a6a800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4a640b6510 0x7f4a641fc410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:01.160 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.139+0000 7f4a6a800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a640b7d70 0x7f4a640c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:01.160 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.139+0000 7f4a6a800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4a640b4420 con 0x7f4a640b6510 2024-05-09T04:34:01.160 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.139+0000 7f4a6a800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4a640b31d0 con 0x7f4a640b7d70 2024-05-09T04:34:01.160 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.139+0000 7f4a6a800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f4a640b3d90 con 0x7f4a4c0041d0 2024-05-09T04:34:01.160 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.139+0000 7f8a3a000640 1 -- 172.21.15.145:0/796977844 wait complete. 2024-05-09T04:34:01.161 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.139+0000 7f4a68a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4a640b6510 0x7f4a641fc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:01.161 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.139+0000 7f4a69400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4a4c0041d0 0x7f4a641f95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:01.161 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.139+0000 7f4a68a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4a640b6510 0x7f4a641fc410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:37776/0 (socket says 172.21.15.145:37776) 2024-05-09T04:34:01.161 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.139+0000 7f4a69400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4a4c0041d0 0x7f4a641f95b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.145:42640/0 (socket says 172.21.15.145:42640) 2024-05-09T04:34:01.161 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.139+0000 7f4a68a00640 1 -- 172.21.15.145:0/3593675094 learned_addr learned my addr 172.21.15.145:0/3593675094 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:01.161 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.139+0000 7f4a69e00640 1 --2- 172.21.15.145:0/3593675094 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a640b7d70 0x7f4a640c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:01.161 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.140+0000 7f4a68a00640 1 -- 172.21.15.145:0/3593675094 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4a4c0041d0 msgr2=0x7f4a641f95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:01.161 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.140+0000 7f4a68a00640 1 --2- 172.21.15.145:0/3593675094 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4a4c0041d0 0x7f4a641f95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.161 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.140+0000 7f3d1ea00640 1 --2- 172.21.15.145:0/3291087378 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f3cfc076320 0x7f3cfc0787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:01.162 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.140+0000 7f4a68a00640 1 -- 172.21.15.145:0/3593675094 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a640b7d70 msgr2=0x7f4a640c8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:01.162 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.140+0000 7f4a68a00640 1 --2- 172.21.15.145:0/3593675094 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a640b7d70 0x7f4a640c8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.162 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.140+0000 7f4a68a00640 1 -- 172.21.15.145:0/3593675094 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a640b8680 con 0x7f4a640b6510 2024-05-09T04:34:01.162 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.140+0000 7f4a68a00640 1 --2- 172.21.15.145:0/3593675094 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4a640b6510 0x7f4a641fc410 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4a5000c4e0 tx=0x7f4a5000c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:01.162 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.141+0000 7f4a61600640 1 -- 172.21.15.145:0/3593675094 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a50017070 con 0x7f4a640b6510 2024-05-09T04:34:01.162 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.141+0000 7f4a61600640 1 -- 172.21.15.145:0/3593675094 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a50005ce0 con 0x7f4a640b6510 2024-05-09T04:34:01.162 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.141+0000 7f4a61600640 1 -- 172.21.15.145:0/3593675094 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a50007270 con 0x7f4a640b6510 2024-05-09T04:34:01.163 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.141+0000 7f99e2000640 1 Processor -- start 2024-05-09T04:34:01.163 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.141+0000 7f3d1ea00640 1 --2- 172.21.15.145:0/3291087378 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f3cfc076320 0x7f3cfc0787e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3d100012e0 tx=0x7f3d10009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:01.163 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.141+0000 7f99e2000640 1 -- start start 2024-05-09T04:34:01.163 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.142+0000 7f4a6a800640 1 -- 172.21.15.145:0/3593675094 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a64164f20 con 0x7f4a640b6510 2024-05-09T04:34:01.164 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.142+0000 7f4a6a800640 1 -- 172.21.15.145:0/3593675094 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a641654e0 con 0x7f4a640b6510 2024-05-09T04:34:01.164 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.143+0000 7f8a3a000640 1 Processor -- start 2024-05-09T04:34:01.164 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.143+0000 7f99e2000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99dc15eec0 0x7f99dc151df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:01.164 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.143+0000 7f99e2000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99dc1602b0 0x7f99dc152940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:01.164 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.143+0000 7f99e2000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99dc162e00 0x7f99dc0b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:01.164 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.143+0000 7f4a6a800640 1 -- 172.21.15.145:0/3593675094 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a640b8960 con 0x7f4a640b6510 2024-05-09T04:34:01.165 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.143+0000 7f99e0c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99dc15eec0 0x7f99dc151df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:01.165 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.143+0000 7f99e0c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99dc15eec0 0x7f99dc151df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.145:42648/0 (socket says 172.21.15.145:42648) 2024-05-09T04:34:01.165 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.144+0000 7f99dbe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99dc1602b0 0x7f99dc152940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:01.165 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.144+0000 7f99e1600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99dc162e00 0x7f99dc0b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:01.165 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.144+0000 7f99dbe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99dc1602b0 0x7f99dc152940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.145:49966/0 (socket says 172.21.15.145:49966) 2024-05-09T04:34:01.165 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.144+0000 7f99e1600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99dc162e00 0x7f99dc0b4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:37780/0 (socket says 172.21.15.145:37780) 2024-05-09T04:34:01.165 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.144+0000 7f99e0c00640 1 -- 172.21.15.145:0/2127639902 learned_addr learned my addr 172.21.15.145:0/2127639902 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:01.166 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.144+0000 7f99e2000640 1 -- 172.21.15.145:0/2127639902 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f99dc06ed00 con 0x7f99dc162e00 2024-05-09T04:34:01.166 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.144+0000 7f99e2000640 1 -- 172.21.15.145:0/2127639902 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f99dc163290 con 0x7f99dc1602b0 2024-05-09T04:34:01.166 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.144+0000 7f99e2000640 1 -- 172.21.15.145:0/2127639902 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f99dc06ee80 con 0x7f99dc15eec0 2024-05-09T04:34:01.166 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.144+0000 7f8a3a000640 1 -- start start 2024-05-09T04:34:01.166 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.144+0000 7f4a61600640 1 -- 172.21.15.145:0/3593675094 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4a5001a070 con 0x7f4a640b6510 2024-05-09T04:34:01.167 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.145+0000 7f99e0c00640 1 -- 172.21.15.145:0/2127639902 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99dc1602b0 msgr2=0x7f99dc152940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:01.167 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.145+0000 7f99e0c00640 1 --2- 172.21.15.145:0/2127639902 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99dc1602b0 0x7f99dc152940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.167 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.145+0000 7f99e0c00640 1 -- 172.21.15.145:0/2127639902 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99dc162e00 msgr2=0x7f99dc0b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:01.167 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.145+0000 7f99e0c00640 1 --2- 172.21.15.145:0/2127639902 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99dc162e00 0x7f99dc0b4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.167 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.145+0000 7f99e0c00640 1 -- 172.21.15.145:0/2127639902 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99dc09d120 con 0x7f99dc15eec0 2024-05-09T04:34:01.167 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.145+0000 7f99e1600640 1 --2- 172.21.15.145:0/2127639902 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99dc162e00 0x7f99dc0b4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:34:01.167 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.145+0000 7f99e0c00640 1 --2- 172.21.15.145:0/2127639902 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99dc15eec0 0x7f99dc151df0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f99d400cbf0 tx=0x7f99d400a430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:01.168 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.146+0000 7f8a3a000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8a2c03c0a0 0x7f8a2c0d5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:01.168 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.146+0000 7f8a3a000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a2c053cb0 0x7f8a2c0e10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:01.168 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.146+0000 7f8a3a000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8a2c0e3160 0x7f8a2c0e1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:01.168 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.146+0000 7f8a3a000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8a2c0e3540 con 0x7f8a2c053cb0 2024-05-09T04:34:01.169 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.146+0000 7f8a3a000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8a2c006640 con 0x7f8a2c03c0a0 2024-05-09T04:34:01.169 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.146+0000 7f8a3a000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8a2c056e10 con 0x7f8a2c0e3160 2024-05-09T04:34:01.169 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.146+0000 7f8a38c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8a2c03c0a0 0x7f8a2c0d5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:01.169 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.146+0000 7f8a38c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8a2c03c0a0 0x7f8a2c0d5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.145:49974/0 (socket says 172.21.15.145:49974) 2024-05-09T04:34:01.169 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.146+0000 7f8a38c00640 1 -- 172.21.15.145:0/41765161 learned_addr learned my addr 172.21.15.145:0/41765161 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:01.169 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.146+0000 7f8a33e00640 1 --2- 172.21.15.145:0/41765161 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a2c053cb0 0x7f8a2c0e10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:01.169 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.147+0000 7f8a39600640 1 --2- 172.21.15.145:0/41765161 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8a2c0e3160 0x7f8a2c0e1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:01.169 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.147+0000 7f8a33e00640 1 -- 172.21.15.145:0/41765161 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8a2c0e3160 msgr2=0x7f8a2c0e1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:01.169 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.147+0000 7f8a33e00640 1 --2- 172.21.15.145:0/41765161 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8a2c0e3160 0x7f8a2c0e1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.169 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.147+0000 7f4a61600640 1 --2- 172.21.15.145:0/3593675094 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f4a44076350 0x7f4a44078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:01.169 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.147+0000 7f4a61600640 1 -- 172.21.15.145:0/3593675094 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4a500967a0 con 0x7f4a640b6510 2024-05-09T04:34:01.170 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.147+0000 7f8a33e00640 1 -- 172.21.15.145:0/41765161 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8a2c03c0a0 msgr2=0x7f8a2c0d5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:01.170 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.147+0000 7f8a33e00640 1 --2- 172.21.15.145:0/41765161 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8a2c03c0a0 0x7f8a2c0d5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.170 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.147+0000 7f8a33e00640 1 -- 172.21.15.145:0/41765161 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a2c0daa80 con 0x7f8a2c053cb0 2024-05-09T04:34:01.170 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.147+0000 7f8a38c00640 1 --2- 172.21.15.145:0/41765161 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8a2c03c0a0 0x7f8a2c0d5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:34:01.170 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.147+0000 7f8a33e00640 1 --2- 172.21.15.145:0/41765161 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a2c053cb0 0x7f8a2c0e10f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8a2000ffc0 tx=0x7f8a200134e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:01.170 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.148+0000 7f4a69400640 1 --2- 172.21.15.145:0/3593675094 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f4a44076350 0x7f4a44078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:01.171 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.149+0000 7f4a69400640 1 --2- 172.21.15.145:0/3593675094 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f4a44076350 0x7f4a44078810 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4a58001eb0 tx=0x7f4a58009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:01.171 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.148+0000 7f6e1ea00640 1 -- 172.21.15.145:0/2050724596 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e18037050 con 0x7f6e280b5930 2024-05-09T04:34:01.171 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.149+0000 7f6e1ea00640 1 -- 172.21.15.145:0/2050724596 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e18043070 con 0x7f6e280b5930 2024-05-09T04:34:01.171 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.149+0000 7f6e2d800640 1 -- 172.21.15.145:0/2050724596 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e280b8770 con 0x7f6e280b5930 2024-05-09T04:34:01.171 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.149+0000 7f6e2d800640 1 -- 172.21.15.145:0/2050724596 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e280b8d30 con 0x7f6e280b5930 2024-05-09T04:34:01.171 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.149+0000 7f6e1ea00640 1 -- 172.21.15.145:0/2050724596 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e18030bb0 con 0x7f6e280b5930 2024-05-09T04:34:01.171 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.149+0000 7f0eab400640 1 -- 172.21.15.145:0/1989200502 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0ea4045070 con 0x7f0eb406ced0 2024-05-09T04:34:01.172 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.151+0000 7f6e2d800640 1 -- 172.21.15.145:0/2050724596 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e00005290 con 0x7f6e280b5930 2024-05-09T04:34:01.172 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.152+0000 7f6e1ea00640 1 -- 172.21.15.145:0/2050724596 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6e18048460 con 0x7f6e280b5930 2024-05-09T04:34:01.172 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.153+0000 7f6e1ea00640 1 --2- 172.21.15.145:0/2050724596 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f6e040762d0 0x7f6e04078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:01.172 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.154+0000 7f8a30c00640 1 -- 172.21.15.145:0/41765161 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a20022070 con 0x7f8a2c053cb0 2024-05-09T04:34:01.172 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.154+0000 7f6e1ea00640 1 -- 172.21.15.145:0/2050724596 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6e180b8290 con 0x7f6e280b5930 2024-05-09T04:34:01.173 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.154+0000 7f8a30c00640 1 -- 172.21.15.145:0/41765161 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a2001d400 con 0x7f8a2c053cb0 2024-05-09T04:34:01.173 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.154+0000 7f8a30c00640 1 -- 172.21.15.145:0/41765161 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2001d7f0 con 0x7f8a2c053cb0 2024-05-09T04:34:01.173 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.154+0000 7f99d8c00640 1 -- 172.21.15.145:0/2127639902 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d4017070 con 0x7f99dc15eec0 2024-05-09T04:34:01.173 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.154+0000 7f99d8c00640 1 -- 172.21.15.145:0/2127639902 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99d400ad40 con 0x7f99dc15eec0 2024-05-09T04:34:01.173 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.154+0000 7f99d8c00640 1 -- 172.21.15.145:0/2127639902 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d4012600 con 0x7f99dc15eec0 2024-05-09T04:34:01.173 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.154+0000 7f99e2000640 1 -- 172.21.15.145:0/2127639902 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99dc09d400 con 0x7f99dc15eec0 2024-05-09T04:34:01.173 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.154+0000 7f99e2000640 1 -- 172.21.15.145:0/2127639902 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f99dc09d9c0 con 0x7f99dc15eec0 2024-05-09T04:34:01.174 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.154+0000 7f8a3a000640 1 -- 172.21.15.145:0/41765161 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a2c0dad60 con 0x7f8a2c053cb0 2024-05-09T04:34:01.174 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.155+0000 7f8a3a000640 1 -- 172.21.15.145:0/41765161 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a2c0db320 con 0x7f8a2c053cb0 2024-05-09T04:34:01.174 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.155+0000 7f6e26000640 1 --2- 172.21.15.145:0/2050724596 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f6e040762d0 0x7f6e04078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:01.174 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.155+0000 7f4a61600640 1 -- 172.21.15.145:0/3593675094 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f4a5005fb90 con 0x7f4a640b6510 2024-05-09T04:34:01.175 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.156+0000 7f3d17400640 1 -- 172.21.15.145:0/3291087378 <== mon.2 v2:172.21.15.192:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3d200b8390 con 0x7f3d200d2120 2024-05-09T04:34:01.175 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.157+0000 7f6e26000640 1 --2- 172.21.15.145:0/2050724596 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f6e040762d0 0x7f6e04078790 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6e280cdd20 tx=0x7f6e10009110 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:01.175 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.157+0000 7f8a3a000640 1 -- 172.21.15.145:0/41765161 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a2c005390 con 0x7f8a2c053cb0 2024-05-09T04:34:01.175 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.158+0000 7f8a30c00640 1 -- 172.21.15.145:0/41765161 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8a20025070 con 0x7f8a2c053cb0 2024-05-09T04:34:01.176 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.157+0000 7f99e2000640 1 -- 172.21.15.145:0/2127639902 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99b4005290 con 0x7f99dc15eec0 2024-05-09T04:34:01.176 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.158+0000 7f99d8c00640 1 -- 172.21.15.145:0/2127639902 <== mon.2 v2:172.21.15.192:3301/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f99d401a070 con 0x7f99dc15eec0 2024-05-09T04:34:01.176 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.159+0000 7f99d8c00640 1 --2- 172.21.15.145:0/2127639902 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f99bc076320 0x7f99bc0787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:01.176 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.160+0000 7f8a30c00640 1 --2- 172.21.15.145:0/41765161 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f8a14076320 0x7f8a140787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:01.176 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.160+0000 7f8a30c00640 1 -- 172.21.15.145:0/41765161 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8a2009f630 con 0x7f8a2c053cb0 2024-05-09T04:34:01.176 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.159+0000 7f99d8c00640 1 -- 172.21.15.145:0/2127639902 <== mon.2 v2:172.21.15.192:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f99d40972e0 con 0x7f99dc15eec0 2024-05-09T04:34:01.177 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.161+0000 7f8a38c00640 1 --2- 172.21.15.145:0/41765161 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f8a14076320 0x7f8a140787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:01.177 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.161+0000 7f99dbe00640 1 --2- 172.21.15.145:0/2127639902 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f99bc076320 0x7f99bc0787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:01.177 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.162+0000 7f8a38c00640 1 --2- 172.21.15.145:0/41765161 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f8a14076320 0x7f8a140787e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8a28035040 tx=0x7f8a2802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:01.177 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.163+0000 7f99dbe00640 1 --2- 172.21.15.145:0/2127639902 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f99bc076320 0x7f99bc0787e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f99dc0ce080 tx=0x7f99c8009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:01.178 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.168+0000 7f6e1ea00640 1 -- 172.21.15.145:0/2050724596 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6e18081be0 con 0x7f6e280b5930 2024-05-09T04:34:01.178 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.168+0000 7f8a30c00640 1 -- 172.21.15.145:0/41765161 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8a20018070 con 0x7f8a2c053cb0 2024-05-09T04:34:01.178 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.172+0000 7f99d8c00640 1 -- 172.21.15.145:0/2127639902 <== mon.2 v2:172.21.15.192:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f99d40605d0 con 0x7f99dc15eec0 2024-05-09T04:34:01.542 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.542+0000 7f698bc00640 1 -- 172.21.15.145:0/3835493073 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f697c0382e0 con 0x7f697c04fa80 2024-05-09T04:34:01.544 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.544+0000 7f6980c00640 1 -- 172.21.15.145:0/3835493073 <== mon.2 v2:172.21.15.192:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6970064db0 con 0x7f697c04fa80 2024-05-09T04:34:01.545 INFO:teuthology.orchestra.run.smithi145.stdout:55834574853 2024-05-09T04:34:01.548 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.548+0000 7f698bc00640 1 -- 172.21.15.145:0/3835493073 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f69640762d0 msgr2=0x7f6964078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:01.548 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.548+0000 7f698bc00640 1 --2- 172.21.15.145:0/3835493073 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f69640762d0 0x7f6964078790 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f697800a520 tx=0x7f6978008040 comp rx=0 tx=0).stop 2024-05-09T04:34:01.549 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.549+0000 7f698bc00640 1 -- 172.21.15.145:0/3835493073 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f697c04fa80 msgr2=0x7f697c0cff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:01.549 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.549+0000 7f698bc00640 1 --2- 172.21.15.145:0/3835493073 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f697c04fa80 0x7f697c0cff00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f697000c7a0 tx=0x7f697000cc60 comp rx=0 tx=0).stop 2024-05-09T04:34:01.549 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.549+0000 7f698bc00640 1 -- 172.21.15.145:0/3835493073 shutdown_connections 2024-05-09T04:34:01.549 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.549+0000 7f698bc00640 1 --2- 172.21.15.145:0/3835493073 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f69640762d0 0x7f6964078790 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.550 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.549+0000 7f698bc00640 1 --2- 172.21.15.145:0/3835493073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f697c0df410 0x7f697c039990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.550 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.550+0000 7f698bc00640 1 --2- 172.21.15.145:0/3835493073 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f697c04fa80 0x7f697c0cff00 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.550 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.550+0000 7f698bc00640 1 --2- 172.21.15.145:0/3835493073 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f697c0387f0 0x7f697c0e1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.550 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.550+0000 7f698bc00640 1 -- 172.21.15.145:0/3835493073 >> 172.21.15.145:0/3835493073 conn(0x7f697c009fa0 msgr2=0x7f697c006100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:01.550 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.551+0000 7f698bc00640 1 -- 172.21.15.145:0/3835493073 shutdown_connections 2024-05-09T04:34:01.550 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.551+0000 7f698bc00640 1 -- 172.21.15.145:0/3835493073 wait complete. 2024-05-09T04:34:01.574 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.574+0000 7faaab800640 1 -- 172.21.15.145:0/2512685809 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7faaa406a310 con 0x7faaa4073420 2024-05-09T04:34:01.575 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.575+0000 7faaa0c00640 1 -- 172.21.15.145:0/2512685809 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7faa940811b0 con 0x7faaa4073420 2024-05-09T04:34:01.578 INFO:teuthology.orchestra.run.smithi145.stdout:55834574854 2024-05-09T04:34:01.579 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.5 2024-05-09T04:34:01.579 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.579+0000 7faaab800640 1 -- 172.21.15.145:0/2512685809 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7faa840763d0 msgr2=0x7faa84078890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:01.579 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.579+0000 7faaab800640 1 --2- 172.21.15.145:0/2512685809 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7faa840763d0 0x7faa84078890 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7faa9c0015a0 tx=0x7faa9c0015f0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.579 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.579+0000 7faaab800640 1 -- 172.21.15.145:0/2512685809 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faaa4073420 msgr2=0x7faaa4152e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:01.579 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.579+0000 7faaab800640 1 --2- 172.21.15.145:0/2512685809 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faaa4073420 0x7faaa4152e30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7faa940097a0 tx=0x7faa940046c0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.579 DEBUG:teuthology.parallel:result is None 2024-05-09T04:34:01.580 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.580+0000 7faaab800640 1 -- 172.21.15.145:0/2512685809 shutdown_connections 2024-05-09T04:34:01.580 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.580+0000 7faaab800640 1 --2- 172.21.15.145:0/2512685809 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7faa840763d0 0x7faa84078890 secure :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7faa9c0015a0 tx=0x7faa9c0015f0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.580 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.580+0000 7faaab800640 1 --2- 172.21.15.145:0/2512685809 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faaa415b750 0x7faaa4155ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.580 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.580+0000 7faaab800640 1 --2- 172.21.15.145:0/2512685809 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faaa415b370 0x7faaa41e4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.580 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.580+0000 7faaab800640 1 --2- 172.21.15.145:0/2512685809 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faaa4073420 0x7faaa4152e30 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.580 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.580+0000 7faaab800640 1 -- 172.21.15.145:0/2512685809 >> 172.21.15.145:0/2512685809 conn(0x7faaa40b2c00 msgr2=0x7faaa40cd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:01.580 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.580+0000 7faaab800640 1 -- 172.21.15.145:0/2512685809 shutdown_connections 2024-05-09T04:34:01.580 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.581+0000 7faaab800640 1 -- 172.21.15.145:0/2512685809 wait complete. 2024-05-09T04:34:01.584 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.585+0000 7f0eb8a00640 1 -- 172.21.15.145:0/1989200502 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f0eb40cfec0 con 0x7f0eb406ced0 2024-05-09T04:34:01.585 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.586+0000 7f0eab400640 1 -- 172.21.15.145:0/1989200502 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0ea408a0d0 con 0x7f0eb406ced0 2024-05-09T04:34:01.585 INFO:teuthology.orchestra.run.smithi145.stdout:55834574855 2024-05-09T04:34:01.587 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.588+0000 7f0eb8a00640 1 -- 172.21.15.145:0/1989200502 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f0e94076300 msgr2=0x7f0e940787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:01.588 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.588+0000 7f0eb8a00640 1 --2- 172.21.15.145:0/1989200502 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f0e94076300 0x7f0e940787c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0eb41f44a0 tx=0x7f0ea0009040 comp rx=0 tx=0).stop 2024-05-09T04:34:01.588 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.588+0000 7f0eb8a00640 1 -- 172.21.15.145:0/1989200502 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0eb406ced0 msgr2=0x7f0eb4153cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:01.588 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.588+0000 7f0eb8a00640 1 --2- 172.21.15.145:0/1989200502 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0eb406ced0 0x7f0eb4153cb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4000c00 tx=0x7f0ea40046c0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.588 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.588+0000 7f0eb8a00640 1 -- 172.21.15.145:0/1989200502 shutdown_connections 2024-05-09T04:34:01.588 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.588+0000 7f0eb8a00640 1 --2- 172.21.15.145:0/1989200502 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f0e94076300 0x7f0e940787c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.588 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.588+0000 7f0eb8a00640 1 --2- 172.21.15.145:0/1989200502 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0eb41558a0 0x7f0eb415ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.588 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.588+0000 7f0eb8a00640 1 --2- 172.21.15.145:0/1989200502 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0eb4070230 0x7f0eb415b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.589 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.588+0000 7f0eb8a00640 1 --2- 172.21.15.145:0/1989200502 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0eb406ced0 0x7f0eb4153cb0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.589 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.588+0000 7f0eb8a00640 1 -- 172.21.15.145:0/1989200502 >> 172.21.15.145:0/1989200502 conn(0x7f0eb40b2d80 msgr2=0x7f0eb40afd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:01.589 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.589+0000 7f0eb8a00640 1 -- 172.21.15.145:0/1989200502 shutdown_connections 2024-05-09T04:34:01.589 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.589+0000 7f0eb8a00640 1 -- 172.21.15.145:0/1989200502 wait complete. 2024-05-09T04:34:01.604 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.604+0000 7f4a6a800640 1 -- 172.21.15.145:0/3593675094 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f4a64076780 con 0x7f4a640b6510 2024-05-09T04:34:01.604 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.604+0000 7f4a61600640 1 -- 172.21.15.145:0/3593675094 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4a50063e90 con 0x7f4a640b6510 2024-05-09T04:34:01.604 INFO:teuthology.orchestra.run.smithi145.stdout:55834574856 2024-05-09T04:34:01.607 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.0 2024-05-09T04:34:01.607 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.607+0000 7f4a6a800640 1 -- 172.21.15.145:0/3593675094 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f4a44076350 msgr2=0x7f4a44078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:01.607 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.607+0000 7f4a6a800640 1 --2- 172.21.15.145:0/3593675094 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f4a44076350 0x7f4a44078810 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4a58001eb0 tx=0x7f4a58009040 comp rx=0 tx=0).stop 2024-05-09T04:34:01.607 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.607+0000 7f4a6a800640 1 -- 172.21.15.145:0/3593675094 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4a640b6510 msgr2=0x7f4a641fc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:01.607 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.607+0000 7f4a6a800640 1 --2- 172.21.15.145:0/3593675094 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4a640b6510 0x7f4a641fc410 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4a5000c4e0 tx=0x7f4a5000c9a0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.607 DEBUG:teuthology.parallel:result is None 2024-05-09T04:34:01.608 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.607+0000 7f4a6a800640 1 -- 172.21.15.145:0/3593675094 shutdown_connections 2024-05-09T04:34:01.608 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.607+0000 7f4a6a800640 1 --2- 172.21.15.145:0/3593675094 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f4a44076350 0x7f4a44078810 secure :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4a58001eb0 tx=0x7f4a58009040 comp rx=0 tx=0).stop 2024-05-09T04:34:01.608 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.607+0000 7f4a6a800640 1 --2- 172.21.15.145:0/3593675094 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a640b7d70 0x7f4a640c8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.608 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.607+0000 7f4a6a800640 1 --2- 172.21.15.145:0/3593675094 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4a640b6510 0x7f4a641fc410 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.608 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.607+0000 7f4a6a800640 1 --2- 172.21.15.145:0/3593675094 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4a4c0041d0 0x7f4a641f95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.608 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.607+0000 7f4a6a800640 1 -- 172.21.15.145:0/3593675094 >> 172.21.15.145:0/3593675094 conn(0x7f4a640d2d80 msgr2=0x7f4a640c5d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:01.609 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.2 2024-05-09T04:34:01.609 DEBUG:teuthology.parallel:result is None 2024-05-09T04:34:01.609 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.608+0000 7f4a6a800640 1 -- 172.21.15.145:0/3593675094 shutdown_connections 2024-05-09T04:34:01.609 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.608+0000 7f4a6a800640 1 -- 172.21.15.145:0/3593675094 wait complete. 2024-05-09T04:34:01.610 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.608+0000 7f6e2d800640 1 -- 172.21.15.145:0/2050724596 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f6e00005580 con 0x7f6e280b5930 2024-05-09T04:34:01.610 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.609+0000 7f6e1ea00640 1 -- 172.21.15.145:0/2050724596 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6e180814e0 con 0x7f6e280b5930 2024-05-09T04:34:01.610 INFO:teuthology.orchestra.run.smithi145.stdout:55834574855 2024-05-09T04:34:01.610 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.610+0000 7f3d25e00640 1 -- 172.21.15.145:0/3291087378 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f3d2006a310 con 0x7f3d200d2120 2024-05-09T04:34:01.610 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.611+0000 7f3d17400640 1 -- 172.21.15.145:0/3291087378 <== mon.2 v2:172.21.15.192:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3d080602d0 con 0x7f3d200d2120 2024-05-09T04:34:01.611 INFO:teuthology.orchestra.run.smithi145.stdout:55834574854 2024-05-09T04:34:01.611 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.611+0000 7f6e2d800640 1 -- 172.21.15.145:0/2050724596 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f6e040762d0 msgr2=0x7f6e04078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:01.611 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.611+0000 7f6e2d800640 1 --2- 172.21.15.145:0/2050724596 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f6e040762d0 0x7f6e04078790 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6e280cdd20 tx=0x7f6e10009110 comp rx=0 tx=0).stop 2024-05-09T04:34:01.611 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.611+0000 7f6e2d800640 1 -- 172.21.15.145:0/2050724596 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6e280b5930 msgr2=0x7f6e280b7e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:01.611 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.611+0000 7f6e2d800640 1 --2- 172.21.15.145:0/2050724596 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6e280b5930 0x7f6e280b7e00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6e180090b0 tx=0x7f6e180046c0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.612 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.613+0000 7f6e2d800640 1 -- 172.21.15.145:0/2050724596 shutdown_connections 2024-05-09T04:34:01.612 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.613+0000 7f6e2d800640 1 --2- 172.21.15.145:0/2050724596 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f6e040762d0 0x7f6e04078790 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.612 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.613+0000 7f6e2d800640 1 --2- 172.21.15.145:0/2050724596 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6e2815f5f0 0x7f6e2814d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.613 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.613+0000 7f6e2d800640 1 --2- 172.21.15.145:0/2050724596 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6e280d2db0 0x7f6e280cecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.613 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.613+0000 7f6e2d800640 1 --2- 172.21.15.145:0/2050724596 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6e280b5930 0x7f6e280b7e00 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.613 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.613+0000 7f6e2d800640 1 -- 172.21.15.145:0/2050724596 >> 172.21.15.145:0/2050724596 conn(0x7f6e2809e520 msgr2=0x7f6e280d0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:01.613 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.613+0000 7f6e2d800640 1 -- 172.21.15.145:0/2050724596 shutdown_connections 2024-05-09T04:34:01.613 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.613+0000 7f6e2d800640 1 -- 172.21.15.145:0/2050724596 wait complete. 2024-05-09T04:34:01.614 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.613+0000 7f3d25e00640 1 -- 172.21.15.145:0/3291087378 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f3cfc076320 msgr2=0x7f3cfc0787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:01.614 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.613+0000 7f3d25e00640 1 --2- 172.21.15.145:0/3291087378 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f3cfc076320 0x7f3cfc0787e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3d100012e0 tx=0x7f3d10009040 comp rx=0 tx=0).stop 2024-05-09T04:34:01.614 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.613+0000 7f3d25e00640 1 -- 172.21.15.145:0/3291087378 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3d200d2120 msgr2=0x7f3d200b7d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:01.614 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.613+0000 7f3d25e00640 1 --2- 172.21.15.145:0/3291087378 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3d200d2120 0x7f3d200b7d30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3d0800b4a0 tx=0x7f3d0800b960 comp rx=0 tx=0).stop 2024-05-09T04:34:01.614 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.613+0000 7f3d25e00640 1 -- 172.21.15.145:0/3291087378 shutdown_connections 2024-05-09T04:34:01.614 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.613+0000 7f3d25e00640 1 --2- 172.21.15.145:0/3291087378 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f3cfc076320 0x7f3cfc0787e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.614 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.613+0000 7f3d25e00640 1 --2- 172.21.15.145:0/3291087378 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3d20162c00 0x7f3d200b9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.614 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.613+0000 7f3d25e00640 1 --2- 172.21.15.145:0/3291087378 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3d200d2120 0x7f3d200b7d30 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.614 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.613+0000 7f3d25e00640 1 --2- 172.21.15.145:0/3291087378 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d200b5fe0 0x7f3d200ce3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.614 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.613+0000 7f3d25e00640 1 -- 172.21.15.145:0/3291087378 >> 172.21.15.145:0/3291087378 conn(0x7f3d2009e510 msgr2=0x7f3d200d0870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:01.614 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.614+0000 7f3d25e00640 1 -- 172.21.15.145:0/3291087378 shutdown_connections 2024-05-09T04:34:01.615 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.614+0000 7f3d25e00640 1 -- 172.21.15.145:0/3291087378 wait complete. 2024-05-09T04:34:01.619 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.619+0000 7f99e2000640 1 -- 172.21.15.145:0/2127639902 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f99b4005580 con 0x7f99dc15eec0 2024-05-09T04:34:01.619 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.620+0000 7f8a3a000640 1 -- 172.21.15.145:0/41765161 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f8a2c056bf0 con 0x7f8a2c053cb0 2024-05-09T04:34:01.619 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.620+0000 7f99d8c00640 1 -- 172.21.15.145:0/2127639902 <== mon.2 v2:172.21.15.192:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f99d40648d0 con 0x7f99dc15eec0 2024-05-09T04:34:01.622 INFO:teuthology.orchestra.run.smithi145.stdout:55834574854 2024-05-09T04:34:01.622 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.620+0000 7f8a30c00640 1 -- 172.21.15.145:0/41765161 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8a200689b0 con 0x7f8a2c053cb0 2024-05-09T04:34:01.623 INFO:teuthology.orchestra.run.smithi145.stdout:55834574853 2024-05-09T04:34:01.623 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.623+0000 7f99e2000640 1 -- 172.21.15.145:0/2127639902 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f99bc076320 msgr2=0x7f99bc0787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:01.623 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.623+0000 7f99e2000640 1 --2- 172.21.15.145:0/2127639902 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f99bc076320 0x7f99bc0787e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f99dc0ce080 tx=0x7f99c8009190 comp rx=0 tx=0).stop 2024-05-09T04:34:01.624 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.623+0000 7f99e2000640 1 -- 172.21.15.145:0/2127639902 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99dc15eec0 msgr2=0x7f99dc151df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:01.624 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.623+0000 7f99e2000640 1 --2- 172.21.15.145:0/2127639902 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99dc15eec0 0x7f99dc151df0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f99d400cbf0 tx=0x7f99d400a430 comp rx=0 tx=0).stop 2024-05-09T04:34:01.624 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.623+0000 7f99e2000640 1 -- 172.21.15.145:0/2127639902 shutdown_connections 2024-05-09T04:34:01.624 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.623+0000 7f99e2000640 1 --2- 172.21.15.145:0/2127639902 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f99bc076320 0x7f99bc0787e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.624 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.623+0000 7f99e2000640 1 --2- 172.21.15.145:0/2127639902 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99dc162e00 0x7f99dc0b4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.624 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.623+0000 7f99e2000640 1 --2- 172.21.15.145:0/2127639902 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99dc1602b0 0x7f99dc152940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.624 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.623+0000 7f99e2000640 1 --2- 172.21.15.145:0/2127639902 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99dc15eec0 0x7f99dc151df0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.624 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.623+0000 7f99e2000640 1 -- 172.21.15.145:0/2127639902 >> 172.21.15.145:0/2127639902 conn(0x7f99dc0b2960 msgr2=0x7f99dc0b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:01.624 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.623+0000 7f99e2000640 1 -- 172.21.15.145:0/2127639902 shutdown_connections 2024-05-09T04:34:01.624 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.623+0000 7f99e2000640 1 -- 172.21.15.145:0/2127639902 wait complete. 2024-05-09T04:34:01.625 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.624+0000 7f8a3a000640 1 -- 172.21.15.145:0/41765161 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f8a14076320 msgr2=0x7f8a140787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:01.625 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.624+0000 7f8a3a000640 1 --2- 172.21.15.145:0/41765161 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f8a14076320 0x7f8a140787e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8a28035040 tx=0x7f8a2802e000 comp rx=0 tx=0).stop 2024-05-09T04:34:01.625 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.624+0000 7f8a3a000640 1 -- 172.21.15.145:0/41765161 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a2c053cb0 msgr2=0x7f8a2c0e10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:01.625 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.624+0000 7f8a3a000640 1 --2- 172.21.15.145:0/41765161 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a2c053cb0 0x7f8a2c0e10f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8a2000ffc0 tx=0x7f8a200134e0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.625 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.624+0000 7f8a3a000640 1 -- 172.21.15.145:0/41765161 shutdown_connections 2024-05-09T04:34:01.625 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.624+0000 7f8a3a000640 1 --2- 172.21.15.145:0/41765161 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f8a14076320 0x7f8a140787e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.625 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.624+0000 7f8a3a000640 1 --2- 172.21.15.145:0/41765161 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8a2c0e3160 0x7f8a2c0e1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.625 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.624+0000 7f8a3a000640 1 --2- 172.21.15.145:0/41765161 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a2c053cb0 0x7f8a2c0e10f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.625 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.624+0000 7f8a3a000640 1 --2- 172.21.15.145:0/41765161 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8a2c03c0a0 0x7f8a2c0d5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.625 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.624+0000 7f8a3a000640 1 -- 172.21.15.145:0/41765161 >> 172.21.15.145:0/41765161 conn(0x7f8a2c058ee0 msgr2=0x7f8a2c038e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:01.626 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.624+0000 7f8a3a000640 1 -- 172.21.15.145:0/41765161 shutdown_connections 2024-05-09T04:34:01.626 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.625+0000 7f8a3a000640 1 -- 172.21.15.145:0/41765161 wait complete. 2024-05-09T04:34:01.633 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.6 2024-05-09T04:34:01.634 DEBUG:teuthology.parallel:result is None 2024-05-09T04:34:01.634 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.7 2024-05-09T04:34:01.634 DEBUG:teuthology.parallel:result is None 2024-05-09T04:34:01.635 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.3 2024-05-09T04:34:01.635 DEBUG:teuthology.parallel:result is None 2024-05-09T04:34:01.643 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.1 2024-05-09T04:34:01.643 DEBUG:teuthology.parallel:result is None 2024-05-09T04:34:01.644 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.4 2024-05-09T04:34:01.644 DEBUG:teuthology.parallel:result is None 2024-05-09T04:34:01.644 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-09T04:34:01.644 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T04:34:01.874 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.874+0000 7f910b200640 1 Processor -- start 2024-05-09T04:34:01.874 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.874+0000 7f910b200640 1 -- start start 2024-05-09T04:34:01.876 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.875+0000 7f910b200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f910c1572c0 0x7f910c15deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:01.876 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.875+0000 7f910b200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f910c0ae710 0x7f910c161220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:01.876 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.875+0000 7f910b200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f910c0cff30 0x7f910c0d0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:01.876 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.875+0000 7f910b200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f910c0b6290 con 0x7f910c1572c0 2024-05-09T04:34:01.876 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.876+0000 7f910b200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f910c0d08c0 con 0x7f910c0cff30 2024-05-09T04:34:01.877 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.876+0000 7f910b200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f910c0aecd0 con 0x7f910c0ae710 2024-05-09T04:34:01.877 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.876+0000 7f9109e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f910c1572c0 0x7f910c15deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:01.877 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.876+0000 7f9109e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f910c1572c0 0x7f910c15deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:37792/0 (socket says 172.21.15.145:37792) 2024-05-09T04:34:01.877 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.876+0000 7f9109e00640 1 -- 172.21.15.145:0/3679397776 learned_addr learned my addr 172.21.15.145:0/3679397776 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:01.877 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.876+0000 7f910a800640 1 --2- 172.21.15.145:0/3679397776 >> v2:172.21.15.192:3300/0 conn(0x7f910c0cff30 0x7f910c0d0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:01.877 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.876+0000 7f9101400640 1 --2- 172.21.15.145:0/3679397776 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f910c0ae710 0x7f910c161220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:01.877 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.876+0000 7f9109e00640 1 -- 172.21.15.145:0/3679397776 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f910c0ae710 msgr2=0x7f910c161220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:01.877 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.876+0000 7f9109e00640 1 --2- 172.21.15.145:0/3679397776 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f910c0ae710 0x7f910c161220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.877 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.876+0000 7f9109e00640 1 -- 172.21.15.145:0/3679397776 >> v2:172.21.15.192:3300/0 conn(0x7f910c0cff30 msgr2=0x7f910c0d0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:01.877 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.876+0000 7f9109e00640 1 --2- 172.21.15.145:0/3679397776 >> v2:172.21.15.192:3300/0 conn(0x7f910c0cff30 0x7f910c0d0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.877 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.876+0000 7f9109e00640 1 -- 172.21.15.145:0/3679397776 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f910c0aefb0 con 0x7f910c1572c0 2024-05-09T04:34:01.877 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.876+0000 7f9101400640 1 --2- 172.21.15.145:0/3679397776 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f910c0ae710 0x7f910c161220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:34:01.878 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.877+0000 7f9109e00640 1 --2- 172.21.15.145:0/3679397776 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f910c1572c0 0x7f910c15deb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f910c0744f0 tx=0x7f90fc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44a86ae98765b105 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:01.879 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.878+0000 7f9109400640 1 -- 172.21.15.145:0/3679397776 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90fc037070 con 0x7f910c1572c0 2024-05-09T04:34:01.879 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.878+0000 7f9109400640 1 -- 172.21.15.145:0/3679397776 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90fc032440 con 0x7f910c1572c0 2024-05-09T04:34:01.879 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.878+0000 7f9109400640 1 -- 172.21.15.145:0/3679397776 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90fc032810 con 0x7f910c1572c0 2024-05-09T04:34:01.879 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.878+0000 7f910b200640 1 -- 172.21.15.145:0/3679397776 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f910c1572c0 msgr2=0x7f910c15deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:01.879 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.878+0000 7f910b200640 1 --2- 172.21.15.145:0/3679397776 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f910c1572c0 0x7f910c15deb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f910c0744f0 tx=0x7f90fc02a1e0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.879 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.878+0000 7f910b200640 1 -- 172.21.15.145:0/3679397776 shutdown_connections 2024-05-09T04:34:01.879 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.878+0000 7f910b200640 1 --2- 172.21.15.145:0/3679397776 >> v2:172.21.15.192:3300/0 conn(0x7f910c0cff30 0x7f910c0d0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.879 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.878+0000 7f910b200640 1 --2- 172.21.15.145:0/3679397776 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f910c0ae710 0x7f910c161220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.879 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.878+0000 7f910b200640 1 --2- 172.21.15.145:0/3679397776 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f910c1572c0 0x7f910c15deb0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.879 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.878+0000 7f910b200640 1 -- 172.21.15.145:0/3679397776 >> 172.21.15.145:0/3679397776 conn(0x7f910c06c8a0 msgr2=0x7f910c06ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:01.879 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.878+0000 7f910b200640 1 -- 172.21.15.145:0/3679397776 shutdown_connections 2024-05-09T04:34:01.879 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.878+0000 7f910b200640 1 -- 172.21.15.145:0/3679397776 wait complete. 2024-05-09T04:34:01.880 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.880+0000 7f910b200640 1 Processor -- start 2024-05-09T04:34:01.880 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.880+0000 7f910b200640 1 -- start start 2024-05-09T04:34:01.882 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.881+0000 7f910b200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f910c0ae710 0x7f910c09c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:01.882 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.881+0000 7f910b200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f910c0cff30 0x7f910c0b2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:01.882 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.881+0000 7f910b200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f910c1572c0 0x7f910c09e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:01.882 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.881+0000 7f910b200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f910c160d00 con 0x7f910c0ae710 2024-05-09T04:34:01.882 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.881+0000 7f910b200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f910c160a20 con 0x7f910c0cff30 2024-05-09T04:34:01.882 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.881+0000 7f910b200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f910c161760 con 0x7f910c1572c0 2024-05-09T04:34:01.883 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.881+0000 7f9109e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f910c0ae710 0x7f910c09c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:01.883 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.881+0000 7f9109e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f910c0ae710 0x7f910c09c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:37796/0 (socket says 172.21.15.145:37796) 2024-05-09T04:34:01.883 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.881+0000 7f9109e00640 1 -- 172.21.15.145:0/1517219009 learned_addr learned my addr 172.21.15.145:0/1517219009 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:01.883 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.882+0000 7f9109e00640 1 -- 172.21.15.145:0/1517219009 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f910c1572c0 msgr2=0x7f910c09e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:01.883 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.882+0000 7f9109e00640 1 --2- 172.21.15.145:0/1517219009 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f910c1572c0 0x7f910c09e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.883 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.882+0000 7f9109e00640 1 -- 172.21.15.145:0/1517219009 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f910c0cff30 msgr2=0x7f910c0b2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:01.883 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.882+0000 7f9109e00640 1 --2- 172.21.15.145:0/1517219009 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f910c0cff30 0x7f910c0b2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:01.883 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.882+0000 7f9109e00640 1 -- 172.21.15.145:0/1517219009 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f910c0b31d0 con 0x7f910c0ae710 2024-05-09T04:34:01.883 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.882+0000 7f9109e00640 1 --2- 172.21.15.145:0/1517219009 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f910c0ae710 0x7f910c09c3d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f90fc02a220 tx=0x7f90fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:01.883 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.882+0000 7f9102a00640 1 -- 172.21.15.145:0/1517219009 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90fc041020 con 0x7f910c0ae710 2024-05-09T04:34:01.883 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.882+0000 7f9102a00640 1 -- 172.21.15.145:0/1517219009 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90fc0306c0 con 0x7f910c0ae710 2024-05-09T04:34:01.883 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.882+0000 7f9102a00640 1 -- 172.21.15.145:0/1517219009 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90fc037040 con 0x7f910c0ae710 2024-05-09T04:34:01.883 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.882+0000 7f910b200640 1 -- 172.21.15.145:0/1517219009 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f910c09d3a0 con 0x7f910c0ae710 2024-05-09T04:34:01.884 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.882+0000 7f910b200640 1 -- 172.21.15.145:0/1517219009 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f910c09cde0 con 0x7f910c0ae710 2024-05-09T04:34:01.884 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.883+0000 7f910b200640 1 -- 172.21.15.145:0/1517219009 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f910c06a310 con 0x7f910c0ae710 2024-05-09T04:34:01.884 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.884+0000 7f9102a00640 1 -- 172.21.15.145:0/1517219009 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f90fc044020 con 0x7f910c0ae710 2024-05-09T04:34:01.885 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.885+0000 7f9102a00640 1 --2- 172.21.15.145:0/1517219009 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f90e80762d0 0x7f90e8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:01.885 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.885+0000 7f9102a00640 1 -- 172.21.15.145:0/1517219009 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f90fc0b7df0 con 0x7f910c0ae710 2024-05-09T04:34:01.885 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.886+0000 7f9101400640 1 --2- 172.21.15.145:0/1517219009 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f90e80762d0 0x7f90e8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:01.886 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.886+0000 7f9101400640 1 --2- 172.21.15.145:0/1517219009 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f90e80762d0 0x7f90e8078790 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f90f4006fd0 tx=0x7f90f4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:01.889 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:01.890+0000 7f9102a00640 1 -- 172.21.15.145:0/1517219009 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f90fc04b030 con 0x7f910c0ae710 2024-05-09T04:34:02.078 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.078+0000 7f910b200640 1 -- 172.21.15.145:0/1517219009 --> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f910c161540 con 0x7f90e80762d0 2024-05-09T04:34:02.088 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.088+0000 7f9102a00640 1 -- 172.21.15.145:0/1517219009 <== mgr.4104 v2:172.21.15.145:6832/2725855396 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251825 (secure 0 0 0) 0x7f910c161540 con 0x7f90e80762d0 2024-05-09T04:34:02.089 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:34:02.093 INFO:teuthology.orchestra.run.smithi145.stderr:dumped all 2024-05-09T04:34:02.096 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.093+0000 7f910b200640 1 -- 172.21.15.145:0/1517219009 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f90e80762d0 msgr2=0x7f90e8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:02.096 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.093+0000 7f910b200640 1 --2- 172.21.15.145:0/1517219009 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f90e80762d0 0x7f90e8078790 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f90f4006fd0 tx=0x7f90f4008040 comp rx=0 tx=0).stop 2024-05-09T04:34:02.096 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.093+0000 7f910b200640 1 -- 172.21.15.145:0/1517219009 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f910c0ae710 msgr2=0x7f910c09c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:02.096 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.093+0000 7f910b200640 1 --2- 172.21.15.145:0/1517219009 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f910c0ae710 0x7f910c09c3d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f90fc02a220 tx=0x7f90fc0046c0 comp rx=0 tx=0).stop 2024-05-09T04:34:02.096 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.093+0000 7f910b200640 1 -- 172.21.15.145:0/1517219009 shutdown_connections 2024-05-09T04:34:02.096 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.093+0000 7f910b200640 1 --2- 172.21.15.145:0/1517219009 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f90e80762d0 0x7f90e8078790 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:02.096 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.093+0000 7f910b200640 1 --2- 172.21.15.145:0/1517219009 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f910c1572c0 0x7f910c09e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:02.096 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.093+0000 7f910b200640 1 --2- 172.21.15.145:0/1517219009 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f910c0cff30 0x7f910c0b2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:02.096 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.093+0000 7f910b200640 1 --2- 172.21.15.145:0/1517219009 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f910c0ae710 0x7f910c09c3d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:02.096 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.093+0000 7f910b200640 1 -- 172.21.15.145:0/1517219009 >> 172.21.15.145:0/1517219009 conn(0x7f910c06c8a0 msgr2=0x7f910c06cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:02.097 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.093+0000 7f910b200640 1 -- 172.21.15.145:0/1517219009 shutdown_connections 2024-05-09T04:34:02.097 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.093+0000 7f910b200640 1 -- 172.21.15.145:0/1517219009 wait complete. 2024-05-09T04:34:02.111 INFO:teuthology.orchestra.run.smithi145.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-05-09T04:34:01.759331+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219012,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755708,"statfs":{"total":773094113280,"available":772869844992,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.592729"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.879754+0000","last_change":"2024-05-09T04:33:52.200596+0000","last_active":"2024-05-09T04:33:54.879754+0000","last_peered":"2024-05-09T04:33:54.879754+0000","last_clean":"2024-05-09T04:33:54.879754+0000","last_became_active":"2024-05-09T04:33:52.199781+0000","last_became_peered":"2024-05-09T04:33:52.199781+0000","last_unstale":"2024-05-09T04:33:54.879754+0000","last_undegraded":"2024-05-09T04:33:54.879754+0000","last_fullsized":"2024-05-09T04:33:54.879754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:46:44.064608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.154485+0000","last_change":"2024-05-09T04:33:52.186325+0000","last_active":"2024-05-09T04:33:54.154485+0000","last_peered":"2024-05-09T04:33:54.154485+0000","last_clean":"2024-05-09T04:33:54.154485+0000","last_became_active":"2024-05-09T04:33:52.186047+0000","last_became_peered":"2024-05-09T04:33:52.186047+0000","last_unstale":"2024-05-09T04:33:54.154485+0000","last_undegraded":"2024-05-09T04:33:54.154485+0000","last_fullsized":"2024-05-09T04:33:54.154485+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:24:37.669022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.194578+0000","last_change":"2024-05-09T04:33:52.187398+0000","last_active":"2024-05-09T04:33:53.194578+0000","last_peered":"2024-05-09T04:33:53.194578+0000","last_clean":"2024-05-09T04:33:53.194578+0000","last_became_active":"2024-05-09T04:33:52.186979+0000","last_became_peered":"2024-05-09T04:33:52.186979+0000","last_unstale":"2024-05-09T04:33:53.194578+0000","last_undegraded":"2024-05-09T04:33:53.194578+0000","last_fullsized":"2024-05-09T04:33:53.194578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:34:29.852986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.211667+0000","last_change":"2024-05-09T04:33:52.210098+0000","last_active":"2024-05-09T04:33:54.211667+0000","last_peered":"2024-05-09T04:33:54.211667+0000","last_clean":"2024-05-09T04:33:54.211667+0000","last_became_active":"2024-05-09T04:33:52.209575+0000","last_became_peered":"2024-05-09T04:33:52.209575+0000","last_unstale":"2024-05-09T04:33:54.211667+0000","last_undegraded":"2024-05-09T04:33:54.211667+0000","last_fullsized":"2024-05-09T04:33:54.211667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:24:41.478222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.878129+0000","last_change":"2024-05-09T04:33:52.195969+0000","last_active":"2024-05-09T04:33:54.878129+0000","last_peered":"2024-05-09T04:33:54.878129+0000","last_clean":"2024-05-09T04:33:54.878129+0000","last_became_active":"2024-05-09T04:33:52.195578+0000","last_became_peered":"2024-05-09T04:33:52.195578+0000","last_unstale":"2024-05-09T04:33:54.878129+0000","last_undegraded":"2024-05-09T04:33:54.878129+0000","last_fullsized":"2024-05-09T04:33:54.878129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:56.927838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.211122+0000","last_change":"2024-05-09T04:33:52.198619+0000","last_active":"2024-05-09T04:33:54.211122+0000","last_peered":"2024-05-09T04:33:54.211122+0000","last_clean":"2024-05-09T04:33:54.211122+0000","last_became_active":"2024-05-09T04:33:52.198345+0000","last_became_peered":"2024-05-09T04:33:52.198345+0000","last_unstale":"2024-05-09T04:33:54.211122+0000","last_undegraded":"2024-05-09T04:33:54.211122+0000","last_fullsized":"2024-05-09T04:33:54.211122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:24:37.669022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.878144+0000","last_change":"2024-05-09T04:33:52.202397+0000","last_active":"2024-05-09T04:33:54.878144+0000","last_peered":"2024-05-09T04:33:54.878144+0000","last_clean":"2024-05-09T04:33:54.878144+0000","last_became_active":"2024-05-09T04:33:52.202176+0000","last_became_peered":"2024-05-09T04:33:52.202176+0000","last_unstale":"2024-05-09T04:33:54.878144+0000","last_undegraded":"2024-05-09T04:33:54.878144+0000","last_fullsized":"2024-05-09T04:33:54.878144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:02:17.813339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.196047+0000","last_change":"2024-05-09T04:33:52.207259+0000","last_active":"2024-05-09T04:33:53.196047+0000","last_peered":"2024-05-09T04:33:53.196047+0000","last_clean":"2024-05-09T04:33:53.196047+0000","last_became_active":"2024-05-09T04:33:52.204206+0000","last_became_peered":"2024-05-09T04:33:52.204206+0000","last_unstale":"2024-05-09T04:33:53.196047+0000","last_undegraded":"2024-05-09T04:33:53.196047+0000","last_fullsized":"2024-05-09T04:33:53.196047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:35:43.478085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.879759+0000","last_change":"2024-05-09T04:33:52.205582+0000","last_active":"2024-05-09T04:33:54.879759+0000","last_peered":"2024-05-09T04:33:54.879759+0000","last_clean":"2024-05-09T04:33:54.879759+0000","last_became_active":"2024-05-09T04:33:52.205237+0000","last_became_peered":"2024-05-09T04:33:52.205237+0000","last_unstale":"2024-05-09T04:33:54.879759+0000","last_undegraded":"2024-05-09T04:33:54.879759+0000","last_fullsized":"2024-05-09T04:33:54.879759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:42.767983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.686517+0000","last_change":"2024-05-09T04:33:52.198058+0000","last_active":"2024-05-09T04:33:53.686517+0000","last_peered":"2024-05-09T04:33:53.686517+0000","last_clean":"2024-05-09T04:33:53.686517+0000","last_became_active":"2024-05-09T04:33:52.197701+0000","last_became_peered":"2024-05-09T04:33:52.197701+0000","last_unstale":"2024-05-09T04:33:53.686517+0000","last_undegraded":"2024-05-09T04:33:53.686517+0000","last_fullsized":"2024-05-09T04:33:53.686517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:24:37.669022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.686063+0000","last_change":"2024-05-09T04:33:52.194755+0000","last_active":"2024-05-09T04:33:53.686063+0000","last_peered":"2024-05-09T04:33:53.686063+0000","last_clean":"2024-05-09T04:33:53.686063+0000","last_became_active":"2024-05-09T04:33:52.194270+0000","last_became_peered":"2024-05-09T04:33:52.194270+0000","last_unstale":"2024-05-09T04:33:53.686063+0000","last_undegraded":"2024-05-09T04:33:53.686063+0000","last_fullsized":"2024-05-09T04:33:53.686063+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:59:28.002585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.195766+0000","last_change":"2024-05-09T04:33:52.200331+0000","last_active":"2024-05-09T04:33:53.195766+0000","last_peered":"2024-05-09T04:33:53.195766+0000","last_clean":"2024-05-09T04:33:53.195766+0000","last_became_active":"2024-05-09T04:33:52.199739+0000","last_became_peered":"2024-05-09T04:33:52.199739+0000","last_unstale":"2024-05-09T04:33:53.195766+0000","last_undegraded":"2024-05-09T04:33:53.195766+0000","last_fullsized":"2024-05-09T04:33:53.195766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:29:12.420301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.196580+0000","last_change":"2024-05-09T04:33:52.211817+0000","last_active":"2024-05-09T04:33:53.196580+0000","last_peered":"2024-05-09T04:33:53.196580+0000","last_clean":"2024-05-09T04:33:53.196580+0000","last_became_active":"2024-05-09T04:33:52.211533+0000","last_became_peered":"2024-05-09T04:33:52.211533+0000","last_unstale":"2024-05-09T04:33:53.196580+0000","last_undegraded":"2024-05-09T04:33:53.196580+0000","last_fullsized":"2024-05-09T04:33:53.196580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:34:29.852986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.193922+0000","last_change":"2024-05-09T04:33:52.177443+0000","last_active":"2024-05-09T04:33:53.193922+0000","last_peered":"2024-05-09T04:33:53.193922+0000","last_clean":"2024-05-09T04:33:53.193922+0000","last_became_active":"2024-05-09T04:33:52.177115+0000","last_became_peered":"2024-05-09T04:33:52.177115+0000","last_unstale":"2024-05-09T04:33:53.193922+0000","last_undegraded":"2024-05-09T04:33:53.193922+0000","last_fullsized":"2024-05-09T04:33:53.193922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:33:25.042543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.154598+0000","last_change":"2024-05-09T04:33:52.209143+0000","last_active":"2024-05-09T04:33:54.154598+0000","last_peered":"2024-05-09T04:33:54.154598+0000","last_clean":"2024-05-09T04:33:54.154598+0000","last_became_active":"2024-05-09T04:33:52.206547+0000","last_became_peered":"2024-05-09T04:33:52.206547+0000","last_unstale":"2024-05-09T04:33:54.154598+0000","last_undegraded":"2024-05-09T04:33:54.154598+0000","last_fullsized":"2024-05-09T04:33:54.154598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:02:17.813339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.211033+0000","last_change":"2024-05-09T04:33:52.198040+0000","last_active":"2024-05-09T04:33:54.211033+0000","last_peered":"2024-05-09T04:33:54.211033+0000","last_clean":"2024-05-09T04:33:54.211033+0000","last_became_active":"2024-05-09T04:33:52.197706+0000","last_became_peered":"2024-05-09T04:33:52.197706+0000","last_unstale":"2024-05-09T04:33:54.211033+0000","last_undegraded":"2024-05-09T04:33:54.211033+0000","last_fullsized":"2024-05-09T04:33:54.211033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:17.801441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.212667+0000","last_change":"2024-05-09T04:33:52.195546+0000","last_active":"2024-05-09T04:33:54.212667+0000","last_peered":"2024-05-09T04:33:54.212667+0000","last_clean":"2024-05-09T04:33:54.212667+0000","last_became_active":"2024-05-09T04:33:52.195138+0000","last_became_peered":"2024-05-09T04:33:52.195138+0000","last_unstale":"2024-05-09T04:33:54.212667+0000","last_undegraded":"2024-05-09T04:33:54.212667+0000","last_fullsized":"2024-05-09T04:33:54.212667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:42.767983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.732773+0000","last_change":"2024-05-09T04:33:52.192303+0000","last_active":"2024-05-09T04:33:54.732773+0000","last_peered":"2024-05-09T04:33:54.732773+0000","last_clean":"2024-05-09T04:33:54.732773+0000","last_became_active":"2024-05-09T04:33:52.191878+0000","last_became_peered":"2024-05-09T04:33:52.191878+0000","last_unstale":"2024-05-09T04:33:54.732773+0000","last_undegraded":"2024-05-09T04:33:54.732773+0000","last_fullsized":"2024-05-09T04:33:54.732773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:22:41.930610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.879711+0000","last_change":"2024-05-09T04:33:52.190589+0000","last_active":"2024-05-09T04:33:54.879711+0000","last_peered":"2024-05-09T04:33:54.879711+0000","last_clean":"2024-05-09T04:33:54.879711+0000","last_became_active":"2024-05-09T04:33:52.190312+0000","last_became_peered":"2024-05-09T04:33:52.190312+0000","last_unstale":"2024-05-09T04:33:54.879711+0000","last_undegraded":"2024-05-09T04:33:54.879711+0000","last_fullsized":"2024-05-09T04:33:54.879711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:19:08.318788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.686133+0000","last_change":"2024-05-09T04:33:52.195370+0000","last_active":"2024-05-09T04:33:53.686133+0000","last_peered":"2024-05-09T04:33:53.686133+0000","last_clean":"2024-05-09T04:33:53.686133+0000","last_became_active":"2024-05-09T04:33:52.194414+0000","last_became_peered":"2024-05-09T04:33:52.194414+0000","last_unstale":"2024-05-09T04:33:53.686133+0000","last_undegraded":"2024-05-09T04:33:53.686133+0000","last_fullsized":"2024-05-09T04:33:53.686133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:29:08.279802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.193862+0000","last_change":"2024-05-09T04:33:52.201922+0000","last_active":"2024-05-09T04:33:53.193862+0000","last_peered":"2024-05-09T04:33:53.193862+0000","last_clean":"2024-05-09T04:33:53.193862+0000","last_became_active":"2024-05-09T04:33:52.201635+0000","last_became_peered":"2024-05-09T04:33:52.201635+0000","last_unstale":"2024-05-09T04:33:53.193862+0000","last_undegraded":"2024-05-09T04:33:53.193862+0000","last_fullsized":"2024-05-09T04:33:53.193862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:27:41.648285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.211533+0000","last_change":"2024-05-09T04:33:52.195480+0000","last_active":"2024-05-09T04:33:54.211533+0000","last_peered":"2024-05-09T04:33:54.211533+0000","last_clean":"2024-05-09T04:33:54.211533+0000","last_became_active":"2024-05-09T04:33:52.195195+0000","last_became_peered":"2024-05-09T04:33:52.195195+0000","last_unstale":"2024-05-09T04:33:54.211533+0000","last_undegraded":"2024-05-09T04:33:54.211533+0000","last_fullsized":"2024-05-09T04:33:54.211533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:02:17.813339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.196691+0000","last_change":"2024-05-09T04:33:52.208038+0000","last_active":"2024-05-09T04:33:53.196691+0000","last_peered":"2024-05-09T04:33:53.196691+0000","last_clean":"2024-05-09T04:33:53.196691+0000","last_became_active":"2024-05-09T04:33:52.207755+0000","last_became_peered":"2024-05-09T04:33:52.207755+0000","last_unstale":"2024-05-09T04:33:53.196691+0000","last_undegraded":"2024-05-09T04:33:53.196691+0000","last_fullsized":"2024-05-09T04:33:53.196691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:59:28.002585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.211317+0000","last_change":"2024-05-09T04:33:52.209997+0000","last_active":"2024-05-09T04:33:54.211317+0000","last_peered":"2024-05-09T04:33:54.211317+0000","last_clean":"2024-05-09T04:33:54.211317+0000","last_became_active":"2024-05-09T04:33:52.209702+0000","last_became_peered":"2024-05-09T04:33:52.209702+0000","last_unstale":"2024-05-09T04:33:54.211317+0000","last_undegraded":"2024-05-09T04:33:54.211317+0000","last_fullsized":"2024-05-09T04:33:54.211317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:56.927838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.197678+0000","last_change":"2024-05-09T04:33:52.207838+0000","last_active":"2024-05-09T04:33:53.197678+0000","last_peered":"2024-05-09T04:33:53.197678+0000","last_clean":"2024-05-09T04:33:53.197678+0000","last_became_active":"2024-05-09T04:33:52.207601+0000","last_became_peered":"2024-05-09T04:33:52.207601+0000","last_unstale":"2024-05-09T04:33:53.197678+0000","last_undegraded":"2024-05-09T04:33:53.197678+0000","last_fullsized":"2024-05-09T04:33:53.197678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:56.927838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.686401+0000","last_change":"2024-05-09T04:33:52.196143+0000","last_active":"2024-05-09T04:33:53.686401+0000","last_peered":"2024-05-09T04:33:53.686401+0000","last_clean":"2024-05-09T04:33:53.686401+0000","last_became_active":"2024-05-09T04:33:52.195814+0000","last_became_peered":"2024-05-09T04:33:52.195814+0000","last_unstale":"2024-05-09T04:33:53.686401+0000","last_undegraded":"2024-05-09T04:33:53.686401+0000","last_fullsized":"2024-05-09T04:33:53.686401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:17.801441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.154429+0000","last_change":"2024-05-09T04:33:52.205497+0000","last_active":"2024-05-09T04:33:54.154429+0000","last_peered":"2024-05-09T04:33:54.154429+0000","last_clean":"2024-05-09T04:33:54.154429+0000","last_became_active":"2024-05-09T04:33:52.205219+0000","last_became_peered":"2024-05-09T04:33:52.205219+0000","last_unstale":"2024-05-09T04:33:54.154429+0000","last_undegraded":"2024-05-09T04:33:54.154429+0000","last_fullsized":"2024-05-09T04:33:54.154429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:17.801441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.197606+0000","last_change":"2024-05-09T04:33:52.213356+0000","last_active":"2024-05-09T04:33:53.197606+0000","last_peered":"2024-05-09T04:33:53.197606+0000","last_clean":"2024-05-09T04:33:53.197606+0000","last_became_active":"2024-05-09T04:33:52.212783+0000","last_became_peered":"2024-05-09T04:33:52.212783+0000","last_unstale":"2024-05-09T04:33:53.197606+0000","last_undegraded":"2024-05-09T04:33:53.197606+0000","last_fullsized":"2024-05-09T04:33:53.197606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:24:37.669022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.194008+0000","last_change":"2024-05-09T04:33:52.209567+0000","last_active":"2024-05-09T04:33:53.194008+0000","last_peered":"2024-05-09T04:33:53.194008+0000","last_clean":"2024-05-09T04:33:53.194008+0000","last_became_active":"2024-05-09T04:33:52.209260+0000","last_became_peered":"2024-05-09T04:33:52.209260+0000","last_unstale":"2024-05-09T04:33:53.194008+0000","last_undegraded":"2024-05-09T04:33:53.194008+0000","last_fullsized":"2024-05-09T04:33:53.194008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:35:43.478085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.211635+0000","last_change":"2024-05-09T04:33:52.210744+0000","last_active":"2024-05-09T04:33:54.211635+0000","last_peered":"2024-05-09T04:33:54.211635+0000","last_clean":"2024-05-09T04:33:54.211635+0000","last_became_active":"2024-05-09T04:33:52.210421+0000","last_became_peered":"2024-05-09T04:33:52.210421+0000","last_unstale":"2024-05-09T04:33:54.211635+0000","last_undegraded":"2024-05-09T04:33:54.211635+0000","last_fullsized":"2024-05-09T04:33:54.211635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:13:19.513750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.732744+0000","last_change":"2024-05-09T04:33:52.193832+0000","last_active":"2024-05-09T04:33:54.732744+0000","last_peered":"2024-05-09T04:33:54.732744+0000","last_clean":"2024-05-09T04:33:54.732744+0000","last_became_active":"2024-05-09T04:33:52.193531+0000","last_became_peered":"2024-05-09T04:33:52.193531+0000","last_unstale":"2024-05-09T04:33:54.732744+0000","last_undegraded":"2024-05-09T04:33:54.732744+0000","last_fullsized":"2024-05-09T04:33:54.732744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:19:44.181447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.685556+0000","last_change":"2024-05-09T04:33:52.198677+0000","last_active":"2024-05-09T04:33:53.685556+0000","last_peered":"2024-05-09T04:33:53.685556+0000","last_clean":"2024-05-09T04:33:53.685556+0000","last_became_active":"2024-05-09T04:33:52.198334+0000","last_became_peered":"2024-05-09T04:33:52.198334+0000","last_unstale":"2024-05-09T04:33:53.685556+0000","last_undegraded":"2024-05-09T04:33:53.685556+0000","last_fullsized":"2024-05-09T04:33:53.685556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:53:58.795910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.194924+0000","last_change":"2024-05-09T04:33:52.192680+0000","last_active":"2024-05-09T04:33:53.194924+0000","last_peered":"2024-05-09T04:33:53.194924+0000","last_clean":"2024-05-09T04:33:53.194924+0000","last_became_active":"2024-05-09T04:33:52.192454+0000","last_became_peered":"2024-05-09T04:33:52.192454+0000","last_unstale":"2024-05-09T04:33:53.194924+0000","last_undegraded":"2024-05-09T04:33:53.194924+0000","last_fullsized":"2024-05-09T04:33:53.194924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:50:52.026927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.196881+0000","last_change":"2024-05-09T04:33:51.177063+0000","last_active":"2024-05-09T04:33:53.196881+0000","last_peered":"2024-05-09T04:33:53.196881+0000","last_clean":"2024-05-09T04:33:53.196881+0000","last_became_active":"2024-05-09T04:33:51.176788+0000","last_became_peered":"2024-05-09T04:33:51.176788+0000","last_unstale":"2024-05-09T04:33:53.196881+0000","last_undegraded":"2024-05-09T04:33:53.196881+0000","last_fullsized":"2024-05-09T04:33:53.196881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:04:10.949412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.194154+0000","last_change":"2024-05-09T04:33:52.190946+0000","last_active":"2024-05-09T04:33:53.194154+0000","last_peered":"2024-05-09T04:33:53.194154+0000","last_clean":"2024-05-09T04:33:53.194154+0000","last_became_active":"2024-05-09T04:33:52.190598+0000","last_became_peered":"2024-05-09T04:33:52.190598+0000","last_unstale":"2024-05-09T04:33:53.194154+0000","last_undegraded":"2024-05-09T04:33:53.194154+0000","last_fullsized":"2024-05-09T04:33:53.194154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:53:58.795910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.685499+0000","last_change":"2024-05-09T04:33:51.177930+0000","last_active":"2024-05-09T04:33:53.685499+0000","last_peered":"2024-05-09T04:33:53.685499+0000","last_clean":"2024-05-09T04:33:53.685499+0000","last_became_active":"2024-05-09T04:33:51.174452+0000","last_became_peered":"2024-05-09T04:33:51.174452+0000","last_unstale":"2024-05-09T04:33:53.685499+0000","last_undegraded":"2024-05-09T04:33:53.685499+0000","last_fullsized":"2024-05-09T04:33:53.685499+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:26:18.474267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.196968+0000","last_change":"2024-05-09T04:33:51.169721+0000","last_active":"2024-05-09T04:33:53.196968+0000","last_peered":"2024-05-09T04:33:53.196968+0000","last_clean":"2024-05-09T04:33:53.196968+0000","last_became_active":"2024-05-09T04:33:51.169422+0000","last_became_peered":"2024-05-09T04:33:51.169422+0000","last_unstale":"2024-05-09T04:33:53.196968+0000","last_undegraded":"2024-05-09T04:33:53.196968+0000","last_fullsized":"2024-05-09T04:33:53.196968+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:36:52.136732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.196263+0000","last_change":"2024-05-09T04:33:52.208855+0000","last_active":"2024-05-09T04:33:53.196263+0000","last_peered":"2024-05-09T04:33:53.196263+0000","last_clean":"2024-05-09T04:33:53.196263+0000","last_became_active":"2024-05-09T04:33:52.208588+0000","last_became_peered":"2024-05-09T04:33:52.208588+0000","last_unstale":"2024-05-09T04:33:53.196263+0000","last_undegraded":"2024-05-09T04:33:53.196263+0000","last_fullsized":"2024-05-09T04:33:53.196263+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:53:58.795910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.211333+0000","last_change":"2024-05-09T04:33:51.178180+0000","last_active":"2024-05-09T04:33:54.211333+0000","last_peered":"2024-05-09T04:33:54.211333+0000","last_clean":"2024-05-09T04:33:54.211333+0000","last_became_active":"2024-05-09T04:33:51.175516+0000","last_became_peered":"2024-05-09T04:33:51.175516+0000","last_unstale":"2024-05-09T04:33:54.211333+0000","last_undegraded":"2024-05-09T04:33:54.211333+0000","last_fullsized":"2024-05-09T04:33:54.211333+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:26:18.474267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.154507+0000","last_change":"2024-05-09T04:33:52.196488+0000","last_active":"2024-05-09T04:33:54.154507+0000","last_peered":"2024-05-09T04:33:54.154507+0000","last_clean":"2024-05-09T04:33:54.154507+0000","last_became_active":"2024-05-09T04:33:52.196175+0000","last_became_peered":"2024-05-09T04:33:52.196175+0000","last_unstale":"2024-05-09T04:33:54.154507+0000","last_undegraded":"2024-05-09T04:33:54.154507+0000","last_fullsized":"2024-05-09T04:33:54.154507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:56.927838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.686280+0000","last_change":"2024-05-09T04:33:51.185310+0000","last_active":"2024-05-09T04:33:53.686280+0000","last_peered":"2024-05-09T04:33:53.686280+0000","last_clean":"2024-05-09T04:33:53.686280+0000","last_became_active":"2024-05-09T04:33:51.185016+0000","last_became_peered":"2024-05-09T04:33:51.185016+0000","last_unstale":"2024-05-09T04:33:53.686280+0000","last_undegraded":"2024-05-09T04:33:53.686280+0000","last_fullsized":"2024-05-09T04:33:53.686280+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:50:51.015799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.212510+0000","last_change":"2024-05-09T04:33:52.193316+0000","last_active":"2024-05-09T04:33:54.212510+0000","last_peered":"2024-05-09T04:33:54.212510+0000","last_clean":"2024-05-09T04:33:54.212510+0000","last_became_active":"2024-05-09T04:33:52.192939+0000","last_became_peered":"2024-05-09T04:33:52.192939+0000","last_unstale":"2024-05-09T04:33:54.212510+0000","last_undegraded":"2024-05-09T04:33:54.212510+0000","last_fullsized":"2024-05-09T04:33:54.212510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:46:44.064608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.212393+0000","last_change":"2024-05-09T04:33:51.183116+0000","last_active":"2024-05-09T04:33:54.212393+0000","last_peered":"2024-05-09T04:33:54.212393+0000","last_clean":"2024-05-09T04:33:54.212393+0000","last_became_active":"2024-05-09T04:33:51.182750+0000","last_became_peered":"2024-05-09T04:33:51.182750+0000","last_unstale":"2024-05-09T04:33:54.212393+0000","last_undegraded":"2024-05-09T04:33:54.212393+0000","last_fullsized":"2024-05-09T04:33:54.212393+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:04:10.949412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.195562+0000","last_change":"2024-05-09T04:33:52.201494+0000","last_active":"2024-05-09T04:33:53.195562+0000","last_peered":"2024-05-09T04:33:53.195562+0000","last_clean":"2024-05-09T04:33:53.195562+0000","last_became_active":"2024-05-09T04:33:52.197939+0000","last_became_peered":"2024-05-09T04:33:52.197939+0000","last_unstale":"2024-05-09T04:33:53.195562+0000","last_undegraded":"2024-05-09T04:33:53.195562+0000","last_fullsized":"2024-05-09T04:33:53.195562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:29:08.279802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.154034+0000","last_change":"2024-05-09T04:33:51.178564+0000","last_active":"2024-05-09T04:33:54.154034+0000","last_peered":"2024-05-09T04:33:54.154034+0000","last_clean":"2024-05-09T04:33:54.154034+0000","last_became_active":"2024-05-09T04:33:51.177732+0000","last_became_peered":"2024-05-09T04:33:51.177732+0000","last_unstale":"2024-05-09T04:33:54.154034+0000","last_undegraded":"2024-05-09T04:33:54.154034+0000","last_fullsized":"2024-05-09T04:33:54.154034+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:50:51.015799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.196170+0000","last_change":"2024-05-09T04:33:52.200019+0000","last_active":"2024-05-09T04:33:53.196170+0000","last_peered":"2024-05-09T04:33:53.196170+0000","last_clean":"2024-05-09T04:33:53.196170+0000","last_became_active":"2024-05-09T04:33:52.199279+0000","last_became_peered":"2024-05-09T04:33:52.199279+0000","last_unstale":"2024-05-09T04:33:53.196170+0000","last_undegraded":"2024-05-09T04:33:53.196170+0000","last_fullsized":"2024-05-09T04:33:53.196170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:19:44.181447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.685981+0000","last_change":"2024-05-09T04:33:51.173622+0000","last_active":"2024-05-09T04:33:53.685981+0000","last_peered":"2024-05-09T04:33:53.685981+0000","last_clean":"2024-05-09T04:33:53.685981+0000","last_became_active":"2024-05-09T04:33:51.173158+0000","last_became_peered":"2024-05-09T04:33:51.173158+0000","last_unstale":"2024-05-09T04:33:53.685981+0000","last_undegraded":"2024-05-09T04:33:53.685981+0000","last_fullsized":"2024-05-09T04:33:53.685981+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:08:08.955016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.878757+0000","last_change":"2024-05-09T04:33:52.214221+0000","last_active":"2024-05-09T04:33:54.878757+0000","last_peered":"2024-05-09T04:33:54.878757+0000","last_clean":"2024-05-09T04:33:54.878757+0000","last_became_active":"2024-05-09T04:33:52.213773+0000","last_became_peered":"2024-05-09T04:33:52.213773+0000","last_unstale":"2024-05-09T04:33:54.878757+0000","last_undegraded":"2024-05-09T04:33:54.878757+0000","last_fullsized":"2024-05-09T04:33:54.878757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:10:42.676456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.194274+0000","last_change":"2024-05-09T04:33:51.172542+0000","last_active":"2024-05-09T04:33:53.194274+0000","last_peered":"2024-05-09T04:33:53.194274+0000","last_clean":"2024-05-09T04:33:53.194274+0000","last_became_active":"2024-05-09T04:33:51.172232+0000","last_became_peered":"2024-05-09T04:33:51.172232+0000","last_unstale":"2024-05-09T04:33:53.194274+0000","last_undegraded":"2024-05-09T04:33:53.194274+0000","last_fullsized":"2024-05-09T04:33:53.194274+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:15.760572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.194183+0000","last_change":"2024-05-09T04:33:52.195027+0000","last_active":"2024-05-09T04:33:53.194183+0000","last_peered":"2024-05-09T04:33:53.194183+0000","last_clean":"2024-05-09T04:33:53.194183+0000","last_became_active":"2024-05-09T04:33:52.194590+0000","last_became_peered":"2024-05-09T04:33:52.194590+0000","last_unstale":"2024-05-09T04:33:53.194183+0000","last_undegraded":"2024-05-09T04:33:53.194183+0000","last_fullsized":"2024-05-09T04:33:53.194183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:34:40.257159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.211870+0000","last_change":"2024-05-09T04:33:51.177887+0000","last_active":"2024-05-09T04:33:54.211870+0000","last_peered":"2024-05-09T04:33:54.211870+0000","last_clean":"2024-05-09T04:33:54.211870+0000","last_became_active":"2024-05-09T04:33:51.175721+0000","last_became_peered":"2024-05-09T04:33:51.175721+0000","last_unstale":"2024-05-09T04:33:54.211870+0000","last_undegraded":"2024-05-09T04:33:54.211870+0000","last_fullsized":"2024-05-09T04:33:54.211870+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:02:50.965016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.878431+0000","last_change":"2024-05-09T04:33:52.211924+0000","last_active":"2024-05-09T04:33:54.878431+0000","last_peered":"2024-05-09T04:33:54.878431+0000","last_clean":"2024-05-09T04:33:54.878431+0000","last_became_active":"2024-05-09T04:33:52.211546+0000","last_became_peered":"2024-05-09T04:33:52.211546+0000","last_unstale":"2024-05-09T04:33:54.878431+0000","last_undegraded":"2024-05-09T04:33:54.878431+0000","last_fullsized":"2024-05-09T04:33:54.878431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:24:41.478222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.685619+0000","last_change":"2024-05-09T04:33:51.176697+0000","last_active":"2024-05-09T04:33:53.685619+0000","last_peered":"2024-05-09T04:33:53.685619+0000","last_clean":"2024-05-09T04:33:53.685619+0000","last_became_active":"2024-05-09T04:33:51.176351+0000","last_became_peered":"2024-05-09T04:33:51.176351+0000","last_unstale":"2024-05-09T04:33:53.685619+0000","last_undegraded":"2024-05-09T04:33:53.685619+0000","last_fullsized":"2024-05-09T04:33:53.685619+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:04:10.949412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.685570+0000","last_change":"2024-05-09T04:33:52.187596+0000","last_active":"2024-05-09T04:33:53.685570+0000","last_peered":"2024-05-09T04:33:53.685570+0000","last_clean":"2024-05-09T04:33:53.685570+0000","last_became_active":"2024-05-09T04:33:52.187287+0000","last_became_peered":"2024-05-09T04:33:52.187287+0000","last_unstale":"2024-05-09T04:33:53.685570+0000","last_undegraded":"2024-05-09T04:33:53.685570+0000","last_fullsized":"2024-05-09T04:33:53.685570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:34:40.257159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.732380+0000","last_change":"2024-05-09T04:33:51.166963+0000","last_active":"2024-05-09T04:33:54.732380+0000","last_peered":"2024-05-09T04:33:54.732380+0000","last_clean":"2024-05-09T04:33:54.732380+0000","last_became_active":"2024-05-09T04:33:51.165585+0000","last_became_peered":"2024-05-09T04:33:51.165585+0000","last_unstale":"2024-05-09T04:33:54.732380+0000","last_undegraded":"2024-05-09T04:33:54.732380+0000","last_fullsized":"2024-05-09T04:33:54.732380+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:04:10.949412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.878130+0000","last_change":"2024-05-09T04:33:52.207666+0000","last_active":"2024-05-09T04:33:54.878130+0000","last_peered":"2024-05-09T04:33:54.878130+0000","last_clean":"2024-05-09T04:33:54.878130+0000","last_became_active":"2024-05-09T04:33:52.207429+0000","last_became_peered":"2024-05-09T04:33:52.207429+0000","last_unstale":"2024-05-09T04:33:54.878130+0000","last_undegraded":"2024-05-09T04:33:54.878130+0000","last_fullsized":"2024-05-09T04:33:54.878130+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:13:19.513750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.196757+0000","last_change":"2024-05-09T04:33:51.172081+0000","last_active":"2024-05-09T04:33:53.196757+0000","last_peered":"2024-05-09T04:33:53.196757+0000","last_clean":"2024-05-09T04:33:53.196757+0000","last_became_active":"2024-05-09T04:33:51.171700+0000","last_became_peered":"2024-05-09T04:33:51.171700+0000","last_unstale":"2024-05-09T04:33:53.196757+0000","last_undegraded":"2024-05-09T04:33:53.196757+0000","last_fullsized":"2024-05-09T04:33:53.196757+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:15.760572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.879267+0000","last_change":"2024-05-09T04:33:52.206889+0000","last_active":"2024-05-09T04:33:54.879267+0000","last_peered":"2024-05-09T04:33:54.879267+0000","last_clean":"2024-05-09T04:33:54.879267+0000","last_became_active":"2024-05-09T04:33:52.206452+0000","last_became_peered":"2024-05-09T04:33:52.206452+0000","last_unstale":"2024-05-09T04:33:54.879267+0000","last_undegraded":"2024-05-09T04:33:54.879267+0000","last_fullsized":"2024-05-09T04:33:54.879267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:40:40.749511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.732880+0000","last_change":"2024-05-09T04:33:51.167434+0000","last_active":"2024-05-09T04:33:54.732880+0000","last_peered":"2024-05-09T04:33:54.732880+0000","last_clean":"2024-05-09T04:33:54.732880+0000","last_became_active":"2024-05-09T04:33:51.167117+0000","last_became_peered":"2024-05-09T04:33:51.167117+0000","last_unstale":"2024-05-09T04:33:54.732880+0000","last_undegraded":"2024-05-09T04:33:54.732880+0000","last_fullsized":"2024-05-09T04:33:54.732880+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:02:50.965016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.880269+0000","last_change":"2024-05-09T04:33:52.207191+0000","last_active":"2024-05-09T04:33:54.880269+0000","last_peered":"2024-05-09T04:33:54.880269+0000","last_clean":"2024-05-09T04:33:54.880269+0000","last_became_active":"2024-05-09T04:33:52.206932+0000","last_became_peered":"2024-05-09T04:33:52.206932+0000","last_unstale":"2024-05-09T04:33:54.880269+0000","last_undegraded":"2024-05-09T04:33:54.880269+0000","last_fullsized":"2024-05-09T04:33:54.880269+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:49:29.149981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.154386+0000","last_change":"2024-05-09T04:33:51.163656+0000","last_active":"2024-05-09T04:33:54.154386+0000","last_peered":"2024-05-09T04:33:54.154386+0000","last_clean":"2024-05-09T04:33:54.154386+0000","last_became_active":"2024-05-09T04:33:51.163342+0000","last_became_peered":"2024-05-09T04:33:51.163342+0000","last_unstale":"2024-05-09T04:33:54.154386+0000","last_undegraded":"2024-05-09T04:33:54.154386+0000","last_fullsized":"2024-05-09T04:33:54.154386+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:19:47.722161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.212177+0000","last_change":"2024-05-09T04:33:52.205802+0000","last_active":"2024-05-09T04:33:54.212177+0000","last_peered":"2024-05-09T04:33:54.212177+0000","last_clean":"2024-05-09T04:33:54.212177+0000","last_became_active":"2024-05-09T04:33:52.205470+0000","last_became_peered":"2024-05-09T04:33:52.205470+0000","last_unstale":"2024-05-09T04:33:54.212177+0000","last_undegraded":"2024-05-09T04:33:54.212177+0000","last_fullsized":"2024-05-09T04:33:54.212177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:40:40.749511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.732648+0000","last_change":"2024-05-09T04:33:51.180702+0000","last_active":"2024-05-09T04:33:54.732648+0000","last_peered":"2024-05-09T04:33:54.732648+0000","last_clean":"2024-05-09T04:33:54.732648+0000","last_became_active":"2024-05-09T04:33:51.180322+0000","last_became_peered":"2024-05-09T04:33:51.180322+0000","last_unstale":"2024-05-09T04:33:54.732648+0000","last_undegraded":"2024-05-09T04:33:54.732648+0000","last_fullsized":"2024-05-09T04:33:54.732648+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:26:18.474267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.878990+0000","last_change":"2024-05-09T04:33:52.204685+0000","last_active":"2024-05-09T04:33:54.878990+0000","last_peered":"2024-05-09T04:33:54.878990+0000","last_clean":"2024-05-09T04:33:54.878990+0000","last_became_active":"2024-05-09T04:33:52.204353+0000","last_became_peered":"2024-05-09T04:33:52.204353+0000","last_unstale":"2024-05-09T04:33:54.878990+0000","last_undegraded":"2024-05-09T04:33:54.878990+0000","last_fullsized":"2024-05-09T04:33:54.878990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:19:13.480834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":110,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.877786+0000","last_change":"2024-05-09T04:33:41.092063+0000","last_active":"2024-05-09T04:33:54.877786+0000","last_peered":"2024-05-09T04:33:54.877786+0000","last_clean":"2024-05-09T04:33:54.877786+0000","last_became_active":"2024-05-09T04:33:41.091194+0000","last_became_peered":"2024-05-09T04:33:41.091194+0000","last_unstale":"2024-05-09T04:33:54.877786+0000","last_undegraded":"2024-05-09T04:33:54.877786+0000","last_fullsized":"2024-05-09T04:33:54.877786+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:40.045339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:40.045339+0000","last_clean_scrub_stamp":"2024-05-09T04:33:40.045339+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:04.908456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.211276+0000","last_change":"2024-05-09T04:33:51.179637+0000","last_active":"2024-05-09T04:33:54.211276+0000","last_peered":"2024-05-09T04:33:54.211276+0000","last_clean":"2024-05-09T04:33:54.211276+0000","last_became_active":"2024-05-09T04:33:51.179311+0000","last_became_peered":"2024-05-09T04:33:51.179311+0000","last_unstale":"2024-05-09T04:33:54.211276+0000","last_undegraded":"2024-05-09T04:33:54.211276+0000","last_fullsized":"2024-05-09T04:33:54.211276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:08:08.955016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.196827+0000","last_change":"2024-05-09T04:33:52.203706+0000","last_active":"2024-05-09T04:33:53.196827+0000","last_peered":"2024-05-09T04:33:53.196827+0000","last_clean":"2024-05-09T04:33:53.196827+0000","last_became_active":"2024-05-09T04:33:52.200053+0000","last_became_peered":"2024-05-09T04:33:52.200053+0000","last_unstale":"2024-05-09T04:33:53.196827+0000","last_undegraded":"2024-05-09T04:33:53.196827+0000","last_fullsized":"2024-05-09T04:33:53.196827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:29:08.279802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.877506+0000","last_change":"2024-05-09T04:33:51.185731+0000","last_active":"2024-05-09T04:33:54.877506+0000","last_peered":"2024-05-09T04:33:54.877506+0000","last_clean":"2024-05-09T04:33:54.877506+0000","last_became_active":"2024-05-09T04:33:51.184982+0000","last_became_peered":"2024-05-09T04:33:51.184982+0000","last_unstale":"2024-05-09T04:33:54.877506+0000","last_undegraded":"2024-05-09T04:33:54.877506+0000","last_fullsized":"2024-05-09T04:33:54.877506+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:29:11.409173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.193579+0000","last_change":"2024-05-09T04:33:52.208347+0000","last_active":"2024-05-09T04:33:53.193579+0000","last_peered":"2024-05-09T04:33:53.193579+0000","last_clean":"2024-05-09T04:33:53.193579+0000","last_became_active":"2024-05-09T04:33:52.206631+0000","last_became_peered":"2024-05-09T04:33:52.206631+0000","last_unstale":"2024-05-09T04:33:53.193579+0000","last_undegraded":"2024-05-09T04:33:53.193579+0000","last_fullsized":"2024-05-09T04:33:53.193579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:00:54.458043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.732806+0000","last_change":"2024-05-09T04:33:51.167013+0000","last_active":"2024-05-09T04:33:54.732806+0000","last_peered":"2024-05-09T04:33:54.732806+0000","last_clean":"2024-05-09T04:33:54.732806+0000","last_became_active":"2024-05-09T04:33:51.165450+0000","last_became_peered":"2024-05-09T04:33:51.165450+0000","last_unstale":"2024-05-09T04:33:54.732806+0000","last_undegraded":"2024-05-09T04:33:54.732806+0000","last_fullsized":"2024-05-09T04:33:54.732806+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:08:08.955016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.154238+0000","last_change":"2024-05-09T04:33:52.190418+0000","last_active":"2024-05-09T04:33:54.154238+0000","last_peered":"2024-05-09T04:33:54.154238+0000","last_clean":"2024-05-09T04:33:54.154238+0000","last_became_active":"2024-05-09T04:33:52.190069+0000","last_became_peered":"2024-05-09T04:33:52.190069+0000","last_unstale":"2024-05-09T04:33:54.154238+0000","last_undegraded":"2024-05-09T04:33:54.154238+0000","last_fullsized":"2024-05-09T04:33:54.154238+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:29:08.279802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.212222+0000","last_change":"2024-05-09T04:33:51.181463+0000","last_active":"2024-05-09T04:33:54.212222+0000","last_peered":"2024-05-09T04:33:54.212222+0000","last_clean":"2024-05-09T04:33:54.212222+0000","last_became_active":"2024-05-09T04:33:51.181213+0000","last_became_peered":"2024-05-09T04:33:51.181213+0000","last_unstale":"2024-05-09T04:33:54.212222+0000","last_undegraded":"2024-05-09T04:33:54.212222+0000","last_fullsized":"2024-05-09T04:33:54.212222+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:50:51.015799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.195331+0000","last_change":"2024-05-09T04:33:52.188609+0000","last_active":"2024-05-09T04:33:53.195331+0000","last_peered":"2024-05-09T04:33:53.195331+0000","last_clean":"2024-05-09T04:33:53.195331+0000","last_became_active":"2024-05-09T04:33:52.188379+0000","last_became_peered":"2024-05-09T04:33:52.188379+0000","last_unstale":"2024-05-09T04:33:53.195331+0000","last_undegraded":"2024-05-09T04:33:53.195331+0000","last_fullsized":"2024-05-09T04:33:53.195331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:19:48.733290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.153924+0000","last_change":"2024-05-09T04:33:51.164123+0000","last_active":"2024-05-09T04:33:54.153924+0000","last_peered":"2024-05-09T04:33:54.153924+0000","last_clean":"2024-05-09T04:33:54.153924+0000","last_became_active":"2024-05-09T04:33:51.163686+0000","last_became_peered":"2024-05-09T04:33:51.163686+0000","last_unstale":"2024-05-09T04:33:54.153924+0000","last_undegraded":"2024-05-09T04:33:54.153924+0000","last_fullsized":"2024-05-09T04:33:54.153924+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:29:11.409173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.212248+0000","last_change":"2024-05-09T04:33:52.209697+0000","last_active":"2024-05-09T04:33:54.212248+0000","last_peered":"2024-05-09T04:33:54.212248+0000","last_clean":"2024-05-09T04:33:54.212248+0000","last_became_active":"2024-05-09T04:33:52.209347+0000","last_became_peered":"2024-05-09T04:33:52.209347+0000","last_unstale":"2024-05-09T04:33:54.212248+0000","last_undegraded":"2024-05-09T04:33:54.212248+0000","last_fullsized":"2024-05-09T04:33:54.212248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:19:08.318788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.879358+0000","last_change":"2024-05-09T04:33:51.184584+0000","last_active":"2024-05-09T04:33:54.879358+0000","last_peered":"2024-05-09T04:33:54.879358+0000","last_clean":"2024-05-09T04:33:54.879358+0000","last_became_active":"2024-05-09T04:33:51.184369+0000","last_became_peered":"2024-05-09T04:33:51.184369+0000","last_unstale":"2024-05-09T04:33:54.879358+0000","last_undegraded":"2024-05-09T04:33:54.879358+0000","last_fullsized":"2024-05-09T04:33:54.879358+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:22:40.919482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.154095+0000","last_change":"2024-05-09T04:33:52.189698+0000","last_active":"2024-05-09T04:33:54.154095+0000","last_peered":"2024-05-09T04:33:54.154095+0000","last_clean":"2024-05-09T04:33:54.154095+0000","last_became_active":"2024-05-09T04:33:52.189388+0000","last_became_peered":"2024-05-09T04:33:52.189388+0000","last_unstale":"2024-05-09T04:33:54.154095+0000","last_undegraded":"2024-05-09T04:33:54.154095+0000","last_fullsized":"2024-05-09T04:33:54.154095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:59:28.002585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.179780+0000","last_change":"2024-05-09T04:33:51.173756+0000","last_active":"2024-05-09T04:33:54.179780+0000","last_peered":"2024-05-09T04:33:54.179780+0000","last_clean":"2024-05-09T04:33:54.179780+0000","last_became_active":"2024-05-09T04:33:51.173416+0000","last_became_peered":"2024-05-09T04:33:51.173416+0000","last_unstale":"2024-05-09T04:33:54.179780+0000","last_undegraded":"2024-05-09T04:33:54.179780+0000","last_fullsized":"2024-05-09T04:33:54.179780+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:02:50.965016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.197927+0000","last_change":"2024-05-09T04:33:52.205567+0000","last_active":"2024-05-09T04:33:53.197927+0000","last_peered":"2024-05-09T04:33:53.197927+0000","last_clean":"2024-05-09T04:33:53.197927+0000","last_became_active":"2024-05-09T04:33:52.205252+0000","last_became_peered":"2024-05-09T04:33:52.205252+0000","last_unstale":"2024-05-09T04:33:53.197927+0000","last_undegraded":"2024-05-09T04:33:53.197927+0000","last_fullsized":"2024-05-09T04:33:53.197927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:13:19.513750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.686413+0000","last_change":"2024-05-09T04:33:51.181117+0000","last_active":"2024-05-09T04:33:53.686413+0000","last_peered":"2024-05-09T04:33:53.686413+0000","last_clean":"2024-05-09T04:33:53.686413+0000","last_became_active":"2024-05-09T04:33:51.179062+0000","last_became_peered":"2024-05-09T04:33:51.179062+0000","last_unstale":"2024-05-09T04:33:53.686413+0000","last_undegraded":"2024-05-09T04:33:53.686413+0000","last_fullsized":"2024-05-09T04:33:53.686413+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:19:47.722161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.195642+0000","last_change":"2024-05-09T04:33:52.181414+0000","last_active":"2024-05-09T04:33:53.195642+0000","last_peered":"2024-05-09T04:33:53.195642+0000","last_clean":"2024-05-09T04:33:53.195642+0000","last_became_active":"2024-05-09T04:33:52.180058+0000","last_became_peered":"2024-05-09T04:33:52.180058+0000","last_unstale":"2024-05-09T04:33:53.195642+0000","last_undegraded":"2024-05-09T04:33:53.195642+0000","last_fullsized":"2024-05-09T04:33:53.195642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:17.801441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.877505+0000","last_change":"2024-05-09T04:33:51.183265+0000","last_active":"2024-05-09T04:33:54.877505+0000","last_peered":"2024-05-09T04:33:54.877505+0000","last_clean":"2024-05-09T04:33:54.877505+0000","last_became_active":"2024-05-09T04:33:51.181781+0000","last_became_peered":"2024-05-09T04:33:51.181781+0000","last_unstale":"2024-05-09T04:33:54.877505+0000","last_undegraded":"2024-05-09T04:33:54.877505+0000","last_fullsized":"2024-05-09T04:33:54.877505+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:19:47.722161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.194563+0000","last_change":"2024-05-09T04:33:52.203247+0000","last_active":"2024-05-09T04:33:53.194563+0000","last_peered":"2024-05-09T04:33:53.194563+0000","last_clean":"2024-05-09T04:33:53.194563+0000","last_became_active":"2024-05-09T04:33:52.203006+0000","last_became_peered":"2024-05-09T04:33:52.203006+0000","last_unstale":"2024-05-09T04:33:53.194563+0000","last_undegraded":"2024-05-09T04:33:53.194563+0000","last_fullsized":"2024-05-09T04:33:53.194563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:02:51.976144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.154676+0000","last_change":"2024-05-09T04:33:51.172718+0000","last_active":"2024-05-09T04:33:54.154676+0000","last_peered":"2024-05-09T04:33:54.154676+0000","last_clean":"2024-05-09T04:33:54.154676+0000","last_became_active":"2024-05-09T04:33:51.172131+0000","last_became_peered":"2024-05-09T04:33:51.172131+0000","last_unstale":"2024-05-09T04:33:54.154676+0000","last_undegraded":"2024-05-09T04:33:54.154676+0000","last_fullsized":"2024-05-09T04:33:54.154676+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:08:08.955016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.196487+0000","last_change":"2024-05-09T04:33:52.201413+0000","last_active":"2024-05-09T04:33:53.196487+0000","last_peered":"2024-05-09T04:33:53.196487+0000","last_clean":"2024-05-09T04:33:53.196487+0000","last_became_active":"2024-05-09T04:33:52.199032+0000","last_became_peered":"2024-05-09T04:33:52.199032+0000","last_unstale":"2024-05-09T04:33:53.196487+0000","last_undegraded":"2024-05-09T04:33:53.196487+0000","last_fullsized":"2024-05-09T04:33:53.196487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:34:40.257159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.154633+0000","last_change":"2024-05-09T04:33:51.159715+0000","last_active":"2024-05-09T04:33:54.154633+0000","last_peered":"2024-05-09T04:33:54.154633+0000","last_clean":"2024-05-09T04:33:54.154633+0000","last_became_active":"2024-05-09T04:33:51.159395+0000","last_became_peered":"2024-05-09T04:33:51.159395+0000","last_unstale":"2024-05-09T04:33:54.154633+0000","last_undegraded":"2024-05-09T04:33:54.154633+0000","last_fullsized":"2024-05-09T04:33:54.154633+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:19:43.170318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.197360+0000","last_change":"2024-05-09T04:33:52.198684+0000","last_active":"2024-05-09T04:33:53.197360+0000","last_peered":"2024-05-09T04:33:53.197360+0000","last_clean":"2024-05-09T04:33:53.197360+0000","last_became_active":"2024-05-09T04:33:52.196451+0000","last_became_peered":"2024-05-09T04:33:52.196451+0000","last_unstale":"2024-05-09T04:33:53.197360+0000","last_undegraded":"2024-05-09T04:33:53.197360+0000","last_fullsized":"2024-05-09T04:33:53.197360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:17.801441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.878591+0000","last_change":"2024-05-09T04:33:51.186050+0000","last_active":"2024-05-09T04:33:54.878591+0000","last_peered":"2024-05-09T04:33:54.878591+0000","last_clean":"2024-05-09T04:33:54.878591+0000","last_became_active":"2024-05-09T04:33:51.185720+0000","last_became_peered":"2024-05-09T04:33:51.185720+0000","last_unstale":"2024-05-09T04:33:54.878591+0000","last_undegraded":"2024-05-09T04:33:54.878591+0000","last_fullsized":"2024-05-09T04:33:54.878591+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:11:05.364073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.685574+0000","last_change":"2024-05-09T04:33:52.191123+0000","last_active":"2024-05-09T04:33:53.685574+0000","last_peered":"2024-05-09T04:33:53.685574+0000","last_clean":"2024-05-09T04:33:53.685574+0000","last_became_active":"2024-05-09T04:33:52.189005+0000","last_became_peered":"2024-05-09T04:33:52.189005+0000","last_unstale":"2024-05-09T04:33:53.685574+0000","last_undegraded":"2024-05-09T04:33:53.685574+0000","last_fullsized":"2024-05-09T04:33:53.685574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:34:29.852986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.212085+0000","last_change":"2024-05-09T04:33:52.208515+0000","last_active":"2024-05-09T04:33:54.212085+0000","last_peered":"2024-05-09T04:33:54.212085+0000","last_clean":"2024-05-09T04:33:54.212085+0000","last_became_active":"2024-05-09T04:33:52.208236+0000","last_became_peered":"2024-05-09T04:33:52.208236+0000","last_unstale":"2024-05-09T04:33:54.212085+0000","last_undegraded":"2024-05-09T04:33:54.212085+0000","last_fullsized":"2024-05-09T04:33:54.212085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:19:13.480834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.878387+0000","last_change":"2024-05-09T04:33:51.187919+0000","last_active":"2024-05-09T04:33:54.878387+0000","last_peered":"2024-05-09T04:33:54.878387+0000","last_clean":"2024-05-09T04:33:54.878387+0000","last_became_active":"2024-05-09T04:33:51.187590+0000","last_became_peered":"2024-05-09T04:33:51.187590+0000","last_unstale":"2024-05-09T04:33:54.878387+0000","last_undegraded":"2024-05-09T04:33:54.878387+0000","last_fullsized":"2024-05-09T04:33:54.878387+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:50:51.015799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.211737+0000","last_change":"2024-05-09T04:33:52.209870+0000","last_active":"2024-05-09T04:33:54.211737+0000","last_peered":"2024-05-09T04:33:54.211737+0000","last_clean":"2024-05-09T04:33:54.211737+0000","last_became_active":"2024-05-09T04:33:52.209199+0000","last_became_peered":"2024-05-09T04:33:52.209199+0000","last_unstale":"2024-05-09T04:33:54.211737+0000","last_undegraded":"2024-05-09T04:33:54.211737+0000","last_fullsized":"2024-05-09T04:33:54.211737+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:10:42.676456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.879005+0000","last_change":"2024-05-09T04:33:51.187870+0000","last_active":"2024-05-09T04:33:54.879005+0000","last_peered":"2024-05-09T04:33:54.879005+0000","last_clean":"2024-05-09T04:33:54.879005+0000","last_became_active":"2024-05-09T04:33:51.187575+0000","last_became_peered":"2024-05-09T04:33:51.187575+0000","last_unstale":"2024-05-09T04:33:54.879005+0000","last_undegraded":"2024-05-09T04:33:54.879005+0000","last_fullsized":"2024-05-09T04:33:54.879005+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:19:43.170318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.195161+0000","last_change":"2024-05-09T04:33:52.201406+0000","last_active":"2024-05-09T04:33:53.195161+0000","last_peered":"2024-05-09T04:33:53.195161+0000","last_clean":"2024-05-09T04:33:53.195161+0000","last_became_active":"2024-05-09T04:33:52.200809+0000","last_became_peered":"2024-05-09T04:33:52.200809+0000","last_unstale":"2024-05-09T04:33:53.195161+0000","last_undegraded":"2024-05-09T04:33:53.195161+0000","last_fullsized":"2024-05-09T04:33:53.195161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:59:28.002585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.197985+0000","last_change":"2024-05-09T04:33:51.168105+0000","last_active":"2024-05-09T04:33:53.197985+0000","last_peered":"2024-05-09T04:33:53.197985+0000","last_clean":"2024-05-09T04:33:53.197985+0000","last_became_active":"2024-05-09T04:33:51.167790+0000","last_became_peered":"2024-05-09T04:33:51.167790+0000","last_unstale":"2024-05-09T04:33:53.197985+0000","last_undegraded":"2024-05-09T04:33:53.197985+0000","last_fullsized":"2024-05-09T04:33:53.197985+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:08:08.955016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.197883+0000","last_change":"2024-05-09T04:33:52.205526+0000","last_active":"2024-05-09T04:33:53.197883+0000","last_peered":"2024-05-09T04:33:53.197883+0000","last_clean":"2024-05-09T04:33:53.197883+0000","last_became_active":"2024-05-09T04:33:52.205231+0000","last_became_peered":"2024-05-09T04:33:52.205231+0000","last_unstale":"2024-05-09T04:33:53.197883+0000","last_undegraded":"2024-05-09T04:33:53.197883+0000","last_fullsized":"2024-05-09T04:33:53.197883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:02:17.813339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.195818+0000","last_change":"2024-05-09T04:33:51.176461+0000","last_active":"2024-05-09T04:33:53.195818+0000","last_peered":"2024-05-09T04:33:53.195818+0000","last_clean":"2024-05-09T04:33:53.195818+0000","last_became_active":"2024-05-09T04:33:51.176175+0000","last_became_peered":"2024-05-09T04:33:51.176175+0000","last_unstale":"2024-05-09T04:33:53.195818+0000","last_undegraded":"2024-05-09T04:33:53.195818+0000","last_fullsized":"2024-05-09T04:33:53.195818+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:01:20.702014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T04:34:02.116 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T04:34:02.347 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.347+0000 7f6d26600640 1 Processor -- start 2024-05-09T04:34:02.347 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.347+0000 7f6d26600640 1 -- start start 2024-05-09T04:34:02.348 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.348+0000 7f6d26600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f6d2006e860 0x7f6d2006ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:02.348 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.348+0000 7f6d26600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6d200b4910 0x7f6d200b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:02.348 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.348+0000 7f6d26600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d2006f180 con 0x7f6d2015d780 2024-05-09T04:34:02.348 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.348+0000 7f6d26600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f6d20161360 con 0x7f6d2006e860 2024-05-09T04:34:02.349 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.348+0000 7f6d26600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f6d201615c0 con 0x7f6d200b4910 2024-05-09T04:34:02.349 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.348+0000 7f6d1ea00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f6d2015d780 0x7f6d20160c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:37066/0 (socket says 172.21.15.145:37066) 2024-05-09T04:34:02.349 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.348+0000 7f6d1ea00640 1 -- 172.21.15.145:0/7820477 learned_addr learned my addr 172.21.15.145:0/7820477 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:34:02.351 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.349+0000 7f6d1fe00640 1 --2- 172.21.15.145:0/7820477 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6d200b4910 0x7f6d200b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:02.351 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.349+0000 7f6d1f400640 1 --2- 172.21.15.145:0/7820477 >> v2:172.21.15.192:3300/0 conn(0x7f6d2006e860 0x7f6d2006ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:02.351 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.349+0000 7f6d1e000640 1 -- 172.21.15.145:0/7820477 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3020088759 0 0) 0x7f6d2006f180 con 0x7f6d2015d780 2024-05-09T04:34:02.351 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.349+0000 7f6d1e000640 1 -- 172.21.15.145:0/7820477 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6d04003520 con 0x7f6d2015d780 2024-05-09T04:34:02.351 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.349+0000 7f6d1e000640 1 -- 172.21.15.145:0/7820477 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1591018455 0 0) 0x7f6d04003520 con 0x7f6d2015d780 2024-05-09T04:34:02.351 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.349+0000 7f6d1e000640 1 -- 172.21.15.145:0/7820477 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6d200b4910 msgr2=0x7f6d200b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:02.351 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.349+0000 7f6d1e000640 1 --2- 172.21.15.145:0/7820477 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6d200b4910 0x7f6d200b4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:02.352 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.349+0000 7f6d1e000640 1 -- 172.21.15.145:0/7820477 >> v2:172.21.15.192:3300/0 conn(0x7f6d2006e860 msgr2=0x7f6d2006ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:02.352 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.349+0000 7f6d1e000640 1 --2- 172.21.15.145:0/7820477 >> v2:172.21.15.192:3300/0 conn(0x7f6d2006e860 0x7f6d2006ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:02.352 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.349+0000 7f6d1e000640 1 -- 172.21.15.145:0/7820477 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d201618a0 con 0x7f6d2015d780 2024-05-09T04:34:02.352 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.349+0000 7f6d1e000640 1 -- 172.21.15.145:0/7820477 <== mon.0 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3474413261 0 0) 0x7f6d0c0016f0 con 0x7f6d2015d780 2024-05-09T04:34:02.352 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.349+0000 7f6d1e000640 1 -- 172.21.15.145:0/7820477 >> v1:172.21.15.145:6789/0 conn(0x7f6d2015d780 legacy=0x7f6d20160c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:02.352 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.349+0000 7f6d1e000640 1 --2- 172.21.15.145:0/7820477 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d04003a00 0x7f6d04003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:02.352 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.349+0000 7f6d1e000640 1 --2- 172.21.15.145:0/7820477 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d04004800 0x7f6d04004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:02.352 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.349+0000 7f6d1e000640 1 --2- 172.21.15.145:0/7820477 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6d200b4910 0x7f6d040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:02.352 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.349+0000 7f6d1e000640 1 -- 172.21.15.145:0/7820477 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6d20161360 con 0x7f6d04004800 2024-05-09T04:34:02.352 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.349+0000 7f6d1e000640 1 -- 172.21.15.145:0/7820477 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6d201615c0 con 0x7f6d04003a00 2024-05-09T04:34:02.352 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.349+0000 7f6d1e000640 1 -- 172.21.15.145:0/7820477 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f6d0400bb50 con 0x7f6d200b4910 2024-05-09T04:34:02.352 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.349+0000 7f6d1f400640 1 --2- 172.21.15.145:0/7820477 >> v2:172.21.15.192:3300/0 conn(0x7f6d2006e860 0x7f6d2006ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:34:02.352 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.350+0000 7f6d1ea00640 1 --2- 172.21.15.145:0/7820477 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d04004800 0x7f6d04004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:02.352 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.350+0000 7f6d1fe00640 1 --2- 172.21.15.145:0/7820477 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6d200b4910 0x7f6d040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:02.352 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.350+0000 7f6d1ea00640 1 -- 172.21.15.145:0/7820477 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6d200b4910 msgr2=0x7f6d040051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:02.353 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.350+0000 7f6d1ea00640 1 --2- 172.21.15.145:0/7820477 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6d200b4910 0x7f6d040051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:02.353 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.350+0000 7f6d1ea00640 1 -- 172.21.15.145:0/7820477 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d04003a00 msgr2=0x7f6d04003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:02.353 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.350+0000 7f6d1ea00640 1 --2- 172.21.15.145:0/7820477 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d04003a00 0x7f6d04003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:02.353 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.350+0000 7f6d1ea00640 1 -- 172.21.15.145:0/7820477 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d04003520 con 0x7f6d04004800 2024-05-09T04:34:02.353 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.351+0000 7f6d1ea00640 1 --2- 172.21.15.145:0/7820477 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d04004800 0x7f6d04004c80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c00c210 tx=0x7f6d04005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b535a4ded4e51d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:02.353 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.351+0000 7f6d1e000640 1 -- 172.21.15.145:0/7820477 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c007d30 con 0x7f6d04004800 2024-05-09T04:34:02.353 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.351+0000 7f6d1e000640 1 -- 172.21.15.145:0/7820477 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d0c005ad0 con 0x7f6d04004800 2024-05-09T04:34:02.353 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.351+0000 7f6d1e000640 1 -- 172.21.15.145:0/7820477 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c007070 con 0x7f6d04004800 2024-05-09T04:34:02.354 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.352+0000 7f6d26600640 1 -- 172.21.15.145:0/7820477 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d04004800 msgr2=0x7f6d04004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:02.355 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.352+0000 7f6d26600640 1 --2- 172.21.15.145:0/7820477 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d04004800 0x7f6d04004c80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c00c210 tx=0x7f6d04005830 comp rx=0 tx=0).stop 2024-05-09T04:34:02.355 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.352+0000 7f6d1fe00640 1 -- 172.21.15.145:0/7820477 reap_dead start 2024-05-09T04:34:02.355 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.352+0000 7f6d26600640 1 -- 172.21.15.145:0/7820477 shutdown_connections 2024-05-09T04:34:02.355 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.352+0000 7f6d26600640 1 -- 172.21.15.145:0/7820477 >> 172.21.15.145:0/7820477 conn(0x7f6d2006c870 msgr2=0x7f6d2006cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:02.355 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.352+0000 7f6d26600640 1 -- 172.21.15.145:0/7820477 shutdown_connections 2024-05-09T04:34:02.355 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.352+0000 7f6d26600640 1 -- 172.21.15.145:0/7820477 wait complete. 2024-05-09T04:34:02.355 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.353+0000 7f6d26600640 1 Processor -- start 2024-05-09T04:34:02.355 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.354+0000 7f6d26600640 1 -- start start 2024-05-09T04:34:02.355 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.354+0000 7f6d26600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d04004800 0x7f6d201506b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:02.355 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.354+0000 7f6d26600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6d200af990 0x7f6d20159e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:02.355 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.355+0000 7f6d26600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d2015a3b0 0x7f6d20072dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:02.355 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.355+0000 7f6d26600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6d20146c00 con 0x7f6d04004800 2024-05-09T04:34:02.355 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.355+0000 7f6d26600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6d200b43f0 con 0x7f6d2015a3b0 2024-05-09T04:34:02.355 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.355+0000 7f6d26600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f6d20146d80 con 0x7f6d200af990 2024-05-09T04:34:02.356 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.355+0000 7f6d1f400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d04004800 0x7f6d201506b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:02.356 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.355+0000 7f6d1f400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d04004800 0x7f6d201506b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:37814/0 (socket says 172.21.15.145:37814) 2024-05-09T04:34:02.357 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.355+0000 7f6d1f400640 1 -- 172.21.15.145:0/569678529 learned_addr learned my addr 172.21.15.145:0/569678529 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:02.357 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.355+0000 7f6d1f400640 1 -- 172.21.15.145:0/569678529 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6d200af990 msgr2=0x7f6d20159e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:02.357 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.355+0000 7f6d1f400640 1 --2- 172.21.15.145:0/569678529 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6d200af990 0x7f6d20159e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:02.357 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.355+0000 7f6d1f400640 1 -- 172.21.15.145:0/569678529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d2015a3b0 msgr2=0x7f6d20072dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:02.357 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.355+0000 7f6d1f400640 1 --2- 172.21.15.145:0/569678529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d2015a3b0 0x7f6d20072dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:02.357 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.355+0000 7f6d1f400640 1 -- 172.21.15.145:0/569678529 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d20143eb0 con 0x7f6d04004800 2024-05-09T04:34:02.357 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.355+0000 7f6d1f400640 1 --2- 172.21.15.145:0/569678529 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d04004800 0x7f6d201506b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6d1000e7e0 tx=0x7f6d1000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:02.357 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.356+0000 7f6d17400640 1 -- 172.21.15.145:0/569678529 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1000f040 con 0x7f6d04004800 2024-05-09T04:34:02.357 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.356+0000 7f6d17400640 1 -- 172.21.15.145:0/569678529 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d10004230 con 0x7f6d04004800 2024-05-09T04:34:02.357 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.356+0000 7f6d17400640 1 -- 172.21.15.145:0/569678529 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1000c6d0 con 0x7f6d04004800 2024-05-09T04:34:02.358 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.356+0000 7f6d26600640 1 -- 172.21.15.145:0/569678529 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d2015a850 con 0x7f6d04004800 2024-05-09T04:34:02.358 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.356+0000 7f6d26600640 1 -- 172.21.15.145:0/569678529 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d2015ae10 con 0x7f6d04004800 2024-05-09T04:34:02.358 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.357+0000 7f6d26600640 1 -- 172.21.15.145:0/569678529 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d2006a310 con 0x7f6d04004800 2024-05-09T04:34:02.358 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.358+0000 7f6d17400640 1 -- 172.21.15.145:0/569678529 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6d1000c870 con 0x7f6d04004800 2024-05-09T04:34:02.359 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.359+0000 7f6d17400640 1 --2- 172.21.15.145:0/569678529 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f6d000762d0 0x7f6d00078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:02.360 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.359+0000 7f6d17400640 1 -- 172.21.15.145:0/569678529 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6d100973e0 con 0x7f6d04004800 2024-05-09T04:34:02.360 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.359+0000 7f6d1ea00640 1 --2- 172.21.15.145:0/569678529 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f6d000762d0 0x7f6d00078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:02.360 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.360+0000 7f6d1ea00640 1 --2- 172.21.15.145:0/569678529 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f6d000762d0 0x7f6d00078790 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c032000 tx=0x7f6d0c007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:02.362 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.363+0000 7f6d17400640 1 -- 172.21.15.145:0/569678529 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6d20143eb0 con 0x7f6d04004800 2024-05-09T04:34:02.549 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.549+0000 7f6d26600640 1 -- 172.21.15.145:0/569678529 --> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6d2006d460 con 0x7f6d000762d0 2024-05-09T04:34:02.567 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.567+0000 7f6d17400640 1 -- 172.21.15.145:0/569678529 <== mgr.4104 v2:172.21.15.145:6832/2725855396 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251825 (secure 0 0 0) 0x7f6d0c07e7d0 con 0x7f6d000762d0 2024-05-09T04:34:02.571 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:34:02.572 INFO:teuthology.orchestra.run.smithi145.stderr:dumped all 2024-05-09T04:34:02.573 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.572+0000 7f6d26600640 1 -- 172.21.15.145:0/569678529 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f6d000762d0 msgr2=0x7f6d00078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:02.573 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.572+0000 7f6d26600640 1 --2- 172.21.15.145:0/569678529 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f6d000762d0 0x7f6d00078790 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c032000 tx=0x7f6d0c007000 comp rx=0 tx=0).stop 2024-05-09T04:34:02.574 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.572+0000 7f6d26600640 1 -- 172.21.15.145:0/569678529 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d04004800 msgr2=0x7f6d201506b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:02.574 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.572+0000 7f6d26600640 1 --2- 172.21.15.145:0/569678529 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d04004800 0x7f6d201506b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6d1000e7e0 tx=0x7f6d1000eca0 comp rx=0 tx=0).stop 2024-05-09T04:34:02.574 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.572+0000 7f6d26600640 1 -- 172.21.15.145:0/569678529 shutdown_connections 2024-05-09T04:34:02.574 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.572+0000 7f6d26600640 1 --2- 172.21.15.145:0/569678529 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f6d000762d0 0x7f6d00078790 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:02.574 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.572+0000 7f6d26600640 1 --2- 172.21.15.145:0/569678529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d2015a3b0 0x7f6d20072dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:02.574 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.572+0000 7f6d26600640 1 --2- 172.21.15.145:0/569678529 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6d200af990 0x7f6d20159e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:02.574 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.572+0000 7f6d26600640 1 --2- 172.21.15.145:0/569678529 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d04004800 0x7f6d201506b0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:02.574 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.572+0000 7f6d26600640 1 -- 172.21.15.145:0/569678529 >> 172.21.15.145:0/569678529 conn(0x7f6d2006c870 msgr2=0x7f6d2015b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:02.574 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.572+0000 7f6d26600640 1 -- 172.21.15.145:0/569678529 shutdown_connections 2024-05-09T04:34:02.574 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.572+0000 7f6d26600640 1 -- 172.21.15.145:0/569678529 wait complete. 2024-05-09T04:34:02.589 INFO:teuthology.orchestra.run.smithi145.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-05-09T04:34:01.759331+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219012,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755708,"statfs":{"total":773094113280,"available":772869844992,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.592729"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.879754+0000","last_change":"2024-05-09T04:33:52.200596+0000","last_active":"2024-05-09T04:33:54.879754+0000","last_peered":"2024-05-09T04:33:54.879754+0000","last_clean":"2024-05-09T04:33:54.879754+0000","last_became_active":"2024-05-09T04:33:52.199781+0000","last_became_peered":"2024-05-09T04:33:52.199781+0000","last_unstale":"2024-05-09T04:33:54.879754+0000","last_undegraded":"2024-05-09T04:33:54.879754+0000","last_fullsized":"2024-05-09T04:33:54.879754+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:46:44.064608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.154485+0000","last_change":"2024-05-09T04:33:52.186325+0000","last_active":"2024-05-09T04:33:54.154485+0000","last_peered":"2024-05-09T04:33:54.154485+0000","last_clean":"2024-05-09T04:33:54.154485+0000","last_became_active":"2024-05-09T04:33:52.186047+0000","last_became_peered":"2024-05-09T04:33:52.186047+0000","last_unstale":"2024-05-09T04:33:54.154485+0000","last_undegraded":"2024-05-09T04:33:54.154485+0000","last_fullsized":"2024-05-09T04:33:54.154485+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:24:37.669022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.194578+0000","last_change":"2024-05-09T04:33:52.187398+0000","last_active":"2024-05-09T04:33:53.194578+0000","last_peered":"2024-05-09T04:33:53.194578+0000","last_clean":"2024-05-09T04:33:53.194578+0000","last_became_active":"2024-05-09T04:33:52.186979+0000","last_became_peered":"2024-05-09T04:33:52.186979+0000","last_unstale":"2024-05-09T04:33:53.194578+0000","last_undegraded":"2024-05-09T04:33:53.194578+0000","last_fullsized":"2024-05-09T04:33:53.194578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:34:29.852986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.211667+0000","last_change":"2024-05-09T04:33:52.210098+0000","last_active":"2024-05-09T04:33:54.211667+0000","last_peered":"2024-05-09T04:33:54.211667+0000","last_clean":"2024-05-09T04:33:54.211667+0000","last_became_active":"2024-05-09T04:33:52.209575+0000","last_became_peered":"2024-05-09T04:33:52.209575+0000","last_unstale":"2024-05-09T04:33:54.211667+0000","last_undegraded":"2024-05-09T04:33:54.211667+0000","last_fullsized":"2024-05-09T04:33:54.211667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:24:41.478222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.878129+0000","last_change":"2024-05-09T04:33:52.195969+0000","last_active":"2024-05-09T04:33:54.878129+0000","last_peered":"2024-05-09T04:33:54.878129+0000","last_clean":"2024-05-09T04:33:54.878129+0000","last_became_active":"2024-05-09T04:33:52.195578+0000","last_became_peered":"2024-05-09T04:33:52.195578+0000","last_unstale":"2024-05-09T04:33:54.878129+0000","last_undegraded":"2024-05-09T04:33:54.878129+0000","last_fullsized":"2024-05-09T04:33:54.878129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:56.927838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.211122+0000","last_change":"2024-05-09T04:33:52.198619+0000","last_active":"2024-05-09T04:33:54.211122+0000","last_peered":"2024-05-09T04:33:54.211122+0000","last_clean":"2024-05-09T04:33:54.211122+0000","last_became_active":"2024-05-09T04:33:52.198345+0000","last_became_peered":"2024-05-09T04:33:52.198345+0000","last_unstale":"2024-05-09T04:33:54.211122+0000","last_undegraded":"2024-05-09T04:33:54.211122+0000","last_fullsized":"2024-05-09T04:33:54.211122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:24:37.669022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.878144+0000","last_change":"2024-05-09T04:33:52.202397+0000","last_active":"2024-05-09T04:33:54.878144+0000","last_peered":"2024-05-09T04:33:54.878144+0000","last_clean":"2024-05-09T04:33:54.878144+0000","last_became_active":"2024-05-09T04:33:52.202176+0000","last_became_peered":"2024-05-09T04:33:52.202176+0000","last_unstale":"2024-05-09T04:33:54.878144+0000","last_undegraded":"2024-05-09T04:33:54.878144+0000","last_fullsized":"2024-05-09T04:33:54.878144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:02:17.813339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.196047+0000","last_change":"2024-05-09T04:33:52.207259+0000","last_active":"2024-05-09T04:33:53.196047+0000","last_peered":"2024-05-09T04:33:53.196047+0000","last_clean":"2024-05-09T04:33:53.196047+0000","last_became_active":"2024-05-09T04:33:52.204206+0000","last_became_peered":"2024-05-09T04:33:52.204206+0000","last_unstale":"2024-05-09T04:33:53.196047+0000","last_undegraded":"2024-05-09T04:33:53.196047+0000","last_fullsized":"2024-05-09T04:33:53.196047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:35:43.478085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.879759+0000","last_change":"2024-05-09T04:33:52.205582+0000","last_active":"2024-05-09T04:33:54.879759+0000","last_peered":"2024-05-09T04:33:54.879759+0000","last_clean":"2024-05-09T04:33:54.879759+0000","last_became_active":"2024-05-09T04:33:52.205237+0000","last_became_peered":"2024-05-09T04:33:52.205237+0000","last_unstale":"2024-05-09T04:33:54.879759+0000","last_undegraded":"2024-05-09T04:33:54.879759+0000","last_fullsized":"2024-05-09T04:33:54.879759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:42.767983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.686517+0000","last_change":"2024-05-09T04:33:52.198058+0000","last_active":"2024-05-09T04:33:53.686517+0000","last_peered":"2024-05-09T04:33:53.686517+0000","last_clean":"2024-05-09T04:33:53.686517+0000","last_became_active":"2024-05-09T04:33:52.197701+0000","last_became_peered":"2024-05-09T04:33:52.197701+0000","last_unstale":"2024-05-09T04:33:53.686517+0000","last_undegraded":"2024-05-09T04:33:53.686517+0000","last_fullsized":"2024-05-09T04:33:53.686517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:24:37.669022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.686063+0000","last_change":"2024-05-09T04:33:52.194755+0000","last_active":"2024-05-09T04:33:53.686063+0000","last_peered":"2024-05-09T04:33:53.686063+0000","last_clean":"2024-05-09T04:33:53.686063+0000","last_became_active":"2024-05-09T04:33:52.194270+0000","last_became_peered":"2024-05-09T04:33:52.194270+0000","last_unstale":"2024-05-09T04:33:53.686063+0000","last_undegraded":"2024-05-09T04:33:53.686063+0000","last_fullsized":"2024-05-09T04:33:53.686063+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:59:28.002585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.195766+0000","last_change":"2024-05-09T04:33:52.200331+0000","last_active":"2024-05-09T04:33:53.195766+0000","last_peered":"2024-05-09T04:33:53.195766+0000","last_clean":"2024-05-09T04:33:53.195766+0000","last_became_active":"2024-05-09T04:33:52.199739+0000","last_became_peered":"2024-05-09T04:33:52.199739+0000","last_unstale":"2024-05-09T04:33:53.195766+0000","last_undegraded":"2024-05-09T04:33:53.195766+0000","last_fullsized":"2024-05-09T04:33:53.195766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:29:12.420301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.196580+0000","last_change":"2024-05-09T04:33:52.211817+0000","last_active":"2024-05-09T04:33:53.196580+0000","last_peered":"2024-05-09T04:33:53.196580+0000","last_clean":"2024-05-09T04:33:53.196580+0000","last_became_active":"2024-05-09T04:33:52.211533+0000","last_became_peered":"2024-05-09T04:33:52.211533+0000","last_unstale":"2024-05-09T04:33:53.196580+0000","last_undegraded":"2024-05-09T04:33:53.196580+0000","last_fullsized":"2024-05-09T04:33:53.196580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:34:29.852986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.193922+0000","last_change":"2024-05-09T04:33:52.177443+0000","last_active":"2024-05-09T04:33:53.193922+0000","last_peered":"2024-05-09T04:33:53.193922+0000","last_clean":"2024-05-09T04:33:53.193922+0000","last_became_active":"2024-05-09T04:33:52.177115+0000","last_became_peered":"2024-05-09T04:33:52.177115+0000","last_unstale":"2024-05-09T04:33:53.193922+0000","last_undegraded":"2024-05-09T04:33:53.193922+0000","last_fullsized":"2024-05-09T04:33:53.193922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:33:25.042543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.154598+0000","last_change":"2024-05-09T04:33:52.209143+0000","last_active":"2024-05-09T04:33:54.154598+0000","last_peered":"2024-05-09T04:33:54.154598+0000","last_clean":"2024-05-09T04:33:54.154598+0000","last_became_active":"2024-05-09T04:33:52.206547+0000","last_became_peered":"2024-05-09T04:33:52.206547+0000","last_unstale":"2024-05-09T04:33:54.154598+0000","last_undegraded":"2024-05-09T04:33:54.154598+0000","last_fullsized":"2024-05-09T04:33:54.154598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:02:17.813339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.211033+0000","last_change":"2024-05-09T04:33:52.198040+0000","last_active":"2024-05-09T04:33:54.211033+0000","last_peered":"2024-05-09T04:33:54.211033+0000","last_clean":"2024-05-09T04:33:54.211033+0000","last_became_active":"2024-05-09T04:33:52.197706+0000","last_became_peered":"2024-05-09T04:33:52.197706+0000","last_unstale":"2024-05-09T04:33:54.211033+0000","last_undegraded":"2024-05-09T04:33:54.211033+0000","last_fullsized":"2024-05-09T04:33:54.211033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:17.801441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.212667+0000","last_change":"2024-05-09T04:33:52.195546+0000","last_active":"2024-05-09T04:33:54.212667+0000","last_peered":"2024-05-09T04:33:54.212667+0000","last_clean":"2024-05-09T04:33:54.212667+0000","last_became_active":"2024-05-09T04:33:52.195138+0000","last_became_peered":"2024-05-09T04:33:52.195138+0000","last_unstale":"2024-05-09T04:33:54.212667+0000","last_undegraded":"2024-05-09T04:33:54.212667+0000","last_fullsized":"2024-05-09T04:33:54.212667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:42.767983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.732773+0000","last_change":"2024-05-09T04:33:52.192303+0000","last_active":"2024-05-09T04:33:54.732773+0000","last_peered":"2024-05-09T04:33:54.732773+0000","last_clean":"2024-05-09T04:33:54.732773+0000","last_became_active":"2024-05-09T04:33:52.191878+0000","last_became_peered":"2024-05-09T04:33:52.191878+0000","last_unstale":"2024-05-09T04:33:54.732773+0000","last_undegraded":"2024-05-09T04:33:54.732773+0000","last_fullsized":"2024-05-09T04:33:54.732773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:22:41.930610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.879711+0000","last_change":"2024-05-09T04:33:52.190589+0000","last_active":"2024-05-09T04:33:54.879711+0000","last_peered":"2024-05-09T04:33:54.879711+0000","last_clean":"2024-05-09T04:33:54.879711+0000","last_became_active":"2024-05-09T04:33:52.190312+0000","last_became_peered":"2024-05-09T04:33:52.190312+0000","last_unstale":"2024-05-09T04:33:54.879711+0000","last_undegraded":"2024-05-09T04:33:54.879711+0000","last_fullsized":"2024-05-09T04:33:54.879711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:19:08.318788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.686133+0000","last_change":"2024-05-09T04:33:52.195370+0000","last_active":"2024-05-09T04:33:53.686133+0000","last_peered":"2024-05-09T04:33:53.686133+0000","last_clean":"2024-05-09T04:33:53.686133+0000","last_became_active":"2024-05-09T04:33:52.194414+0000","last_became_peered":"2024-05-09T04:33:52.194414+0000","last_unstale":"2024-05-09T04:33:53.686133+0000","last_undegraded":"2024-05-09T04:33:53.686133+0000","last_fullsized":"2024-05-09T04:33:53.686133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:29:08.279802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.193862+0000","last_change":"2024-05-09T04:33:52.201922+0000","last_active":"2024-05-09T04:33:53.193862+0000","last_peered":"2024-05-09T04:33:53.193862+0000","last_clean":"2024-05-09T04:33:53.193862+0000","last_became_active":"2024-05-09T04:33:52.201635+0000","last_became_peered":"2024-05-09T04:33:52.201635+0000","last_unstale":"2024-05-09T04:33:53.193862+0000","last_undegraded":"2024-05-09T04:33:53.193862+0000","last_fullsized":"2024-05-09T04:33:53.193862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:27:41.648285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.211533+0000","last_change":"2024-05-09T04:33:52.195480+0000","last_active":"2024-05-09T04:33:54.211533+0000","last_peered":"2024-05-09T04:33:54.211533+0000","last_clean":"2024-05-09T04:33:54.211533+0000","last_became_active":"2024-05-09T04:33:52.195195+0000","last_became_peered":"2024-05-09T04:33:52.195195+0000","last_unstale":"2024-05-09T04:33:54.211533+0000","last_undegraded":"2024-05-09T04:33:54.211533+0000","last_fullsized":"2024-05-09T04:33:54.211533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:02:17.813339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.196691+0000","last_change":"2024-05-09T04:33:52.208038+0000","last_active":"2024-05-09T04:33:53.196691+0000","last_peered":"2024-05-09T04:33:53.196691+0000","last_clean":"2024-05-09T04:33:53.196691+0000","last_became_active":"2024-05-09T04:33:52.207755+0000","last_became_peered":"2024-05-09T04:33:52.207755+0000","last_unstale":"2024-05-09T04:33:53.196691+0000","last_undegraded":"2024-05-09T04:33:53.196691+0000","last_fullsized":"2024-05-09T04:33:53.196691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:59:28.002585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.211317+0000","last_change":"2024-05-09T04:33:52.209997+0000","last_active":"2024-05-09T04:33:54.211317+0000","last_peered":"2024-05-09T04:33:54.211317+0000","last_clean":"2024-05-09T04:33:54.211317+0000","last_became_active":"2024-05-09T04:33:52.209702+0000","last_became_peered":"2024-05-09T04:33:52.209702+0000","last_unstale":"2024-05-09T04:33:54.211317+0000","last_undegraded":"2024-05-09T04:33:54.211317+0000","last_fullsized":"2024-05-09T04:33:54.211317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:56.927838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.197678+0000","last_change":"2024-05-09T04:33:52.207838+0000","last_active":"2024-05-09T04:33:53.197678+0000","last_peered":"2024-05-09T04:33:53.197678+0000","last_clean":"2024-05-09T04:33:53.197678+0000","last_became_active":"2024-05-09T04:33:52.207601+0000","last_became_peered":"2024-05-09T04:33:52.207601+0000","last_unstale":"2024-05-09T04:33:53.197678+0000","last_undegraded":"2024-05-09T04:33:53.197678+0000","last_fullsized":"2024-05-09T04:33:53.197678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:56.927838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.686401+0000","last_change":"2024-05-09T04:33:52.196143+0000","last_active":"2024-05-09T04:33:53.686401+0000","last_peered":"2024-05-09T04:33:53.686401+0000","last_clean":"2024-05-09T04:33:53.686401+0000","last_became_active":"2024-05-09T04:33:52.195814+0000","last_became_peered":"2024-05-09T04:33:52.195814+0000","last_unstale":"2024-05-09T04:33:53.686401+0000","last_undegraded":"2024-05-09T04:33:53.686401+0000","last_fullsized":"2024-05-09T04:33:53.686401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:17.801441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.154429+0000","last_change":"2024-05-09T04:33:52.205497+0000","last_active":"2024-05-09T04:33:54.154429+0000","last_peered":"2024-05-09T04:33:54.154429+0000","last_clean":"2024-05-09T04:33:54.154429+0000","last_became_active":"2024-05-09T04:33:52.205219+0000","last_became_peered":"2024-05-09T04:33:52.205219+0000","last_unstale":"2024-05-09T04:33:54.154429+0000","last_undegraded":"2024-05-09T04:33:54.154429+0000","last_fullsized":"2024-05-09T04:33:54.154429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:17.801441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.197606+0000","last_change":"2024-05-09T04:33:52.213356+0000","last_active":"2024-05-09T04:33:53.197606+0000","last_peered":"2024-05-09T04:33:53.197606+0000","last_clean":"2024-05-09T04:33:53.197606+0000","last_became_active":"2024-05-09T04:33:52.212783+0000","last_became_peered":"2024-05-09T04:33:52.212783+0000","last_unstale":"2024-05-09T04:33:53.197606+0000","last_undegraded":"2024-05-09T04:33:53.197606+0000","last_fullsized":"2024-05-09T04:33:53.197606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:24:37.669022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.194008+0000","last_change":"2024-05-09T04:33:52.209567+0000","last_active":"2024-05-09T04:33:53.194008+0000","last_peered":"2024-05-09T04:33:53.194008+0000","last_clean":"2024-05-09T04:33:53.194008+0000","last_became_active":"2024-05-09T04:33:52.209260+0000","last_became_peered":"2024-05-09T04:33:52.209260+0000","last_unstale":"2024-05-09T04:33:53.194008+0000","last_undegraded":"2024-05-09T04:33:53.194008+0000","last_fullsized":"2024-05-09T04:33:53.194008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:35:43.478085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.211635+0000","last_change":"2024-05-09T04:33:52.210744+0000","last_active":"2024-05-09T04:33:54.211635+0000","last_peered":"2024-05-09T04:33:54.211635+0000","last_clean":"2024-05-09T04:33:54.211635+0000","last_became_active":"2024-05-09T04:33:52.210421+0000","last_became_peered":"2024-05-09T04:33:52.210421+0000","last_unstale":"2024-05-09T04:33:54.211635+0000","last_undegraded":"2024-05-09T04:33:54.211635+0000","last_fullsized":"2024-05-09T04:33:54.211635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:13:19.513750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.732744+0000","last_change":"2024-05-09T04:33:52.193832+0000","last_active":"2024-05-09T04:33:54.732744+0000","last_peered":"2024-05-09T04:33:54.732744+0000","last_clean":"2024-05-09T04:33:54.732744+0000","last_became_active":"2024-05-09T04:33:52.193531+0000","last_became_peered":"2024-05-09T04:33:52.193531+0000","last_unstale":"2024-05-09T04:33:54.732744+0000","last_undegraded":"2024-05-09T04:33:54.732744+0000","last_fullsized":"2024-05-09T04:33:54.732744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:19:44.181447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.685556+0000","last_change":"2024-05-09T04:33:52.198677+0000","last_active":"2024-05-09T04:33:53.685556+0000","last_peered":"2024-05-09T04:33:53.685556+0000","last_clean":"2024-05-09T04:33:53.685556+0000","last_became_active":"2024-05-09T04:33:52.198334+0000","last_became_peered":"2024-05-09T04:33:52.198334+0000","last_unstale":"2024-05-09T04:33:53.685556+0000","last_undegraded":"2024-05-09T04:33:53.685556+0000","last_fullsized":"2024-05-09T04:33:53.685556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:53:58.795910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.194924+0000","last_change":"2024-05-09T04:33:52.192680+0000","last_active":"2024-05-09T04:33:53.194924+0000","last_peered":"2024-05-09T04:33:53.194924+0000","last_clean":"2024-05-09T04:33:53.194924+0000","last_became_active":"2024-05-09T04:33:52.192454+0000","last_became_peered":"2024-05-09T04:33:52.192454+0000","last_unstale":"2024-05-09T04:33:53.194924+0000","last_undegraded":"2024-05-09T04:33:53.194924+0000","last_fullsized":"2024-05-09T04:33:53.194924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:50:52.026927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.196881+0000","last_change":"2024-05-09T04:33:51.177063+0000","last_active":"2024-05-09T04:33:53.196881+0000","last_peered":"2024-05-09T04:33:53.196881+0000","last_clean":"2024-05-09T04:33:53.196881+0000","last_became_active":"2024-05-09T04:33:51.176788+0000","last_became_peered":"2024-05-09T04:33:51.176788+0000","last_unstale":"2024-05-09T04:33:53.196881+0000","last_undegraded":"2024-05-09T04:33:53.196881+0000","last_fullsized":"2024-05-09T04:33:53.196881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:04:10.949412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.194154+0000","last_change":"2024-05-09T04:33:52.190946+0000","last_active":"2024-05-09T04:33:53.194154+0000","last_peered":"2024-05-09T04:33:53.194154+0000","last_clean":"2024-05-09T04:33:53.194154+0000","last_became_active":"2024-05-09T04:33:52.190598+0000","last_became_peered":"2024-05-09T04:33:52.190598+0000","last_unstale":"2024-05-09T04:33:53.194154+0000","last_undegraded":"2024-05-09T04:33:53.194154+0000","last_fullsized":"2024-05-09T04:33:53.194154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:53:58.795910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.685499+0000","last_change":"2024-05-09T04:33:51.177930+0000","last_active":"2024-05-09T04:33:53.685499+0000","last_peered":"2024-05-09T04:33:53.685499+0000","last_clean":"2024-05-09T04:33:53.685499+0000","last_became_active":"2024-05-09T04:33:51.174452+0000","last_became_peered":"2024-05-09T04:33:51.174452+0000","last_unstale":"2024-05-09T04:33:53.685499+0000","last_undegraded":"2024-05-09T04:33:53.685499+0000","last_fullsized":"2024-05-09T04:33:53.685499+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:26:18.474267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.196968+0000","last_change":"2024-05-09T04:33:51.169721+0000","last_active":"2024-05-09T04:33:53.196968+0000","last_peered":"2024-05-09T04:33:53.196968+0000","last_clean":"2024-05-09T04:33:53.196968+0000","last_became_active":"2024-05-09T04:33:51.169422+0000","last_became_peered":"2024-05-09T04:33:51.169422+0000","last_unstale":"2024-05-09T04:33:53.196968+0000","last_undegraded":"2024-05-09T04:33:53.196968+0000","last_fullsized":"2024-05-09T04:33:53.196968+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:36:52.136732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.196263+0000","last_change":"2024-05-09T04:33:52.208855+0000","last_active":"2024-05-09T04:33:53.196263+0000","last_peered":"2024-05-09T04:33:53.196263+0000","last_clean":"2024-05-09T04:33:53.196263+0000","last_became_active":"2024-05-09T04:33:52.208588+0000","last_became_peered":"2024-05-09T04:33:52.208588+0000","last_unstale":"2024-05-09T04:33:53.196263+0000","last_undegraded":"2024-05-09T04:33:53.196263+0000","last_fullsized":"2024-05-09T04:33:53.196263+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:53:58.795910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.211333+0000","last_change":"2024-05-09T04:33:51.178180+0000","last_active":"2024-05-09T04:33:54.211333+0000","last_peered":"2024-05-09T04:33:54.211333+0000","last_clean":"2024-05-09T04:33:54.211333+0000","last_became_active":"2024-05-09T04:33:51.175516+0000","last_became_peered":"2024-05-09T04:33:51.175516+0000","last_unstale":"2024-05-09T04:33:54.211333+0000","last_undegraded":"2024-05-09T04:33:54.211333+0000","last_fullsized":"2024-05-09T04:33:54.211333+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:26:18.474267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.154507+0000","last_change":"2024-05-09T04:33:52.196488+0000","last_active":"2024-05-09T04:33:54.154507+0000","last_peered":"2024-05-09T04:33:54.154507+0000","last_clean":"2024-05-09T04:33:54.154507+0000","last_became_active":"2024-05-09T04:33:52.196175+0000","last_became_peered":"2024-05-09T04:33:52.196175+0000","last_unstale":"2024-05-09T04:33:54.154507+0000","last_undegraded":"2024-05-09T04:33:54.154507+0000","last_fullsized":"2024-05-09T04:33:54.154507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:59:56.927838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.686280+0000","last_change":"2024-05-09T04:33:51.185310+0000","last_active":"2024-05-09T04:33:53.686280+0000","last_peered":"2024-05-09T04:33:53.686280+0000","last_clean":"2024-05-09T04:33:53.686280+0000","last_became_active":"2024-05-09T04:33:51.185016+0000","last_became_peered":"2024-05-09T04:33:51.185016+0000","last_unstale":"2024-05-09T04:33:53.686280+0000","last_undegraded":"2024-05-09T04:33:53.686280+0000","last_fullsized":"2024-05-09T04:33:53.686280+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:50:51.015799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.212510+0000","last_change":"2024-05-09T04:33:52.193316+0000","last_active":"2024-05-09T04:33:54.212510+0000","last_peered":"2024-05-09T04:33:54.212510+0000","last_clean":"2024-05-09T04:33:54.212510+0000","last_became_active":"2024-05-09T04:33:52.192939+0000","last_became_peered":"2024-05-09T04:33:52.192939+0000","last_unstale":"2024-05-09T04:33:54.212510+0000","last_undegraded":"2024-05-09T04:33:54.212510+0000","last_fullsized":"2024-05-09T04:33:54.212510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:46:44.064608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.212393+0000","last_change":"2024-05-09T04:33:51.183116+0000","last_active":"2024-05-09T04:33:54.212393+0000","last_peered":"2024-05-09T04:33:54.212393+0000","last_clean":"2024-05-09T04:33:54.212393+0000","last_became_active":"2024-05-09T04:33:51.182750+0000","last_became_peered":"2024-05-09T04:33:51.182750+0000","last_unstale":"2024-05-09T04:33:54.212393+0000","last_undegraded":"2024-05-09T04:33:54.212393+0000","last_fullsized":"2024-05-09T04:33:54.212393+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:04:10.949412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.195562+0000","last_change":"2024-05-09T04:33:52.201494+0000","last_active":"2024-05-09T04:33:53.195562+0000","last_peered":"2024-05-09T04:33:53.195562+0000","last_clean":"2024-05-09T04:33:53.195562+0000","last_became_active":"2024-05-09T04:33:52.197939+0000","last_became_peered":"2024-05-09T04:33:52.197939+0000","last_unstale":"2024-05-09T04:33:53.195562+0000","last_undegraded":"2024-05-09T04:33:53.195562+0000","last_fullsized":"2024-05-09T04:33:53.195562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:29:08.279802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.154034+0000","last_change":"2024-05-09T04:33:51.178564+0000","last_active":"2024-05-09T04:33:54.154034+0000","last_peered":"2024-05-09T04:33:54.154034+0000","last_clean":"2024-05-09T04:33:54.154034+0000","last_became_active":"2024-05-09T04:33:51.177732+0000","last_became_peered":"2024-05-09T04:33:51.177732+0000","last_unstale":"2024-05-09T04:33:54.154034+0000","last_undegraded":"2024-05-09T04:33:54.154034+0000","last_fullsized":"2024-05-09T04:33:54.154034+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:50:51.015799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.196170+0000","last_change":"2024-05-09T04:33:52.200019+0000","last_active":"2024-05-09T04:33:53.196170+0000","last_peered":"2024-05-09T04:33:53.196170+0000","last_clean":"2024-05-09T04:33:53.196170+0000","last_became_active":"2024-05-09T04:33:52.199279+0000","last_became_peered":"2024-05-09T04:33:52.199279+0000","last_unstale":"2024-05-09T04:33:53.196170+0000","last_undegraded":"2024-05-09T04:33:53.196170+0000","last_fullsized":"2024-05-09T04:33:53.196170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:19:44.181447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.685981+0000","last_change":"2024-05-09T04:33:51.173622+0000","last_active":"2024-05-09T04:33:53.685981+0000","last_peered":"2024-05-09T04:33:53.685981+0000","last_clean":"2024-05-09T04:33:53.685981+0000","last_became_active":"2024-05-09T04:33:51.173158+0000","last_became_peered":"2024-05-09T04:33:51.173158+0000","last_unstale":"2024-05-09T04:33:53.685981+0000","last_undegraded":"2024-05-09T04:33:53.685981+0000","last_fullsized":"2024-05-09T04:33:53.685981+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:08:08.955016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.878757+0000","last_change":"2024-05-09T04:33:52.214221+0000","last_active":"2024-05-09T04:33:54.878757+0000","last_peered":"2024-05-09T04:33:54.878757+0000","last_clean":"2024-05-09T04:33:54.878757+0000","last_became_active":"2024-05-09T04:33:52.213773+0000","last_became_peered":"2024-05-09T04:33:52.213773+0000","last_unstale":"2024-05-09T04:33:54.878757+0000","last_undegraded":"2024-05-09T04:33:54.878757+0000","last_fullsized":"2024-05-09T04:33:54.878757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:10:42.676456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.194274+0000","last_change":"2024-05-09T04:33:51.172542+0000","last_active":"2024-05-09T04:33:53.194274+0000","last_peered":"2024-05-09T04:33:53.194274+0000","last_clean":"2024-05-09T04:33:53.194274+0000","last_became_active":"2024-05-09T04:33:51.172232+0000","last_became_peered":"2024-05-09T04:33:51.172232+0000","last_unstale":"2024-05-09T04:33:53.194274+0000","last_undegraded":"2024-05-09T04:33:53.194274+0000","last_fullsized":"2024-05-09T04:33:53.194274+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:15.760572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.194183+0000","last_change":"2024-05-09T04:33:52.195027+0000","last_active":"2024-05-09T04:33:53.194183+0000","last_peered":"2024-05-09T04:33:53.194183+0000","last_clean":"2024-05-09T04:33:53.194183+0000","last_became_active":"2024-05-09T04:33:52.194590+0000","last_became_peered":"2024-05-09T04:33:52.194590+0000","last_unstale":"2024-05-09T04:33:53.194183+0000","last_undegraded":"2024-05-09T04:33:53.194183+0000","last_fullsized":"2024-05-09T04:33:53.194183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:34:40.257159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.211870+0000","last_change":"2024-05-09T04:33:51.177887+0000","last_active":"2024-05-09T04:33:54.211870+0000","last_peered":"2024-05-09T04:33:54.211870+0000","last_clean":"2024-05-09T04:33:54.211870+0000","last_became_active":"2024-05-09T04:33:51.175721+0000","last_became_peered":"2024-05-09T04:33:51.175721+0000","last_unstale":"2024-05-09T04:33:54.211870+0000","last_undegraded":"2024-05-09T04:33:54.211870+0000","last_fullsized":"2024-05-09T04:33:54.211870+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:02:50.965016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.878431+0000","last_change":"2024-05-09T04:33:52.211924+0000","last_active":"2024-05-09T04:33:54.878431+0000","last_peered":"2024-05-09T04:33:54.878431+0000","last_clean":"2024-05-09T04:33:54.878431+0000","last_became_active":"2024-05-09T04:33:52.211546+0000","last_became_peered":"2024-05-09T04:33:52.211546+0000","last_unstale":"2024-05-09T04:33:54.878431+0000","last_undegraded":"2024-05-09T04:33:54.878431+0000","last_fullsized":"2024-05-09T04:33:54.878431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:24:41.478222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.685619+0000","last_change":"2024-05-09T04:33:51.176697+0000","last_active":"2024-05-09T04:33:53.685619+0000","last_peered":"2024-05-09T04:33:53.685619+0000","last_clean":"2024-05-09T04:33:53.685619+0000","last_became_active":"2024-05-09T04:33:51.176351+0000","last_became_peered":"2024-05-09T04:33:51.176351+0000","last_unstale":"2024-05-09T04:33:53.685619+0000","last_undegraded":"2024-05-09T04:33:53.685619+0000","last_fullsized":"2024-05-09T04:33:53.685619+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:04:10.949412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.685570+0000","last_change":"2024-05-09T04:33:52.187596+0000","last_active":"2024-05-09T04:33:53.685570+0000","last_peered":"2024-05-09T04:33:53.685570+0000","last_clean":"2024-05-09T04:33:53.685570+0000","last_became_active":"2024-05-09T04:33:52.187287+0000","last_became_peered":"2024-05-09T04:33:52.187287+0000","last_unstale":"2024-05-09T04:33:53.685570+0000","last_undegraded":"2024-05-09T04:33:53.685570+0000","last_fullsized":"2024-05-09T04:33:53.685570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:34:40.257159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.732380+0000","last_change":"2024-05-09T04:33:51.166963+0000","last_active":"2024-05-09T04:33:54.732380+0000","last_peered":"2024-05-09T04:33:54.732380+0000","last_clean":"2024-05-09T04:33:54.732380+0000","last_became_active":"2024-05-09T04:33:51.165585+0000","last_became_peered":"2024-05-09T04:33:51.165585+0000","last_unstale":"2024-05-09T04:33:54.732380+0000","last_undegraded":"2024-05-09T04:33:54.732380+0000","last_fullsized":"2024-05-09T04:33:54.732380+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:04:10.949412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.878130+0000","last_change":"2024-05-09T04:33:52.207666+0000","last_active":"2024-05-09T04:33:54.878130+0000","last_peered":"2024-05-09T04:33:54.878130+0000","last_clean":"2024-05-09T04:33:54.878130+0000","last_became_active":"2024-05-09T04:33:52.207429+0000","last_became_peered":"2024-05-09T04:33:52.207429+0000","last_unstale":"2024-05-09T04:33:54.878130+0000","last_undegraded":"2024-05-09T04:33:54.878130+0000","last_fullsized":"2024-05-09T04:33:54.878130+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:13:19.513750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.196757+0000","last_change":"2024-05-09T04:33:51.172081+0000","last_active":"2024-05-09T04:33:53.196757+0000","last_peered":"2024-05-09T04:33:53.196757+0000","last_clean":"2024-05-09T04:33:53.196757+0000","last_became_active":"2024-05-09T04:33:51.171700+0000","last_became_peered":"2024-05-09T04:33:51.171700+0000","last_unstale":"2024-05-09T04:33:53.196757+0000","last_undegraded":"2024-05-09T04:33:53.196757+0000","last_fullsized":"2024-05-09T04:33:53.196757+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:15.760572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.879267+0000","last_change":"2024-05-09T04:33:52.206889+0000","last_active":"2024-05-09T04:33:54.879267+0000","last_peered":"2024-05-09T04:33:54.879267+0000","last_clean":"2024-05-09T04:33:54.879267+0000","last_became_active":"2024-05-09T04:33:52.206452+0000","last_became_peered":"2024-05-09T04:33:52.206452+0000","last_unstale":"2024-05-09T04:33:54.879267+0000","last_undegraded":"2024-05-09T04:33:54.879267+0000","last_fullsized":"2024-05-09T04:33:54.879267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:40:40.749511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.732880+0000","last_change":"2024-05-09T04:33:51.167434+0000","last_active":"2024-05-09T04:33:54.732880+0000","last_peered":"2024-05-09T04:33:54.732880+0000","last_clean":"2024-05-09T04:33:54.732880+0000","last_became_active":"2024-05-09T04:33:51.167117+0000","last_became_peered":"2024-05-09T04:33:51.167117+0000","last_unstale":"2024-05-09T04:33:54.732880+0000","last_undegraded":"2024-05-09T04:33:54.732880+0000","last_fullsized":"2024-05-09T04:33:54.732880+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:02:50.965016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.880269+0000","last_change":"2024-05-09T04:33:52.207191+0000","last_active":"2024-05-09T04:33:54.880269+0000","last_peered":"2024-05-09T04:33:54.880269+0000","last_clean":"2024-05-09T04:33:54.880269+0000","last_became_active":"2024-05-09T04:33:52.206932+0000","last_became_peered":"2024-05-09T04:33:52.206932+0000","last_unstale":"2024-05-09T04:33:54.880269+0000","last_undegraded":"2024-05-09T04:33:54.880269+0000","last_fullsized":"2024-05-09T04:33:54.880269+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:49:29.149981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.154386+0000","last_change":"2024-05-09T04:33:51.163656+0000","last_active":"2024-05-09T04:33:54.154386+0000","last_peered":"2024-05-09T04:33:54.154386+0000","last_clean":"2024-05-09T04:33:54.154386+0000","last_became_active":"2024-05-09T04:33:51.163342+0000","last_became_peered":"2024-05-09T04:33:51.163342+0000","last_unstale":"2024-05-09T04:33:54.154386+0000","last_undegraded":"2024-05-09T04:33:54.154386+0000","last_fullsized":"2024-05-09T04:33:54.154386+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:19:47.722161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.212177+0000","last_change":"2024-05-09T04:33:52.205802+0000","last_active":"2024-05-09T04:33:54.212177+0000","last_peered":"2024-05-09T04:33:54.212177+0000","last_clean":"2024-05-09T04:33:54.212177+0000","last_became_active":"2024-05-09T04:33:52.205470+0000","last_became_peered":"2024-05-09T04:33:52.205470+0000","last_unstale":"2024-05-09T04:33:54.212177+0000","last_undegraded":"2024-05-09T04:33:54.212177+0000","last_fullsized":"2024-05-09T04:33:54.212177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:40:40.749511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.732648+0000","last_change":"2024-05-09T04:33:51.180702+0000","last_active":"2024-05-09T04:33:54.732648+0000","last_peered":"2024-05-09T04:33:54.732648+0000","last_clean":"2024-05-09T04:33:54.732648+0000","last_became_active":"2024-05-09T04:33:51.180322+0000","last_became_peered":"2024-05-09T04:33:51.180322+0000","last_unstale":"2024-05-09T04:33:54.732648+0000","last_undegraded":"2024-05-09T04:33:54.732648+0000","last_fullsized":"2024-05-09T04:33:54.732648+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:26:18.474267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.878990+0000","last_change":"2024-05-09T04:33:52.204685+0000","last_active":"2024-05-09T04:33:54.878990+0000","last_peered":"2024-05-09T04:33:54.878990+0000","last_clean":"2024-05-09T04:33:54.878990+0000","last_became_active":"2024-05-09T04:33:52.204353+0000","last_became_peered":"2024-05-09T04:33:52.204353+0000","last_unstale":"2024-05-09T04:33:54.878990+0000","last_undegraded":"2024-05-09T04:33:54.878990+0000","last_fullsized":"2024-05-09T04:33:54.878990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:19:13.480834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":110,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.877786+0000","last_change":"2024-05-09T04:33:41.092063+0000","last_active":"2024-05-09T04:33:54.877786+0000","last_peered":"2024-05-09T04:33:54.877786+0000","last_clean":"2024-05-09T04:33:54.877786+0000","last_became_active":"2024-05-09T04:33:41.091194+0000","last_became_peered":"2024-05-09T04:33:41.091194+0000","last_unstale":"2024-05-09T04:33:54.877786+0000","last_undegraded":"2024-05-09T04:33:54.877786+0000","last_fullsized":"2024-05-09T04:33:54.877786+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:40.045339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:40.045339+0000","last_clean_scrub_stamp":"2024-05-09T04:33:40.045339+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:04.908456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.211276+0000","last_change":"2024-05-09T04:33:51.179637+0000","last_active":"2024-05-09T04:33:54.211276+0000","last_peered":"2024-05-09T04:33:54.211276+0000","last_clean":"2024-05-09T04:33:54.211276+0000","last_became_active":"2024-05-09T04:33:51.179311+0000","last_became_peered":"2024-05-09T04:33:51.179311+0000","last_unstale":"2024-05-09T04:33:54.211276+0000","last_undegraded":"2024-05-09T04:33:54.211276+0000","last_fullsized":"2024-05-09T04:33:54.211276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:08:08.955016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.196827+0000","last_change":"2024-05-09T04:33:52.203706+0000","last_active":"2024-05-09T04:33:53.196827+0000","last_peered":"2024-05-09T04:33:53.196827+0000","last_clean":"2024-05-09T04:33:53.196827+0000","last_became_active":"2024-05-09T04:33:52.200053+0000","last_became_peered":"2024-05-09T04:33:52.200053+0000","last_unstale":"2024-05-09T04:33:53.196827+0000","last_undegraded":"2024-05-09T04:33:53.196827+0000","last_fullsized":"2024-05-09T04:33:53.196827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:29:08.279802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.877506+0000","last_change":"2024-05-09T04:33:51.185731+0000","last_active":"2024-05-09T04:33:54.877506+0000","last_peered":"2024-05-09T04:33:54.877506+0000","last_clean":"2024-05-09T04:33:54.877506+0000","last_became_active":"2024-05-09T04:33:51.184982+0000","last_became_peered":"2024-05-09T04:33:51.184982+0000","last_unstale":"2024-05-09T04:33:54.877506+0000","last_undegraded":"2024-05-09T04:33:54.877506+0000","last_fullsized":"2024-05-09T04:33:54.877506+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:29:11.409173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.193579+0000","last_change":"2024-05-09T04:33:52.208347+0000","last_active":"2024-05-09T04:33:53.193579+0000","last_peered":"2024-05-09T04:33:53.193579+0000","last_clean":"2024-05-09T04:33:53.193579+0000","last_became_active":"2024-05-09T04:33:52.206631+0000","last_became_peered":"2024-05-09T04:33:52.206631+0000","last_unstale":"2024-05-09T04:33:53.193579+0000","last_undegraded":"2024-05-09T04:33:53.193579+0000","last_fullsized":"2024-05-09T04:33:53.193579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:00:54.458043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.732806+0000","last_change":"2024-05-09T04:33:51.167013+0000","last_active":"2024-05-09T04:33:54.732806+0000","last_peered":"2024-05-09T04:33:54.732806+0000","last_clean":"2024-05-09T04:33:54.732806+0000","last_became_active":"2024-05-09T04:33:51.165450+0000","last_became_peered":"2024-05-09T04:33:51.165450+0000","last_unstale":"2024-05-09T04:33:54.732806+0000","last_undegraded":"2024-05-09T04:33:54.732806+0000","last_fullsized":"2024-05-09T04:33:54.732806+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:08:08.955016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.154238+0000","last_change":"2024-05-09T04:33:52.190418+0000","last_active":"2024-05-09T04:33:54.154238+0000","last_peered":"2024-05-09T04:33:54.154238+0000","last_clean":"2024-05-09T04:33:54.154238+0000","last_became_active":"2024-05-09T04:33:52.190069+0000","last_became_peered":"2024-05-09T04:33:52.190069+0000","last_unstale":"2024-05-09T04:33:54.154238+0000","last_undegraded":"2024-05-09T04:33:54.154238+0000","last_fullsized":"2024-05-09T04:33:54.154238+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:29:08.279802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.212222+0000","last_change":"2024-05-09T04:33:51.181463+0000","last_active":"2024-05-09T04:33:54.212222+0000","last_peered":"2024-05-09T04:33:54.212222+0000","last_clean":"2024-05-09T04:33:54.212222+0000","last_became_active":"2024-05-09T04:33:51.181213+0000","last_became_peered":"2024-05-09T04:33:51.181213+0000","last_unstale":"2024-05-09T04:33:54.212222+0000","last_undegraded":"2024-05-09T04:33:54.212222+0000","last_fullsized":"2024-05-09T04:33:54.212222+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:50:51.015799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.195331+0000","last_change":"2024-05-09T04:33:52.188609+0000","last_active":"2024-05-09T04:33:53.195331+0000","last_peered":"2024-05-09T04:33:53.195331+0000","last_clean":"2024-05-09T04:33:53.195331+0000","last_became_active":"2024-05-09T04:33:52.188379+0000","last_became_peered":"2024-05-09T04:33:52.188379+0000","last_unstale":"2024-05-09T04:33:53.195331+0000","last_undegraded":"2024-05-09T04:33:53.195331+0000","last_fullsized":"2024-05-09T04:33:53.195331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:19:48.733290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.153924+0000","last_change":"2024-05-09T04:33:51.164123+0000","last_active":"2024-05-09T04:33:54.153924+0000","last_peered":"2024-05-09T04:33:54.153924+0000","last_clean":"2024-05-09T04:33:54.153924+0000","last_became_active":"2024-05-09T04:33:51.163686+0000","last_became_peered":"2024-05-09T04:33:51.163686+0000","last_unstale":"2024-05-09T04:33:54.153924+0000","last_undegraded":"2024-05-09T04:33:54.153924+0000","last_fullsized":"2024-05-09T04:33:54.153924+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:29:11.409173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.212248+0000","last_change":"2024-05-09T04:33:52.209697+0000","last_active":"2024-05-09T04:33:54.212248+0000","last_peered":"2024-05-09T04:33:54.212248+0000","last_clean":"2024-05-09T04:33:54.212248+0000","last_became_active":"2024-05-09T04:33:52.209347+0000","last_became_peered":"2024-05-09T04:33:52.209347+0000","last_unstale":"2024-05-09T04:33:54.212248+0000","last_undegraded":"2024-05-09T04:33:54.212248+0000","last_fullsized":"2024-05-09T04:33:54.212248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:19:08.318788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.879358+0000","last_change":"2024-05-09T04:33:51.184584+0000","last_active":"2024-05-09T04:33:54.879358+0000","last_peered":"2024-05-09T04:33:54.879358+0000","last_clean":"2024-05-09T04:33:54.879358+0000","last_became_active":"2024-05-09T04:33:51.184369+0000","last_became_peered":"2024-05-09T04:33:51.184369+0000","last_unstale":"2024-05-09T04:33:54.879358+0000","last_undegraded":"2024-05-09T04:33:54.879358+0000","last_fullsized":"2024-05-09T04:33:54.879358+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:22:40.919482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.154095+0000","last_change":"2024-05-09T04:33:52.189698+0000","last_active":"2024-05-09T04:33:54.154095+0000","last_peered":"2024-05-09T04:33:54.154095+0000","last_clean":"2024-05-09T04:33:54.154095+0000","last_became_active":"2024-05-09T04:33:52.189388+0000","last_became_peered":"2024-05-09T04:33:52.189388+0000","last_unstale":"2024-05-09T04:33:54.154095+0000","last_undegraded":"2024-05-09T04:33:54.154095+0000","last_fullsized":"2024-05-09T04:33:54.154095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:59:28.002585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.179780+0000","last_change":"2024-05-09T04:33:51.173756+0000","last_active":"2024-05-09T04:33:54.179780+0000","last_peered":"2024-05-09T04:33:54.179780+0000","last_clean":"2024-05-09T04:33:54.179780+0000","last_became_active":"2024-05-09T04:33:51.173416+0000","last_became_peered":"2024-05-09T04:33:51.173416+0000","last_unstale":"2024-05-09T04:33:54.179780+0000","last_undegraded":"2024-05-09T04:33:54.179780+0000","last_fullsized":"2024-05-09T04:33:54.179780+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:02:50.965016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.197927+0000","last_change":"2024-05-09T04:33:52.205567+0000","last_active":"2024-05-09T04:33:53.197927+0000","last_peered":"2024-05-09T04:33:53.197927+0000","last_clean":"2024-05-09T04:33:53.197927+0000","last_became_active":"2024-05-09T04:33:52.205252+0000","last_became_peered":"2024-05-09T04:33:52.205252+0000","last_unstale":"2024-05-09T04:33:53.197927+0000","last_undegraded":"2024-05-09T04:33:53.197927+0000","last_fullsized":"2024-05-09T04:33:53.197927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:13:19.513750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.686413+0000","last_change":"2024-05-09T04:33:51.181117+0000","last_active":"2024-05-09T04:33:53.686413+0000","last_peered":"2024-05-09T04:33:53.686413+0000","last_clean":"2024-05-09T04:33:53.686413+0000","last_became_active":"2024-05-09T04:33:51.179062+0000","last_became_peered":"2024-05-09T04:33:51.179062+0000","last_unstale":"2024-05-09T04:33:53.686413+0000","last_undegraded":"2024-05-09T04:33:53.686413+0000","last_fullsized":"2024-05-09T04:33:53.686413+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:19:47.722161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.195642+0000","last_change":"2024-05-09T04:33:52.181414+0000","last_active":"2024-05-09T04:33:53.195642+0000","last_peered":"2024-05-09T04:33:53.195642+0000","last_clean":"2024-05-09T04:33:53.195642+0000","last_became_active":"2024-05-09T04:33:52.180058+0000","last_became_peered":"2024-05-09T04:33:52.180058+0000","last_unstale":"2024-05-09T04:33:53.195642+0000","last_undegraded":"2024-05-09T04:33:53.195642+0000","last_fullsized":"2024-05-09T04:33:53.195642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:17.801441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.877505+0000","last_change":"2024-05-09T04:33:51.183265+0000","last_active":"2024-05-09T04:33:54.877505+0000","last_peered":"2024-05-09T04:33:54.877505+0000","last_clean":"2024-05-09T04:33:54.877505+0000","last_became_active":"2024-05-09T04:33:51.181781+0000","last_became_peered":"2024-05-09T04:33:51.181781+0000","last_unstale":"2024-05-09T04:33:54.877505+0000","last_undegraded":"2024-05-09T04:33:54.877505+0000","last_fullsized":"2024-05-09T04:33:54.877505+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:19:47.722161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.194563+0000","last_change":"2024-05-09T04:33:52.203247+0000","last_active":"2024-05-09T04:33:53.194563+0000","last_peered":"2024-05-09T04:33:53.194563+0000","last_clean":"2024-05-09T04:33:53.194563+0000","last_became_active":"2024-05-09T04:33:52.203006+0000","last_became_peered":"2024-05-09T04:33:52.203006+0000","last_unstale":"2024-05-09T04:33:53.194563+0000","last_undegraded":"2024-05-09T04:33:53.194563+0000","last_fullsized":"2024-05-09T04:33:53.194563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:02:51.976144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.154676+0000","last_change":"2024-05-09T04:33:51.172718+0000","last_active":"2024-05-09T04:33:54.154676+0000","last_peered":"2024-05-09T04:33:54.154676+0000","last_clean":"2024-05-09T04:33:54.154676+0000","last_became_active":"2024-05-09T04:33:51.172131+0000","last_became_peered":"2024-05-09T04:33:51.172131+0000","last_unstale":"2024-05-09T04:33:54.154676+0000","last_undegraded":"2024-05-09T04:33:54.154676+0000","last_fullsized":"2024-05-09T04:33:54.154676+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:08:08.955016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.196487+0000","last_change":"2024-05-09T04:33:52.201413+0000","last_active":"2024-05-09T04:33:53.196487+0000","last_peered":"2024-05-09T04:33:53.196487+0000","last_clean":"2024-05-09T04:33:53.196487+0000","last_became_active":"2024-05-09T04:33:52.199032+0000","last_became_peered":"2024-05-09T04:33:52.199032+0000","last_unstale":"2024-05-09T04:33:53.196487+0000","last_undegraded":"2024-05-09T04:33:53.196487+0000","last_fullsized":"2024-05-09T04:33:53.196487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:34:40.257159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.154633+0000","last_change":"2024-05-09T04:33:51.159715+0000","last_active":"2024-05-09T04:33:54.154633+0000","last_peered":"2024-05-09T04:33:54.154633+0000","last_clean":"2024-05-09T04:33:54.154633+0000","last_became_active":"2024-05-09T04:33:51.159395+0000","last_became_peered":"2024-05-09T04:33:51.159395+0000","last_unstale":"2024-05-09T04:33:54.154633+0000","last_undegraded":"2024-05-09T04:33:54.154633+0000","last_fullsized":"2024-05-09T04:33:54.154633+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:19:43.170318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.197360+0000","last_change":"2024-05-09T04:33:52.198684+0000","last_active":"2024-05-09T04:33:53.197360+0000","last_peered":"2024-05-09T04:33:53.197360+0000","last_clean":"2024-05-09T04:33:53.197360+0000","last_became_active":"2024-05-09T04:33:52.196451+0000","last_became_peered":"2024-05-09T04:33:52.196451+0000","last_unstale":"2024-05-09T04:33:53.197360+0000","last_undegraded":"2024-05-09T04:33:53.197360+0000","last_fullsized":"2024-05-09T04:33:53.197360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:17.801441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.878591+0000","last_change":"2024-05-09T04:33:51.186050+0000","last_active":"2024-05-09T04:33:54.878591+0000","last_peered":"2024-05-09T04:33:54.878591+0000","last_clean":"2024-05-09T04:33:54.878591+0000","last_became_active":"2024-05-09T04:33:51.185720+0000","last_became_peered":"2024-05-09T04:33:51.185720+0000","last_unstale":"2024-05-09T04:33:54.878591+0000","last_undegraded":"2024-05-09T04:33:54.878591+0000","last_fullsized":"2024-05-09T04:33:54.878591+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:11:05.364073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.685574+0000","last_change":"2024-05-09T04:33:52.191123+0000","last_active":"2024-05-09T04:33:53.685574+0000","last_peered":"2024-05-09T04:33:53.685574+0000","last_clean":"2024-05-09T04:33:53.685574+0000","last_became_active":"2024-05-09T04:33:52.189005+0000","last_became_peered":"2024-05-09T04:33:52.189005+0000","last_unstale":"2024-05-09T04:33:53.685574+0000","last_undegraded":"2024-05-09T04:33:53.685574+0000","last_fullsized":"2024-05-09T04:33:53.685574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:34:29.852986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.212085+0000","last_change":"2024-05-09T04:33:52.208515+0000","last_active":"2024-05-09T04:33:54.212085+0000","last_peered":"2024-05-09T04:33:54.212085+0000","last_clean":"2024-05-09T04:33:54.212085+0000","last_became_active":"2024-05-09T04:33:52.208236+0000","last_became_peered":"2024-05-09T04:33:52.208236+0000","last_unstale":"2024-05-09T04:33:54.212085+0000","last_undegraded":"2024-05-09T04:33:54.212085+0000","last_fullsized":"2024-05-09T04:33:54.212085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:19:13.480834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.878387+0000","last_change":"2024-05-09T04:33:51.187919+0000","last_active":"2024-05-09T04:33:54.878387+0000","last_peered":"2024-05-09T04:33:54.878387+0000","last_clean":"2024-05-09T04:33:54.878387+0000","last_became_active":"2024-05-09T04:33:51.187590+0000","last_became_peered":"2024-05-09T04:33:51.187590+0000","last_unstale":"2024-05-09T04:33:54.878387+0000","last_undegraded":"2024-05-09T04:33:54.878387+0000","last_fullsized":"2024-05-09T04:33:54.878387+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:50:51.015799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.211737+0000","last_change":"2024-05-09T04:33:52.209870+0000","last_active":"2024-05-09T04:33:54.211737+0000","last_peered":"2024-05-09T04:33:54.211737+0000","last_clean":"2024-05-09T04:33:54.211737+0000","last_became_active":"2024-05-09T04:33:52.209199+0000","last_became_peered":"2024-05-09T04:33:52.209199+0000","last_unstale":"2024-05-09T04:33:54.211737+0000","last_undegraded":"2024-05-09T04:33:54.211737+0000","last_fullsized":"2024-05-09T04:33:54.211737+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:10:42.676456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:54.879005+0000","last_change":"2024-05-09T04:33:51.187870+0000","last_active":"2024-05-09T04:33:54.879005+0000","last_peered":"2024-05-09T04:33:54.879005+0000","last_clean":"2024-05-09T04:33:54.879005+0000","last_became_active":"2024-05-09T04:33:51.187575+0000","last_became_peered":"2024-05-09T04:33:51.187575+0000","last_unstale":"2024-05-09T04:33:54.879005+0000","last_undegraded":"2024-05-09T04:33:54.879005+0000","last_fullsized":"2024-05-09T04:33:54.879005+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:19:43.170318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.195161+0000","last_change":"2024-05-09T04:33:52.201406+0000","last_active":"2024-05-09T04:33:53.195161+0000","last_peered":"2024-05-09T04:33:53.195161+0000","last_clean":"2024-05-09T04:33:53.195161+0000","last_became_active":"2024-05-09T04:33:52.200809+0000","last_became_peered":"2024-05-09T04:33:52.200809+0000","last_unstale":"2024-05-09T04:33:53.195161+0000","last_undegraded":"2024-05-09T04:33:53.195161+0000","last_fullsized":"2024-05-09T04:33:53.195161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:59:28.002585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.197985+0000","last_change":"2024-05-09T04:33:51.168105+0000","last_active":"2024-05-09T04:33:53.197985+0000","last_peered":"2024-05-09T04:33:53.197985+0000","last_clean":"2024-05-09T04:33:53.197985+0000","last_became_active":"2024-05-09T04:33:51.167790+0000","last_became_peered":"2024-05-09T04:33:51.167790+0000","last_unstale":"2024-05-09T04:33:53.197985+0000","last_undegraded":"2024-05-09T04:33:53.197985+0000","last_fullsized":"2024-05-09T04:33:53.197985+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:08:08.955016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.197883+0000","last_change":"2024-05-09T04:33:52.205526+0000","last_active":"2024-05-09T04:33:53.197883+0000","last_peered":"2024-05-09T04:33:53.197883+0000","last_clean":"2024-05-09T04:33:53.197883+0000","last_became_active":"2024-05-09T04:33:52.205231+0000","last_became_peered":"2024-05-09T04:33:52.205231+0000","last_unstale":"2024-05-09T04:33:53.197883+0000","last_undegraded":"2024-05-09T04:33:53.197883+0000","last_fullsized":"2024-05-09T04:33:53.197883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:51.129368+0000","last_clean_scrub_stamp":"2024-05-09T04:33:51.129368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:02:17.813339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-05-09T04:33:53.195818+0000","last_change":"2024-05-09T04:33:51.176461+0000","last_active":"2024-05-09T04:33:53.195818+0000","last_peered":"2024-05-09T04:33:53.195818+0000","last_clean":"2024-05-09T04:33:53.195818+0000","last_became_active":"2024-05-09T04:33:51.176175+0000","last_became_peered":"2024-05-09T04:33:51.176175+0000","last_unstale":"2024-05-09T04:33:53.195818+0000","last_undegraded":"2024-05-09T04:33:53.195818+0000","last_fullsized":"2024-05-09T04:33:53.195818+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:50.118240+0000","last_clean_scrub_stamp":"2024-05-09T04:33:50.118240+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:01:20.702014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T04:34:02.593 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-09T04:34:02.593 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-09T04:34:02.593 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-05-09T04:34:02.593 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-09T04:34:02.830 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.830+0000 7f119ae00640 1 Processor -- start 2024-05-09T04:34:02.830 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.830+0000 7f119ae00640 1 -- start start 2024-05-09T04:34:02.831 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.831+0000 7f119ae00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f1194162a20 0x7f11940d1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:02.831 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.831+0000 7f119ae00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f11940d0d20 0x7f11940d1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:02.831 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.831+0000 7f119ae00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11940d3080 con 0x7f1194162e00 2024-05-09T04:34:02.831 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.831+0000 7f119ae00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f11940d3cd0 con 0x7f11940d0d20 2024-05-09T04:34:02.831 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.831+0000 7f119ae00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f11940d3360 con 0x7f1194162a20 2024-05-09T04:34:02.831 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.832+0000 7f1193400640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f1194162e00 0x7f11941632f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:37070/0 (socket says 172.21.15.145:37070) 2024-05-09T04:34:02.832 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.832+0000 7f1193400640 1 -- 172.21.15.145:0/3086970297 learned_addr learned my addr 172.21.15.145:0/3086970297 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:34:02.834 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.832+0000 7f1198e00640 1 --2- 172.21.15.145:0/3086970297 >> v2:172.21.15.145:3300/0 conn(0x7f11940d0d20 0x7f11940d1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:02.834 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.832+0000 7f1193e00640 1 --2- 172.21.15.145:0/3086970297 >> v2:172.21.15.192:3300/0 conn(0x7f1194162a20 0x7f11940d1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:02.834 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.832+0000 7f1192a00640 1 -- 172.21.15.145:0/3086970297 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 201574752 0 0) 0x7f11940d3080 con 0x7f1194162e00 2024-05-09T04:34:02.834 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.832+0000 7f1192a00640 1 -- 172.21.15.145:0/3086970297 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1178003520 con 0x7f1194162e00 2024-05-09T04:34:02.834 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.832+0000 7f1192a00640 1 -- 172.21.15.145:0/3086970297 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1824465210 0 0) 0x7f1178003520 con 0x7f1194162e00 2024-05-09T04:34:02.834 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.832+0000 7f1192a00640 1 -- 172.21.15.145:0/3086970297 >> v2:172.21.15.192:3300/0 conn(0x7f1194162a20 msgr2=0x7f11940d1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:02.834 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.832+0000 7f1192a00640 1 --2- 172.21.15.145:0/3086970297 >> v2:172.21.15.192:3300/0 conn(0x7f1194162a20 0x7f11940d1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:02.834 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.832+0000 7f1192a00640 1 -- 172.21.15.145:0/3086970297 >> v2:172.21.15.145:3300/0 conn(0x7f11940d0d20 msgr2=0x7f11940d1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:02.834 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.832+0000 7f1192a00640 1 --2- 172.21.15.145:0/3086970297 >> v2:172.21.15.145:3300/0 conn(0x7f11940d0d20 0x7f11940d1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:02.834 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.832+0000 7f1192a00640 1 -- 172.21.15.145:0/3086970297 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11940d3960 con 0x7f1194162e00 2024-05-09T04:34:02.834 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.832+0000 7f1192a00640 1 -- 172.21.15.145:0/3086970297 <== mon.0 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3474413261 0 0) 0x7f11800016f0 con 0x7f1194162e00 2024-05-09T04:34:02.834 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.832+0000 7f1192a00640 1 -- 172.21.15.145:0/3086970297 >> v1:172.21.15.145:6789/0 conn(0x7f1194162e00 legacy=0x7f11941632f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:02.835 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.832+0000 7f1192a00640 1 --2- 172.21.15.145:0/3086970297 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1178003aa0 0x7f1178003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:02.835 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.833+0000 7f1192a00640 1 --2- 172.21.15.145:0/3086970297 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1178004800 0x7f1178004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:02.835 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.833+0000 7f1192a00640 1 --2- 172.21.15.145:0/3086970297 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f11780051c0 0x7f117800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:02.835 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.833+0000 7f1192a00640 1 -- 172.21.15.145:0/3086970297 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f11940d3cd0 con 0x7f11780051c0 2024-05-09T04:34:02.835 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.833+0000 7f1192a00640 1 -- 172.21.15.145:0/3086970297 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f11940d3360 con 0x7f1178003aa0 2024-05-09T04:34:02.835 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.833+0000 7f1192a00640 1 -- 172.21.15.145:0/3086970297 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f117800c040 con 0x7f1178004800 2024-05-09T04:34:02.835 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.833+0000 7f1198e00640 1 --2- 172.21.15.145:0/3086970297 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f11780051c0 0x7f117800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:02.835 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.833+0000 7f1193400640 1 --2- 172.21.15.145:0/3086970297 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1178004800 0x7f1178004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:02.835 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.833+0000 7f1198e00640 1 -- 172.21.15.145:0/3086970297 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1178004800 msgr2=0x7f1178004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:02.835 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.833+0000 7f1198e00640 1 --2- 172.21.15.145:0/3086970297 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1178004800 0x7f1178004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:02.835 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.833+0000 7f1198e00640 1 -- 172.21.15.145:0/3086970297 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1178003aa0 msgr2=0x7f1178003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:02.835 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.833+0000 7f1198e00640 1 --2- 172.21.15.145:0/3086970297 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1178003aa0 0x7f1178003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:02.835 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.833+0000 7f1198e00640 1 -- 172.21.15.145:0/3086970297 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1178003520 con 0x7f11780051c0 2024-05-09T04:34:02.835 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.834+0000 7f1198e00640 1 -- 172.21.15.145:0/3086970297 reap_dead start 2024-05-09T04:34:02.835 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.834+0000 7f1198e00640 1 --2- 172.21.15.145:0/3086970297 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f11780051c0 0x7f117800b9c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f118c009fc0 tx=0x7f1178005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0331e30773e08a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:02.837 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.834+0000 7f1192a00640 1 -- 172.21.15.145:0/3086970297 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118c035020 con 0x7f11780051c0 2024-05-09T04:34:02.837 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.834+0000 7f1192a00640 1 -- 172.21.15.145:0/3086970297 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f118c003bd0 con 0x7f11780051c0 2024-05-09T04:34:02.837 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.834+0000 7f1192a00640 1 -- 172.21.15.145:0/3086970297 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118c009c10 con 0x7f11780051c0 2024-05-09T04:34:02.837 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.835+0000 7f119ae00640 1 -- 172.21.15.145:0/3086970297 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f11780051c0 msgr2=0x7f117800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:02.837 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.835+0000 7f119ae00640 1 --2- 172.21.15.145:0/3086970297 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f11780051c0 0x7f117800b9c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f118c009fc0 tx=0x7f1178005830 comp rx=0 tx=0).stop 2024-05-09T04:34:02.837 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.835+0000 7f119ae00640 1 -- 172.21.15.145:0/3086970297 shutdown_connections 2024-05-09T04:34:02.837 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.835+0000 7f119ae00640 1 --2- 172.21.15.145:0/3086970297 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f11780051c0 0x7f117800b9c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:02.837 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.835+0000 7f119ae00640 1 -- 172.21.15.145:0/3086970297 >> 172.21.15.145:0/3086970297 conn(0x7f119409e540 msgr2=0x7f11940d5200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:02.837 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.835+0000 7f119ae00640 1 -- 172.21.15.145:0/3086970297 shutdown_connections 2024-05-09T04:34:02.837 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.835+0000 7f119ae00640 1 -- 172.21.15.145:0/3086970297 wait complete. 2024-05-09T04:34:02.838 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.837+0000 7f119ae00640 1 Processor -- start 2024-05-09T04:34:02.838 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.837+0000 7f119ae00640 1 -- start start 2024-05-09T04:34:02.840 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.838+0000 7f119ae00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11780051c0 0x7f119415bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:02.840 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.838+0000 7f119ae00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f11940cd890 0x7f11940cc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:02.840 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.838+0000 7f119ae00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f11940ccc60 0x7f11940cd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:02.840 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.838+0000 7f119ae00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f11940b5e80 con 0x7f11940cd890 2024-05-09T04:34:02.840 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.838+0000 7f119ae00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f11940b5d00 con 0x7f11780051c0 2024-05-09T04:34:02.840 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.838+0000 7f119ae00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f11940b6000 con 0x7f11940ccc60 2024-05-09T04:34:02.840 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.838+0000 7f1193400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f11940cd890 0x7f11940cc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:02.840 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.838+0000 7f1193400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f11940cd890 0x7f11940cc2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:37838/0 (socket says 172.21.15.145:37838) 2024-05-09T04:34:02.840 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.838+0000 7f1193400640 1 -- 172.21.15.145:0/628585440 learned_addr learned my addr 172.21.15.145:0/628585440 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:02.840 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.838+0000 7f1198e00640 1 --2- 172.21.15.145:0/628585440 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f11940ccc60 0x7f11940cd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:02.840 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.838+0000 7f1193400640 1 -- 172.21.15.145:0/628585440 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f11940ccc60 msgr2=0x7f11940cd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:02.840 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.838+0000 7f1193400640 1 --2- 172.21.15.145:0/628585440 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f11940ccc60 0x7f11940cd0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:02.840 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.838+0000 7f1193400640 1 -- 172.21.15.145:0/628585440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11780051c0 msgr2=0x7f119415bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:02.840 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.838+0000 7f1193400640 1 --2- 172.21.15.145:0/628585440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11780051c0 0x7f119415bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:02.840 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.838+0000 7f1193400640 1 -- 172.21.15.145:0/628585440 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11940cc810 con 0x7f11940cd890 2024-05-09T04:34:02.841 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.839+0000 7f1193400640 1 --2- 172.21.15.145:0/628585440 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f11940cd890 0x7f11940cc2d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f118000d7a0 tx=0x7f118000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:02.841 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.839+0000 7f118be00640 1 -- 172.21.15.145:0/628585440 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11800074e0 con 0x7f11940cd890 2024-05-09T04:34:02.841 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.839+0000 7f118be00640 1 -- 172.21.15.145:0/628585440 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f118000de50 con 0x7f11940cd890 2024-05-09T04:34:02.841 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.839+0000 7f118be00640 1 -- 172.21.15.145:0/628585440 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1180004a00 con 0x7f11940cd890 2024-05-09T04:34:02.841 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.839+0000 7f119ae00640 1 -- 172.21.15.145:0/628585440 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11940ce8f0 con 0x7f11940cd890 2024-05-09T04:34:02.841 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.839+0000 7f119ae00640 1 -- 172.21.15.145:0/628585440 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11940ceeb0 con 0x7f11940cd890 2024-05-09T04:34:02.841 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.840+0000 7f119ae00640 1 -- 172.21.15.145:0/628585440 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f119406a310 con 0x7f11940cd890 2024-05-09T04:34:02.841 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.841+0000 7f118be00640 1 -- 172.21.15.145:0/628585440 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f118001b020 con 0x7f11940cd890 2024-05-09T04:34:02.842 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.842+0000 7f118be00640 1 --2- 172.21.15.145:0/628585440 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f11740762d0 0x7f1174078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:02.842 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.842+0000 7f118be00640 1 -- 172.21.15.145:0/628585440 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1180097460 con 0x7f11940cd890 2024-05-09T04:34:02.842 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.842+0000 7f1193e00640 1 --2- 172.21.15.145:0/628585440 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f11740762d0 0x7f1174078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:02.842 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.843+0000 7f1193e00640 1 --2- 172.21.15.145:0/628585440 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f11740762d0 0x7f1174078790 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1184006a90 tx=0x7f1184006a20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:02.846 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:02.847+0000 7f118be00640 1 -- 172.21.15.145:0/628585440 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f1180060db0 con 0x7f11940cd890 2024-05-09T04:34:03.076 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.076+0000 7f119ae00640 1 -- 172.21.15.145:0/628585440 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f11940cca40 con 0x7f11940cd890 2024-05-09T04:34:03.077 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.077+0000 7f118be00640 1 -- 172.21.15.145:0/628585440 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f11800606b0 con 0x7f11940cd890 2024-05-09T04:34:03.077 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:34:03.077 INFO:teuthology.orchestra.run.smithi145.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-09T04:34:03.080 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.079+0000 7f119ae00640 1 -- 172.21.15.145:0/628585440 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f11740762d0 msgr2=0x7f1174078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:03.080 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.079+0000 7f119ae00640 1 --2- 172.21.15.145:0/628585440 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f11740762d0 0x7f1174078790 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1184006a90 tx=0x7f1184006a20 comp rx=0 tx=0).stop 2024-05-09T04:34:03.080 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.079+0000 7f119ae00640 1 -- 172.21.15.145:0/628585440 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f11940cd890 msgr2=0x7f11940cc2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:03.080 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.079+0000 7f119ae00640 1 --2- 172.21.15.145:0/628585440 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f11940cd890 0x7f11940cc2d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f118000d7a0 tx=0x7f118000dc60 comp rx=0 tx=0).stop 2024-05-09T04:34:03.080 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.080+0000 7f119ae00640 1 -- 172.21.15.145:0/628585440 shutdown_connections 2024-05-09T04:34:03.080 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.080+0000 7f119ae00640 1 --2- 172.21.15.145:0/628585440 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f11740762d0 0x7f1174078790 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:03.080 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.080+0000 7f119ae00640 1 --2- 172.21.15.145:0/628585440 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f11940ccc60 0x7f11940cd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:03.080 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.080+0000 7f119ae00640 1 --2- 172.21.15.145:0/628585440 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f11940cd890 0x7f11940cc2d0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:03.080 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.080+0000 7f119ae00640 1 --2- 172.21.15.145:0/628585440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11780051c0 0x7f119415bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:03.081 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.080+0000 7f119ae00640 1 -- 172.21.15.145:0/628585440 >> 172.21.15.145:0/628585440 conn(0x7f119409e540 msgr2=0x7f119406eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:03.081 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.080+0000 7f119ae00640 1 -- 172.21.15.145:0/628585440 shutdown_connections 2024-05-09T04:34:03.081 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.080+0000 7f119ae00640 1 -- 172.21.15.145:0/628585440 wait complete. 2024-05-09T04:34:03.096 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-05-09T04:34:03.096 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T04:34:03.298 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.297+0000 7f18c3400640 1 Processor -- start 2024-05-09T04:34:03.298 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.298+0000 7f18c3400640 1 -- start start 2024-05-09T04:34:03.299 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.299+0000 7f18c3400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18bc162a20 0x7f18bc0d1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:03.299 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.299+0000 7f18c3400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f18bc0d0d20 0x7f18bc0d1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:03.299 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.299+0000 7f18c3400640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18bc0d3080 con 0x7f18bc162e00 2024-05-09T04:34:03.299 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.299+0000 7f18c3400640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f18bc0d3cd0 con 0x7f18bc0d0d20 2024-05-09T04:34:03.299 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.299+0000 7f18c3400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f18bc0d3360 con 0x7f18bc162a20 2024-05-09T04:34:03.300 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.300+0000 7f18c0a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18bc162a20 0x7f18bc0d1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:03.300 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.300+0000 7f18c0a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18bc162a20 0x7f18bc0d1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.145:42728/0 (socket says 172.21.15.145:42728) 2024-05-09T04:34:03.300 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.300+0000 7f18c0a00640 1 -- 172.21.15.145:0/4010027559 learned_addr learned my addr 172.21.15.145:0/4010027559 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:03.300 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.300+0000 7f18c1400640 1 --2- 172.21.15.145:0/4010027559 >> v2:172.21.15.192:3300/0 conn(0x7f18bc0d0d20 0x7f18bc0d1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:03.301 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.300+0000 7f18c0a00640 1 -- 172.21.15.145:0/4010027559 >> v2:172.21.15.192:3300/0 conn(0x7f18bc0d0d20 msgr2=0x7f18bc0d1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:03.301 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.300+0000 7f18c0a00640 1 --2- 172.21.15.145:0/4010027559 >> v2:172.21.15.192:3300/0 conn(0x7f18bc0d0d20 0x7f18bc0d1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:03.301 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.300+0000 7f18c0a00640 1 -- 172.21.15.145:0/4010027559 >> v1:172.21.15.192:6789/0 conn(0x7f18bc162e00 legacy=0x7f18bc1632f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:03.301 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.301+0000 7f18c0a00640 1 -- 172.21.15.145:0/4010027559 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18bc0d3960 con 0x7f18bc162a20 2024-05-09T04:34:03.301 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.301+0000 7f18c1400640 1 --2- 172.21.15.145:0/4010027559 >> v2:172.21.15.192:3300/0 conn(0x7f18bc0d0d20 0x7f18bc0d1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:34:03.301 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.302+0000 7f18c0a00640 1 --2- 172.21.15.145:0/4010027559 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18bc162a20 0x7f18bc0d1830 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f18ac009870 tx=0x7f18ac02a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2334366d64f229ea server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:03.302 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.302+0000 7f18bb400640 1 -- 172.21.15.145:0/4010027559 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ac037070 con 0x7f18bc162a20 2024-05-09T04:34:03.302 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.302+0000 7f18bb400640 1 -- 172.21.15.145:0/4010027559 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18ac02ae50 con 0x7f18bc162a20 2024-05-09T04:34:03.302 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.302+0000 7f18bb400640 1 -- 172.21.15.145:0/4010027559 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ac032860 con 0x7f18bc162a20 2024-05-09T04:34:03.303 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.303+0000 7f18c3400640 1 -- 172.21.15.145:0/4010027559 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18bc162a20 msgr2=0x7f18bc0d1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:03.303 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.303+0000 7f18c3400640 1 --2- 172.21.15.145:0/4010027559 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18bc162a20 0x7f18bc0d1830 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f18ac009870 tx=0x7f18ac02a300 comp rx=0 tx=0).stop 2024-05-09T04:34:03.303 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.303+0000 7f18c3400640 1 -- 172.21.15.145:0/4010027559 shutdown_connections 2024-05-09T04:34:03.303 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.303+0000 7f18c3400640 1 --2- 172.21.15.145:0/4010027559 >> v2:172.21.15.192:3300/0 conn(0x7f18bc0d0d20 0x7f18bc0d1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:03.303 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.303+0000 7f18c3400640 1 --2- 172.21.15.145:0/4010027559 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18bc162a20 0x7f18bc0d1830 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:03.303 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.303+0000 7f18c3400640 1 -- 172.21.15.145:0/4010027559 >> 172.21.15.145:0/4010027559 conn(0x7f18bc09e540 msgr2=0x7f18bc0d5200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:03.303 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.303+0000 7f18c3400640 1 -- 172.21.15.145:0/4010027559 shutdown_connections 2024-05-09T04:34:03.304 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.303+0000 7f18c3400640 1 -- 172.21.15.145:0/4010027559 wait complete. 2024-05-09T04:34:03.305 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.305+0000 7f18c3400640 1 Processor -- start 2024-05-09T04:34:03.305 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.305+0000 7f18c3400640 1 -- start start 2024-05-09T04:34:03.308 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.306+0000 7f18c3400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18bc0d0d20 0x7f18bc0b7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:03.308 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.306+0000 7f18c3400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f18bc162a20 0x7f18bc0b94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:03.308 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.306+0000 7f18c3400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18bc162e00 0x7f18bc15d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:03.308 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.307+0000 7f18c3400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f18bc0b5e80 con 0x7f18bc162a20 2024-05-09T04:34:03.308 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.307+0000 7f18c3400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f18bc0b5d00 con 0x7f18bc162e00 2024-05-09T04:34:03.308 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.307+0000 7f18c3400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f18bc0b6000 con 0x7f18bc0d0d20 2024-05-09T04:34:03.308 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.307+0000 7f18bbe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f18bc162a20 0x7f18bc0b94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:03.308 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.307+0000 7f18bbe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f18bc162a20 0x7f18bc0b94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:37842/0 (socket says 172.21.15.145:37842) 2024-05-09T04:34:03.308 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.307+0000 7f18bbe00640 1 -- 172.21.15.145:0/3140354781 learned_addr learned my addr 172.21.15.145:0/3140354781 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:03.308 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.307+0000 7f18c0a00640 1 --2- 172.21.15.145:0/3140354781 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18bc0d0d20 0x7f18bc0b7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:03.308 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.307+0000 7f18c1400640 1 --2- 172.21.15.145:0/3140354781 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18bc162e00 0x7f18bc15d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:03.308 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.307+0000 7f18bbe00640 1 -- 172.21.15.145:0/3140354781 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18bc0d0d20 msgr2=0x7f18bc0b7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:03.309 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.307+0000 7f18bbe00640 1 --2- 172.21.15.145:0/3140354781 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18bc0d0d20 0x7f18bc0b7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:03.309 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.307+0000 7f18bbe00640 1 -- 172.21.15.145:0/3140354781 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18bc162e00 msgr2=0x7f18bc15d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:03.309 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.307+0000 7f18bbe00640 1 --2- 172.21.15.145:0/3140354781 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18bc162e00 0x7f18bc15d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:03.309 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.307+0000 7f18bbe00640 1 -- 172.21.15.145:0/3140354781 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18bc0b8400 con 0x7f18bc162a20 2024-05-09T04:34:03.309 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.307+0000 7f18c0a00640 1 --2- 172.21.15.145:0/3140354781 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18bc0d0d20 0x7f18bc0b7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:34:03.309 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.308+0000 7f18bbe00640 1 --2- 172.21.15.145:0/3140354781 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f18bc162a20 0x7f18bc0b94a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f18a400c3a0 tx=0x7f18a400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:03.309 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.308+0000 7f18b8c00640 1 -- 172.21.15.145:0/3140354781 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a4017070 con 0x7f18bc162a20 2024-05-09T04:34:03.309 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.308+0000 7f18b8c00640 1 -- 172.21.15.145:0/3140354781 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18a40052d0 con 0x7f18bc162a20 2024-05-09T04:34:03.309 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.308+0000 7f18b8c00640 1 -- 172.21.15.145:0/3140354781 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a4003830 con 0x7f18bc162a20 2024-05-09T04:34:03.309 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.308+0000 7f18c3400640 1 -- 172.21.15.145:0/3140354781 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18bc0b86e0 con 0x7f18bc162a20 2024-05-09T04:34:03.309 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.308+0000 7f18c3400640 1 -- 172.21.15.145:0/3140354781 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18bc0b8ca0 con 0x7f18bc162a20 2024-05-09T04:34:03.310 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.309+0000 7f18c3400640 1 -- 172.21.15.145:0/3140354781 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18bc06a310 con 0x7f18bc162a20 2024-05-09T04:34:03.310 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.310+0000 7f18b8c00640 1 -- 172.21.15.145:0/3140354781 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f18a401a070 con 0x7f18bc162a20 2024-05-09T04:34:03.311 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.311+0000 7f18b8c00640 1 --2- 172.21.15.145:0/3140354781 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f18980762d0 0x7f1898078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:03.311 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.311+0000 7f18b8c00640 1 -- 172.21.15.145:0/3140354781 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f18a4096a00 con 0x7f18bc162a20 2024-05-09T04:34:03.311 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.312+0000 7f18c0a00640 1 --2- 172.21.15.145:0/3140354781 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f18980762d0 0x7f1898078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:03.312 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.312+0000 7f18c0a00640 1 --2- 172.21.15.145:0/3140354781 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f18980762d0 0x7f1898078790 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f18ac034000 tx=0x7f18ac02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:03.315 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.316+0000 7f18b8c00640 1 -- 172.21.15.145:0/3140354781 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f18a40602c0 con 0x7f18bc162a20 2024-05-09T04:34:03.559 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.560+0000 7f18c3400640 1 -- 172.21.15.145:0/3140354781 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f18bc15f290 con 0x7f18bc162a20 2024-05-09T04:34:03.560 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.561+0000 7f18b8c00640 1 -- 172.21.15.145:0/3140354781 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5187 (secure 0 0 0) 0x7f18a40682b0 con 0x7f18bc162a20 2024-05-09T04:34:03.561 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:34:03.561 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/1284540932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":1284540932},{"type":"v1","addr":"172.21.15.145:6835","nonce":1284540932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/1308656248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1308656248},{"type":"v1","addr":"172.21.15.145:6837","nonce":1308656248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3323136057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3323136057},{"type":"v1","addr":"172.21.15.192:6833","nonce":3323136057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T04:33:51.628921+0000","modified":"2024-05-09T04:33:54.699907+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.192:6835/1113351558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1113351558},{"type":"v1","addr":"172.21.15.192:6835","nonce":1113351558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4270,"qdb_cluster":[4270]},"id":1}]} 2024-05-09T04:34:03.561 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 8 2024-05-09T04:34:03.563 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.563+0000 7f18c3400640 1 -- 172.21.15.145:0/3140354781 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f18980762d0 msgr2=0x7f1898078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:03.563 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.563+0000 7f18c3400640 1 --2- 172.21.15.145:0/3140354781 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f18980762d0 0x7f1898078790 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f18ac034000 tx=0x7f18ac02e000 comp rx=0 tx=0).stop 2024-05-09T04:34:03.563 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.563+0000 7f18c3400640 1 -- 172.21.15.145:0/3140354781 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f18bc162a20 msgr2=0x7f18bc0b94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:03.563 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.563+0000 7f18c3400640 1 --2- 172.21.15.145:0/3140354781 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f18bc162a20 0x7f18bc0b94a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f18a400c3a0 tx=0x7f18a400c860 comp rx=0 tx=0).stop 2024-05-09T04:34:03.564 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.564+0000 7f18c3400640 1 -- 172.21.15.145:0/3140354781 shutdown_connections 2024-05-09T04:34:03.564 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.564+0000 7f18c3400640 1 --2- 172.21.15.145:0/3140354781 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f18980762d0 0x7f1898078790 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:03.564 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.564+0000 7f18c3400640 1 --2- 172.21.15.145:0/3140354781 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18bc162e00 0x7f18bc15d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:03.564 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.564+0000 7f18c3400640 1 --2- 172.21.15.145:0/3140354781 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f18bc162a20 0x7f18bc0b94a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:03.564 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.564+0000 7f18c3400640 1 --2- 172.21.15.145:0/3140354781 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18bc0d0d20 0x7f18bc0b7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:03.564 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.564+0000 7f18c3400640 1 -- 172.21.15.145:0/3140354781 >> 172.21.15.145:0/3140354781 conn(0x7f18bc09e540 msgr2=0x7f18bc15e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:03.564 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.564+0000 7f18c3400640 1 -- 172.21.15.145:0/3140354781 shutdown_connections 2024-05-09T04:34:03.564 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.564+0000 7f18c3400640 1 -- 172.21.15.145:0/3140354781 wait complete. 2024-05-09T04:34:03.581 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T04:34:03.610 DEBUG:teuthology.orchestra.run.smithi145:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T04:34:03.635 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T04:34:03.778 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.778+0000 7f9907e00640 1 Processor -- start 2024-05-09T04:34:03.778 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.778+0000 7f9907e00640 1 -- start start 2024-05-09T04:34:03.779 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.779+0000 7f9907e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f9900163b10 0x7f990006eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:03.779 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.779+0000 7f9907e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9900163610 0x7f990006d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:03.779 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.779+0000 7f9907e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f990006ddc0 con 0x7f990015f340 2024-05-09T04:34:03.779 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.779+0000 7f9907e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f99000b4f50 con 0x7f9900163b10 2024-05-09T04:34:03.779 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.779+0000 7f9907e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f99000b5230 con 0x7f9900163610 2024-05-09T04:34:03.780 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.780+0000 7f9905e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9900163610 0x7f990006d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:03.781 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.780+0000 7f9905400640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f990015f340 0x7f9900162f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.145:49668/0 (socket says 172.21.15.145:49668) 2024-05-09T04:34:03.781 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.780+0000 7f9905400640 1 -- 172.21.15.145:0/1972287997 learned_addr learned my addr 172.21.15.145:0/1972287997 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:34:03.781 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.780+0000 7f9904a00640 1 --2- 172.21.15.145:0/1972287997 >> v2:172.21.15.192:3300/0 conn(0x7f9900163b10 0x7f990006eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:03.781 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.781+0000 7f98ffe00640 1 -- 172.21.15.145:0/1972287997 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 825147141 0 0) 0x7f990006ddc0 con 0x7f990015f340 2024-05-09T04:34:03.781 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.781+0000 7f98ffe00640 1 -- 172.21.15.145:0/1972287997 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f98e8003520 con 0x7f990015f340 2024-05-09T04:34:03.781 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.781+0000 7f9905e00640 1 -- 172.21.15.145:0/1972287997 >> v2:172.21.15.192:3300/0 conn(0x7f9900163b10 msgr2=0x7f990006eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:03.781 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.781+0000 7f9905e00640 1 --2- 172.21.15.145:0/1972287997 >> v2:172.21.15.192:3300/0 conn(0x7f9900163b10 0x7f990006eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:03.781 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.781+0000 7f9905e00640 1 -- 172.21.15.145:0/1972287997 >> v1:172.21.15.192:6789/0 conn(0x7f990015f340 legacy=0x7f9900162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:03.781 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.781+0000 7f9905e00640 1 -- 172.21.15.145:0/1972287997 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99000b5510 con 0x7f9900163610 2024-05-09T04:34:03.782 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.782+0000 7f9905e00640 1 --2- 172.21.15.145:0/1972287997 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9900163610 0x7f990006d490 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f98f8009d90 tx=0x7f98f802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d15ba9966c4a703e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:03.782 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.782+0000 7f98ffe00640 1 -- 172.21.15.145:0/1972287997 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f8007740 con 0x7f9900163610 2024-05-09T04:34:03.782 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.782+0000 7f98ffe00640 1 -- 172.21.15.145:0/1972287997 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98f80078e0 con 0x7f9900163610 2024-05-09T04:34:03.782 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.782+0000 7f98ffe00640 1 -- 172.21.15.145:0/1972287997 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f8007c90 con 0x7f9900163610 2024-05-09T04:34:03.783 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.783+0000 7f9907e00640 1 -- 172.21.15.145:0/1972287997 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9900163610 msgr2=0x7f990006d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:03.783 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.783+0000 7f9907e00640 1 --2- 172.21.15.145:0/1972287997 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9900163610 0x7f990006d490 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f98f8009d90 tx=0x7f98f802a660 comp rx=0 tx=0).stop 2024-05-09T04:34:03.783 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.783+0000 7f9907e00640 1 -- 172.21.15.145:0/1972287997 shutdown_connections 2024-05-09T04:34:03.783 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.783+0000 7f9907e00640 1 --2- 172.21.15.145:0/1972287997 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9900163610 0x7f990006d490 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:03.783 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.783+0000 7f9907e00640 1 --2- 172.21.15.145:0/1972287997 >> v2:172.21.15.192:3300/0 conn(0x7f9900163b10 0x7f990006eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:03.783 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.783+0000 7f9907e00640 1 -- 172.21.15.145:0/1972287997 >> 172.21.15.145:0/1972287997 conn(0x7f99000d1790 msgr2=0x7f99000d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:03.783 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.783+0000 7f9907e00640 1 -- 172.21.15.145:0/1972287997 shutdown_connections 2024-05-09T04:34:03.784 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.783+0000 7f9907e00640 1 -- 172.21.15.145:0/1972287997 wait complete. 2024-05-09T04:34:03.784 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.785+0000 7f9907e00640 1 Processor -- start 2024-05-09T04:34:03.785 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.785+0000 7f9907e00640 1 -- start start 2024-05-09T04:34:03.787 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.786+0000 7f9907e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f990015f340 0x7f9900161f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:03.787 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.786+0000 7f9907e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9900163610 0x7f99001f85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:03.787 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.786+0000 7f9907e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9900163b10 0x7f99001f8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:03.787 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.786+0000 7f9907e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f990006d9d0 con 0x7f9900163610 2024-05-09T04:34:03.787 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.786+0000 7f9907e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f990015f720 con 0x7f990015f340 2024-05-09T04:34:03.787 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.786+0000 7f9907e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f990006db50 con 0x7f9900163b10 2024-05-09T04:34:03.787 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.786+0000 7f9904a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9900163610 0x7f99001f85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:03.787 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.786+0000 7f9904a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9900163610 0x7f99001f85b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:37856/0 (socket says 172.21.15.145:37856) 2024-05-09T04:34:03.787 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.786+0000 7f9904a00640 1 -- 172.21.15.145:0/4130407963 learned_addr learned my addr 172.21.15.145:0/4130407963 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:03.788 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.787+0000 7f9905400640 1 --2- 172.21.15.145:0/4130407963 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f990015f340 0x7f9900161f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:03.788 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.787+0000 7f9905e00640 1 --2- 172.21.15.145:0/4130407963 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9900163b10 0x7f99001f8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:03.788 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.787+0000 7f9904a00640 1 -- 172.21.15.145:0/4130407963 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9900163b10 msgr2=0x7f99001f8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:03.788 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.787+0000 7f9904a00640 1 --2- 172.21.15.145:0/4130407963 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9900163b10 0x7f99001f8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:03.788 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.787+0000 7f9904a00640 1 -- 172.21.15.145:0/4130407963 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f990015f340 msgr2=0x7f9900161f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:03.788 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.787+0000 7f9904a00640 1 --2- 172.21.15.145:0/4130407963 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f990015f340 0x7f9900161f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:03.788 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.787+0000 7f9904a00640 1 -- 172.21.15.145:0/4130407963 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9900143bb0 con 0x7f9900163610 2024-05-09T04:34:03.788 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.787+0000 7f9904a00640 1 --2- 172.21.15.145:0/4130407963 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9900163610 0x7f99001f85b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f98ec00e8a0 tx=0x7f98ec00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:03.788 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.787+0000 7f98fd600640 1 -- 172.21.15.145:0/4130407963 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ec017070 con 0x7f9900163610 2024-05-09T04:34:03.788 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.787+0000 7f98fd600640 1 -- 172.21.15.145:0/4130407963 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98ec00c940 con 0x7f9900163610 2024-05-09T04:34:03.788 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.787+0000 7f98fd600640 1 -- 172.21.15.145:0/4130407963 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ec00cd30 con 0x7f9900163610 2024-05-09T04:34:03.788 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.787+0000 7f9907e00640 1 -- 172.21.15.145:0/4130407963 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9900157fc0 con 0x7f9900163610 2024-05-09T04:34:03.788 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.787+0000 7f9907e00640 1 -- 172.21.15.145:0/4130407963 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9900158280 con 0x7f9900163610 2024-05-09T04:34:03.789 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.788+0000 7f9907e00640 1 -- 172.21.15.145:0/4130407963 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f990006a310 con 0x7f9900163610 2024-05-09T04:34:03.789 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.789+0000 7f98fd600640 1 -- 172.21.15.145:0/4130407963 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f98ec01a070 con 0x7f9900163610 2024-05-09T04:34:03.790 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.790+0000 7f98fd600640 1 --2- 172.21.15.145:0/4130407963 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f98e00762d0 0x7f98e0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:03.790 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.790+0000 7f98fd600640 1 -- 172.21.15.145:0/4130407963 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f98ec097580 con 0x7f9900163610 2024-05-09T04:34:03.790 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.791+0000 7f9905400640 1 --2- 172.21.15.145:0/4130407963 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f98e00762d0 0x7f98e0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:03.791 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.791+0000 7f9905400640 1 --2- 172.21.15.145:0/4130407963 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f98e00762d0 0x7f98e0078790 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f98f4007a60 tx=0x7f98f4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:03.794 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:03.795+0000 7f98fd600640 1 -- 172.21.15.145:0/4130407963 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f98ec060e40 con 0x7f9900163610 2024-05-09T04:34:04.038 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.039+0000 7f9907e00640 1 -- 172.21.15.145:0/4130407963 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f99000d1e50 con 0x7f9900163610 2024-05-09T04:34:04.040 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.040+0000 7f98fd600640 1 -- 172.21.15.145:0/4130407963 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5187 (secure 0 0 0) 0x7f98ec060740 con 0x7f9900163610 2024-05-09T04:34:04.040 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:34:04.040 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/1284540932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":1284540932},{"type":"v1","addr":"172.21.15.145:6835","nonce":1284540932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/1308656248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1308656248},{"type":"v1","addr":"172.21.15.145:6837","nonce":1308656248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3323136057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3323136057},{"type":"v1","addr":"172.21.15.192:6833","nonce":3323136057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T04:33:51.628921+0000","modified":"2024-05-09T04:33:54.699907+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.192:6835/1113351558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1113351558},{"type":"v1","addr":"172.21.15.192:6835","nonce":1113351558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4270,"qdb_cluster":[4270]},"id":1}]} 2024-05-09T04:34:04.041 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 8 2024-05-09T04:34:04.043 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.043+0000 7f9907e00640 1 -- 172.21.15.145:0/4130407963 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f98e00762d0 msgr2=0x7f98e0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:04.043 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.043+0000 7f9907e00640 1 --2- 172.21.15.145:0/4130407963 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f98e00762d0 0x7f98e0078790 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f98f4007a60 tx=0x7f98f4009040 comp rx=0 tx=0).stop 2024-05-09T04:34:04.043 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.043+0000 7f9907e00640 1 -- 172.21.15.145:0/4130407963 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9900163610 msgr2=0x7f99001f85b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:04.043 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.043+0000 7f9907e00640 1 --2- 172.21.15.145:0/4130407963 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9900163610 0x7f99001f85b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f98ec00e8a0 tx=0x7f98ec00ed60 comp rx=0 tx=0).stop 2024-05-09T04:34:04.043 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.043+0000 7f9907e00640 1 -- 172.21.15.145:0/4130407963 shutdown_connections 2024-05-09T04:34:04.043 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.043+0000 7f9907e00640 1 --2- 172.21.15.145:0/4130407963 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f98e00762d0 0x7f98e0078790 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:04.043 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.043+0000 7f9907e00640 1 --2- 172.21.15.145:0/4130407963 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9900163b10 0x7f99001f8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:04.043 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.043+0000 7f9907e00640 1 --2- 172.21.15.145:0/4130407963 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9900163610 0x7f99001f85b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:04.044 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.043+0000 7f9907e00640 1 --2- 172.21.15.145:0/4130407963 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f990015f340 0x7f9900161f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:04.044 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.043+0000 7f9907e00640 1 -- 172.21.15.145:0/4130407963 >> 172.21.15.145:0/4130407963 conn(0x7f99000d1790 msgr2=0x7f990006f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:04.044 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.043+0000 7f9907e00640 1 -- 172.21.15.145:0/4130407963 shutdown_connections 2024-05-09T04:34:04.044 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.043+0000 7f9907e00640 1 -- 172.21.15.145:0/4130407963 wait complete. 2024-05-09T04:34:04.062 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T04:34:04.303 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.303+0000 7f9814600640 1 Processor -- start 2024-05-09T04:34:04.303 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.303+0000 7f9814600640 1 -- start start 2024-05-09T04:34:04.305 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.304+0000 7f9814600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f980c06e860 0x7f980c06ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:04.305 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.304+0000 7f9814600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f980c0b4910 0x7f980c0b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:04.305 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.304+0000 7f9814600640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f980c06f180 con 0x7f980c15d780 2024-05-09T04:34:04.305 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.304+0000 7f9814600640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f980c161360 con 0x7f980c0b4910 2024-05-09T04:34:04.305 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.304+0000 7f9814600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f980c1615c0 con 0x7f980c06e860 2024-05-09T04:34:04.305 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.305+0000 7f9812600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f980c0b4910 0x7f980c0b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:04.305 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.305+0000 7f9812600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f980c0b4910 0x7f980c0b4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:37872/0 (socket says 172.21.15.145:37872) 2024-05-09T04:34:04.305 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.305+0000 7f9812600640 1 -- 172.21.15.145:0/848141202 learned_addr learned my addr 172.21.15.145:0/848141202 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:04.306 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.305+0000 7f9811c00640 1 --2- 172.21.15.145:0/848141202 >> v2:172.21.15.192:3300/0 conn(0x7f980c06e860 0x7f980c06ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:04.306 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.305+0000 7f9812600640 1 -- 172.21.15.145:0/848141202 >> v2:172.21.15.192:3300/0 conn(0x7f980c06e860 msgr2=0x7f980c06ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:04.306 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.305+0000 7f9812600640 1 --2- 172.21.15.145:0/848141202 >> v2:172.21.15.192:3300/0 conn(0x7f980c06e860 0x7f980c06ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:04.306 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.305+0000 7f9812600640 1 -- 172.21.15.145:0/848141202 >> v1:172.21.15.192:6789/0 conn(0x7f980c15d780 legacy=0x7f980c160c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:04.306 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.305+0000 7f9812600640 1 -- 172.21.15.145:0/848141202 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f980c1618a0 con 0x7f980c0b4910 2024-05-09T04:34:04.307 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.306+0000 7f9812600640 1 --2- 172.21.15.145:0/848141202 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f980c0b4910 0x7f980c0b4d50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f980c0745a0 tx=0x7f980402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70e5d6a3bf47b375 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:04.307 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.306+0000 7f980be00640 1 -- 172.21.15.145:0/848141202 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9804036070 con 0x7f980c0b4910 2024-05-09T04:34:04.307 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.307+0000 7f980be00640 1 -- 172.21.15.145:0/848141202 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9804031600 con 0x7f980c0b4910 2024-05-09T04:34:04.307 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.307+0000 7f980be00640 1 -- 172.21.15.145:0/848141202 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98040319d0 con 0x7f980c0b4910 2024-05-09T04:34:04.307 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.307+0000 7f9814600640 1 -- 172.21.15.145:0/848141202 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f980c0b4910 msgr2=0x7f980c0b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:04.307 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.307+0000 7f9814600640 1 --2- 172.21.15.145:0/848141202 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f980c0b4910 0x7f980c0b4d50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f980c0745a0 tx=0x7f980402a6a0 comp rx=0 tx=0).stop 2024-05-09T04:34:04.307 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.307+0000 7f9814600640 1 -- 172.21.15.145:0/848141202 shutdown_connections 2024-05-09T04:34:04.308 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.307+0000 7f9814600640 1 --2- 172.21.15.145:0/848141202 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f980c0b4910 0x7f980c0b4d50 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:04.308 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.307+0000 7f9814600640 1 --2- 172.21.15.145:0/848141202 >> v2:172.21.15.192:3300/0 conn(0x7f980c06e860 0x7f980c06ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:04.308 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.307+0000 7f9814600640 1 -- 172.21.15.145:0/848141202 >> 172.21.15.145:0/848141202 conn(0x7f980c06c870 msgr2=0x7f980c06cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:04.308 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.307+0000 7f9814600640 1 -- 172.21.15.145:0/848141202 shutdown_connections 2024-05-09T04:34:04.308 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.307+0000 7f9814600640 1 -- 172.21.15.145:0/848141202 wait complete. 2024-05-09T04:34:04.309 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.309+0000 7f9814600640 1 Processor -- start 2024-05-09T04:34:04.309 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.309+0000 7f9814600640 1 -- start start 2024-05-09T04:34:04.311 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.311+0000 7f9814600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f980c06e860 0x7f980c13e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:04.311 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.311+0000 7f9814600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f980c0b4910 0x7f980c13cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:04.311 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.311+0000 7f9814600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f980c15d780 0x7f980c15fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:04.312 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.311+0000 7f9814600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f980c146c00 con 0x7f980c06e860 2024-05-09T04:34:04.312 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.311+0000 7f9814600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f980c0b43f0 con 0x7f980c0b4910 2024-05-09T04:34:04.312 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.311+0000 7f9814600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f980c146d80 con 0x7f980c15d780 2024-05-09T04:34:04.312 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.311+0000 7f9811c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f980c06e860 0x7f980c13e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:04.312 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.311+0000 7f9811c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f980c06e860 0x7f980c13e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:37880/0 (socket says 172.21.15.145:37880) 2024-05-09T04:34:04.312 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.311+0000 7f9811c00640 1 -- 172.21.15.145:0/866820742 learned_addr learned my addr 172.21.15.145:0/866820742 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:04.312 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.311+0000 7f9811200640 1 --2- 172.21.15.145:0/866820742 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f980c0b4910 0x7f980c13cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:04.312 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.311+0000 7f9812600640 1 --2- 172.21.15.145:0/866820742 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f980c15d780 0x7f980c15fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:04.312 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.311+0000 7f9811c00640 1 -- 172.21.15.145:0/866820742 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f980c15d780 msgr2=0x7f980c15fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:04.312 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.311+0000 7f9811c00640 1 --2- 172.21.15.145:0/866820742 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f980c15d780 0x7f980c15fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:04.312 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.311+0000 7f9811c00640 1 -- 172.21.15.145:0/866820742 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f980c0b4910 msgr2=0x7f980c13cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:04.312 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.311+0000 7f9811c00640 1 --2- 172.21.15.145:0/866820742 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f980c0b4910 0x7f980c13cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:04.312 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.311+0000 7f9811c00640 1 -- 172.21.15.145:0/866820742 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f980c142b20 con 0x7f980c06e860 2024-05-09T04:34:04.312 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.311+0000 7f9811c00640 1 --2- 172.21.15.145:0/866820742 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f980c06e860 0x7f980c13e5e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f980000c820 tx=0x7f980000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:04.313 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.311+0000 7f9809600640 1 -- 172.21.15.145:0/866820742 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9800015070 con 0x7f980c06e860 2024-05-09T04:34:04.313 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.311+0000 7f9809600640 1 -- 172.21.15.145:0/866820742 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9800004240 con 0x7f980c06e860 2024-05-09T04:34:04.313 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.311+0000 7f9809600640 1 -- 172.21.15.145:0/866820742 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980000ce10 con 0x7f980c06e860 2024-05-09T04:34:04.313 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.311+0000 7f9814600640 1 -- 172.21.15.145:0/866820742 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f980c13d820 con 0x7f980c06e860 2024-05-09T04:34:04.313 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.312+0000 7f9814600640 1 -- 172.21.15.145:0/866820742 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f980c13dde0 con 0x7f980c06e860 2024-05-09T04:34:04.313 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.313+0000 7f9814600640 1 -- 172.21.15.145:0/866820742 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f980c068df0 con 0x7f980c06e860 2024-05-09T04:34:04.313 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.314+0000 7f9809600640 1 -- 172.21.15.145:0/866820742 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9800007830 con 0x7f980c06e860 2024-05-09T04:34:04.315 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.315+0000 7f9809600640 1 --2- 172.21.15.145:0/866820742 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f97ec076300 0x7f97ec0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:04.315 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.315+0000 7f9809600640 1 -- 172.21.15.145:0/866820742 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9800097410 con 0x7f980c06e860 2024-05-09T04:34:04.315 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.315+0000 7f9811200640 1 --2- 172.21.15.145:0/866820742 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f97ec076300 0x7f97ec0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:04.315 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.316+0000 7f9811200640 1 --2- 172.21.15.145:0/866820742 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f97ec076300 0x7f97ec0787c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f97f8006fd0 tx=0x7f97f8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:04.319 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.320+0000 7f9809600640 1 -- 172.21.15.145:0/866820742 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f980c068df0 con 0x7f980c06e860 2024-05-09T04:34:04.528 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.528+0000 7f9814600640 1 -- 172.21.15.145:0/866820742 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f980c06a310 con 0x7f980c06e860 2024-05-09T04:34:04.531 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.531+0000 7f9809600640 1 -- 172.21.15.145:0/866820742 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16653 (secure 0 0 0) 0x7f980c142b20 con 0x7f980c06e860 2024-05-09T04:34:04.532 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:34:04.532 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":20,"fsid":"957ee6c1-073b-44cf-9b93-9491b3fdfc1d","created":"2024-05-09T04:33:26.544367+0000","modified":"2024-05-09T04:33:53.178020+0000","last_up_change":"2024-05-09T04:33:39.039318+0000","last_in_change":"2024-05-09T04:33:29.902361+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T04:33:39.795344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T04:33:49.651743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T04:33:50.596065+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22e550ad-847b-4952-80e6-63d859ed1a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6801","nonce":3182572912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6803","nonce":3182572912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6807","nonce":3182572912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6805","nonce":3182572912}]},"public_addr":"172.21.15.145:6801/3182572912","cluster_addr":"172.21.15.145:6803/3182572912","heartbeat_back_addr":"172.21.15.145:6807/3182572912","heartbeat_front_addr":"172.21.15.145:6805/3182572912","state":["exists","up"]},{"osd":1,"uuid":"0f6b9fca-0c41-4145-8d74-598ddb42f2a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6825","nonce":1845225555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6827","nonce":1845225555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6831","nonce":1845225555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6829","nonce":1845225555}]},"public_addr":"172.21.15.145:6825/1845225555","cluster_addr":"172.21.15.145:6827/1845225555","heartbeat_back_addr":"172.21.15.145:6831/1845225555","heartbeat_front_addr":"172.21.15.145:6829/1845225555","state":["exists","up"]},{"osd":2,"uuid":"b3b1f67e-5790-483f-8610-e05a5bce4c30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6817","nonce":2168958993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6819","nonce":2168958993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6823","nonce":2168958993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6821","nonce":2168958993}]},"public_addr":"172.21.15.145:6817/2168958993","cluster_addr":"172.21.15.145:6819/2168958993","heartbeat_back_addr":"172.21.15.145:6823/2168958993","heartbeat_front_addr":"172.21.15.145:6821/2168958993","state":["exists","up"]},{"osd":3,"uuid":"884787a5-e037-4d31-9aae-743030eba4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6809","nonce":337009826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6811","nonce":337009826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6815","nonce":337009826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6813","nonce":337009826}]},"public_addr":"172.21.15.145:6809/337009826","cluster_addr":"172.21.15.145:6811/337009826","heartbeat_back_addr":"172.21.15.145:6815/337009826","heartbeat_front_addr":"172.21.15.145:6813/337009826","state":["exists","up"]},{"osd":4,"uuid":"0759fc1e-c5fc-4a2f-86bb-e8eccd83f833","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6818","nonce":3117700995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6821","nonce":3117700995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6827","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6829","nonce":3117700995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6823","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6825","nonce":3117700995}]},"public_addr":"172.21.15.192:6818/3117700995","cluster_addr":"172.21.15.192:6821/3117700995","heartbeat_back_addr":"172.21.15.192:6829/3117700995","heartbeat_front_addr":"172.21.15.192:6825/3117700995","state":["exists","up"]},{"osd":5,"uuid":"9443cc22-9379-46e3-9173-98b592bf2c16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6801","nonce":1328373268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6803","nonce":1328373268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6807","nonce":1328373268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6805","nonce":1328373268}]},"public_addr":"172.21.15.192:6801/1328373268","cluster_addr":"172.21.15.192:6803/1328373268","heartbeat_back_addr":"172.21.15.192:6807/1328373268","heartbeat_front_addr":"172.21.15.192:6805/1328373268","state":["exists","up"]},{"osd":6,"uuid":"d05a3311-505a-45b5-85b9-aa3113e96987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6817","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6820","nonce":1868011057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6824","nonce":1868011057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6831","nonce":1868011057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6828","nonce":1868011057}]},"public_addr":"172.21.15.192:6820/1868011057","cluster_addr":"172.21.15.192:6824/1868011057","heartbeat_back_addr":"172.21.15.192:6831/1868011057","heartbeat_front_addr":"172.21.15.192:6828/1868011057","state":["exists","up"]},{"osd":7,"uuid":"c7c23dbc-2029-43ec-9f53-25f3d58019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6809","nonce":3368512157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6811","nonce":3368512157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6815","nonce":3368512157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6813","nonce":3368512157}]},"public_addr":"172.21.15.192:6809/3368512157","cluster_addr":"172.21.15.192:6811/3368512157","heartbeat_back_addr":"172.21.15.192:6815/3368512157","heartbeat_front_addr":"172.21.15.192:6813/3368512157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.091188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.032567+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:34.797468+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.228839+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.622400+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.473103+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.566092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.349395+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T04:34:04.535 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.534+0000 7f9814600640 1 -- 172.21.15.145:0/866820742 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f97ec076300 msgr2=0x7f97ec0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:04.535 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.534+0000 7f9814600640 1 --2- 172.21.15.145:0/866820742 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f97ec076300 0x7f97ec0787c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f97f8006fd0 tx=0x7f97f8008040 comp rx=0 tx=0).stop 2024-05-09T04:34:04.535 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.534+0000 7f9814600640 1 -- 172.21.15.145:0/866820742 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f980c06e860 msgr2=0x7f980c13e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:04.535 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.534+0000 7f9814600640 1 --2- 172.21.15.145:0/866820742 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f980c06e860 0x7f980c13e5e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f980000c820 tx=0x7f980000cce0 comp rx=0 tx=0).stop 2024-05-09T04:34:04.535 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.535+0000 7f9814600640 1 -- 172.21.15.145:0/866820742 shutdown_connections 2024-05-09T04:34:04.535 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.535+0000 7f9814600640 1 --2- 172.21.15.145:0/866820742 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f97ec076300 0x7f97ec0787c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:04.535 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.535+0000 7f9814600640 1 --2- 172.21.15.145:0/866820742 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f980c15d780 0x7f980c15fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:04.535 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.535+0000 7f9814600640 1 --2- 172.21.15.145:0/866820742 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f980c0b4910 0x7f980c13cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:04.535 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.535+0000 7f9814600640 1 --2- 172.21.15.145:0/866820742 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f980c06e860 0x7f980c13e5e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:04.535 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.535+0000 7f9814600640 1 -- 172.21.15.145:0/866820742 >> 172.21.15.145:0/866820742 conn(0x7f980c06c870 msgr2=0x7f980c15b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:04.535 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.535+0000 7f9814600640 1 -- 172.21.15.145:0/866820742 shutdown_connections 2024-05-09T04:34:04.535 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.535+0000 7f9814600640 1 -- 172.21.15.145:0/866820742 wait complete. 2024-05-09T04:34:04.553 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T04:34:04.749 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.749+0000 7fea83e00640 1 Processor -- start 2024-05-09T04:34:04.749 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.749+0000 7fea83e00640 1 -- start start 2024-05-09T04:34:04.751 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.751+0000 7fea83e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fea7c15fbc0 0x7fea7c142200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:04.751 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.751+0000 7fea83e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fea7c15ffa0 0x7fea7c0cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:04.751 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.751+0000 7fea83e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fea7c0cfe10 0x7fea7c15c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:04.751 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.751+0000 7fea83e00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fea7c072220 con 0x7fea7c0cfe10 2024-05-09T04:34:04.751 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.751+0000 7fea83e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fea7c13f340 con 0x7fea7c15fbc0 2024-05-09T04:34:04.752 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.751+0000 7fea83e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fea7c0d02b0 con 0x7fea7c15ffa0 2024-05-09T04:34:04.752 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.751+0000 7fea81e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fea7c0cfe10 0x7fea7c15c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:04.752 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.751+0000 7fea81e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fea7c0cfe10 0x7fea7c15c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:37884/0 (socket says 172.21.15.145:37884) 2024-05-09T04:34:04.752 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.751+0000 7fea81e00640 1 -- 172.21.15.145:0/2130660677 learned_addr learned my addr 172.21.15.145:0/2130660677 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:04.752 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.751+0000 7fea80a00640 1 --2- 172.21.15.145:0/2130660677 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fea7c15ffa0 0x7fea7c0cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:04.752 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.751+0000 7fea81400640 1 --2- 172.21.15.145:0/2130660677 >> v2:172.21.15.192:3300/0 conn(0x7fea7c15fbc0 0x7fea7c142200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:04.752 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.752+0000 7fea81e00640 1 -- 172.21.15.145:0/2130660677 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fea7c15ffa0 msgr2=0x7fea7c0cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:04.752 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.752+0000 7fea81e00640 1 --2- 172.21.15.145:0/2130660677 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fea7c15ffa0 0x7fea7c0cf8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:04.752 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.752+0000 7fea81e00640 1 -- 172.21.15.145:0/2130660677 >> v2:172.21.15.192:3300/0 conn(0x7fea7c15fbc0 msgr2=0x7fea7c142200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:04.752 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.752+0000 7fea81e00640 1 --2- 172.21.15.145:0/2130660677 >> v2:172.21.15.192:3300/0 conn(0x7fea7c15fbc0 0x7fea7c142200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:04.752 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.752+0000 7fea81e00640 1 -- 172.21.15.145:0/2130660677 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea7c0d0590 con 0x7fea7c0cfe10 2024-05-09T04:34:04.753 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.753+0000 7fea81e00640 1 --2- 172.21.15.145:0/2130660677 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fea7c0cfe10 0x7fea7c15c5d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fea7c0705e0 tx=0x7fea7402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f09f50b1945bf0c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:04.753 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.753+0000 7fea7be00640 1 -- 172.21.15.145:0/2130660677 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea74036070 con 0x7fea7c0cfe10 2024-05-09T04:34:04.753 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.753+0000 7fea7be00640 1 -- 172.21.15.145:0/2130660677 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea74031600 con 0x7fea7c0cfe10 2024-05-09T04:34:04.753 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.753+0000 7fea7be00640 1 -- 172.21.15.145:0/2130660677 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea740319d0 con 0x7fea7c0cfe10 2024-05-09T04:34:04.754 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.754+0000 7fea83e00640 1 -- 172.21.15.145:0/2130660677 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fea7c0cfe10 msgr2=0x7fea7c15c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:04.754 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.754+0000 7fea83e00640 1 --2- 172.21.15.145:0/2130660677 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fea7c0cfe10 0x7fea7c15c5d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fea7c0705e0 tx=0x7fea7402a610 comp rx=0 tx=0).stop 2024-05-09T04:34:04.754 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.754+0000 7fea83e00640 1 -- 172.21.15.145:0/2130660677 shutdown_connections 2024-05-09T04:34:04.754 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.754+0000 7fea83e00640 1 --2- 172.21.15.145:0/2130660677 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fea7c0cfe10 0x7fea7c15c5d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:04.754 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.754+0000 7fea83e00640 1 --2- 172.21.15.145:0/2130660677 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fea7c15ffa0 0x7fea7c0cf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:04.754 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.754+0000 7fea83e00640 1 --2- 172.21.15.145:0/2130660677 >> v2:172.21.15.192:3300/0 conn(0x7fea7c15fbc0 0x7fea7c142200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:04.754 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.754+0000 7fea83e00640 1 -- 172.21.15.145:0/2130660677 >> 172.21.15.145:0/2130660677 conn(0x7fea7c06c870 msgr2=0x7fea7c06cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:04.754 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.754+0000 7fea83e00640 1 -- 172.21.15.145:0/2130660677 shutdown_connections 2024-05-09T04:34:04.754 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.754+0000 7fea83e00640 1 -- 172.21.15.145:0/2130660677 wait complete. 2024-05-09T04:34:04.755 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.756+0000 7fea83e00640 1 Processor -- start 2024-05-09T04:34:04.755 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.756+0000 7fea83e00640 1 -- start start 2024-05-09T04:34:04.758 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.757+0000 7fea83e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fea7c0cfe10 0x7fea7c13d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:04.758 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.757+0000 7fea83e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea7c15fbc0 0x7fea7c15e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:04.758 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.757+0000 7fea83e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fea7c15ffa0 0x7fea7c140f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:04.758 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.757+0000 7fea83e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fea7c160400 con 0x7fea7c15ffa0 2024-05-09T04:34:04.758 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.757+0000 7fea83e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fea7c0d0b50 con 0x7fea7c15fbc0 2024-05-09T04:34:04.758 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.757+0000 7fea83e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fea7c15cb10 con 0x7fea7c0cfe10 2024-05-09T04:34:04.758 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.757+0000 7fea81e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fea7c15ffa0 0x7fea7c140f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:04.758 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.757+0000 7fea81e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fea7c15ffa0 0x7fea7c140f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:37892/0 (socket says 172.21.15.145:37892) 2024-05-09T04:34:04.758 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.757+0000 7fea81e00640 1 -- 172.21.15.145:0/3911989323 learned_addr learned my addr 172.21.15.145:0/3911989323 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:04.759 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.757+0000 7fea81400640 1 --2- 172.21.15.145:0/3911989323 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fea7c0cfe10 0x7fea7c13d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:04.759 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.757+0000 7fea80a00640 1 --2- 172.21.15.145:0/3911989323 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea7c15fbc0 0x7fea7c15e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:04.759 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.757+0000 7fea81e00640 1 -- 172.21.15.145:0/3911989323 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fea7c0cfe10 msgr2=0x7fea7c13d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:04.759 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.757+0000 7fea81e00640 1 --2- 172.21.15.145:0/3911989323 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fea7c0cfe10 0x7fea7c13d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:04.759 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.757+0000 7fea81e00640 1 -- 172.21.15.145:0/3911989323 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea7c15fbc0 msgr2=0x7fea7c15e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:04.759 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.757+0000 7fea81e00640 1 --2- 172.21.15.145:0/3911989323 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea7c15fbc0 0x7fea7c15e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:04.759 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.757+0000 7fea81e00640 1 -- 172.21.15.145:0/3911989323 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea7c153720 con 0x7fea7c15ffa0 2024-05-09T04:34:04.759 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.757+0000 7fea81400640 1 --2- 172.21.15.145:0/3911989323 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fea7c0cfe10 0x7fea7c13d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:34:04.759 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.757+0000 7fea80a00640 1 --2- 172.21.15.145:0/3911989323 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea7c15fbc0 0x7fea7c15e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:34:04.759 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.758+0000 7fea81e00640 1 --2- 172.21.15.145:0/3911989323 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fea7c15ffa0 0x7fea7c140f40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fea74034040 tx=0x7fea74004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:04.759 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.758+0000 7fea79600640 1 -- 172.21.15.145:0/3911989323 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea74036050 con 0x7fea7c15ffa0 2024-05-09T04:34:04.759 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.758+0000 7fea79600640 1 -- 172.21.15.145:0/3911989323 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea74042070 con 0x7fea7c15ffa0 2024-05-09T04:34:04.759 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.758+0000 7fea79600640 1 -- 172.21.15.145:0/3911989323 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea74009510 con 0x7fea7c15ffa0 2024-05-09T04:34:04.759 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.758+0000 7fea83e00640 1 -- 172.21.15.145:0/3911989323 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea7c153a00 con 0x7fea7c15ffa0 2024-05-09T04:34:04.759 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.758+0000 7fea83e00640 1 -- 172.21.15.145:0/3911989323 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea7c13de40 con 0x7fea7c15ffa0 2024-05-09T04:34:04.760 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.759+0000 7fea83e00640 1 -- 172.21.15.145:0/3911989323 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea7c06a310 con 0x7fea7c15ffa0 2024-05-09T04:34:04.760 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.760+0000 7fea79600640 1 -- 172.21.15.145:0/3911989323 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fea74007bd0 con 0x7fea7c15ffa0 2024-05-09T04:34:04.761 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.761+0000 7fea79600640 1 --2- 172.21.15.145:0/3911989323 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fea5c0762d0 0x7fea5c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:04.761 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.761+0000 7fea79600640 1 -- 172.21.15.145:0/3911989323 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fea740b84c0 con 0x7fea7c15ffa0 2024-05-09T04:34:04.761 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.761+0000 7fea81400640 1 --2- 172.21.15.145:0/3911989323 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fea5c0762d0 0x7fea5c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:04.762 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.762+0000 7fea81400640 1 --2- 172.21.15.145:0/3911989323 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fea5c0762d0 0x7fea5c078790 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fea70006fd0 tx=0x7fea70008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:04.765 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:04.766+0000 7fea79600640 1 -- 172.21.15.145:0/3911989323 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fea74081e10 con 0x7fea7c15ffa0 2024-05-09T04:34:05.009 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.011+0000 7fea83e00640 1 -- 172.21.15.145:0/3911989323 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea7c0cf9b0 con 0x7fea7c15ffa0 2024-05-09T04:34:05.010 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.012+0000 7fea79600640 1 -- 172.21.15.145:0/3911989323 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5187 (secure 0 0 0) 0x7fea74081710 con 0x7fea7c15ffa0 2024-05-09T04:34:05.010 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:34:05.011 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/1284540932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":1284540932},{"type":"v1","addr":"172.21.15.145:6835","nonce":1284540932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/1308656248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1308656248},{"type":"v1","addr":"172.21.15.145:6837","nonce":1308656248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3323136057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3323136057},{"type":"v1","addr":"172.21.15.192:6833","nonce":3323136057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T04:33:51.628921+0000","modified":"2024-05-09T04:33:54.699907+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.192:6835/1113351558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1113351558},{"type":"v1","addr":"172.21.15.192:6835","nonce":1113351558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4270,"qdb_cluster":[4270]},"id":1}]} 2024-05-09T04:34:05.011 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 8 2024-05-09T04:34:05.013 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.014+0000 7fea83e00640 1 -- 172.21.15.145:0/3911989323 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fea5c0762d0 msgr2=0x7fea5c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:05.014 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.014+0000 7fea83e00640 1 --2- 172.21.15.145:0/3911989323 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fea5c0762d0 0x7fea5c078790 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fea70006fd0 tx=0x7fea70008040 comp rx=0 tx=0).stop 2024-05-09T04:34:05.014 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.014+0000 7fea83e00640 1 -- 172.21.15.145:0/3911989323 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fea7c15ffa0 msgr2=0x7fea7c140f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:05.014 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.014+0000 7fea83e00640 1 --2- 172.21.15.145:0/3911989323 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fea7c15ffa0 0x7fea7c140f40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fea74034040 tx=0x7fea74004a40 comp rx=0 tx=0).stop 2024-05-09T04:34:05.014 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.015+0000 7fea83e00640 1 -- 172.21.15.145:0/3911989323 shutdown_connections 2024-05-09T04:34:05.014 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.015+0000 7fea83e00640 1 --2- 172.21.15.145:0/3911989323 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fea5c0762d0 0x7fea5c078790 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:05.014 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.015+0000 7fea83e00640 1 --2- 172.21.15.145:0/3911989323 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fea7c15ffa0 0x7fea7c140f40 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:05.014 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.015+0000 7fea83e00640 1 --2- 172.21.15.145:0/3911989323 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea7c15fbc0 0x7fea7c15e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:05.014 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.015+0000 7fea83e00640 1 --2- 172.21.15.145:0/3911989323 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fea7c0cfe10 0x7fea7c13d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:05.014 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.015+0000 7fea83e00640 1 -- 172.21.15.145:0/3911989323 >> 172.21.15.145:0/3911989323 conn(0x7fea7c06c870 msgr2=0x7fea7c0b3510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:05.014 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.015+0000 7fea83e00640 1 -- 172.21.15.145:0/3911989323 shutdown_connections 2024-05-09T04:34:05.014 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.015+0000 7fea83e00640 1 -- 172.21.15.145:0/3911989323 wait complete. 2024-05-09T04:34:05.436 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-09T04:33:51.628921+0000', 'modified': '2024-05-09T04:33:54.699907+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4270}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4270': {'gid': 4270, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.192:6835/1113351558', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6834', 'nonce': 1113351558}, {'type': 'v1', 'addr': '172.21.15.192:6835', 'nonce': 1113351558}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4270, 'qdb_cluster': [4270]} 2024-05-09T04:34:05.436 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-09T04:34:05.436 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-09T04:34:05.644 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.645+0000 7f8f9b200640 1 Processor -- start 2024-05-09T04:34:05.644 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.645+0000 7f8f9b200640 1 -- start start 2024-05-09T04:34:05.646 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.647+0000 7f8f9b200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f8f94145ad0 0x7f8f9406e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:05.646 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.647+0000 7f8f9b200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8f940ce440 0x7f8f9415d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:05.646 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.647+0000 7f8f9b200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f9406ebd0 con 0x7f8f940ce060 2024-05-09T04:34:05.646 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.647+0000 7f8f9b200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f8f94073460 con 0x7f8f94145ad0 2024-05-09T04:34:05.646 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.647+0000 7f8f9b200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8f9406f750 con 0x7f8f940ce440 2024-05-09T04:34:05.646 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.648+0000 7f8f93e00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f8f940ce060 0x7f8f941453c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:37074/0 (socket says 172.21.15.145:37074) 2024-05-09T04:34:05.647 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.648+0000 7f8f93e00640 1 -- 172.21.15.145:0/1673707061 learned_addr learned my addr 172.21.15.145:0/1673707061 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:34:05.648 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.648+0000 7f8f99200640 1 --2- 172.21.15.145:0/1673707061 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8f940ce440 0x7f8f9415d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:05.648 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.648+0000 7f8f8b400640 1 --2- 172.21.15.145:0/1673707061 >> v2:172.21.15.192:3300/0 conn(0x7f8f94145ad0 0x7f8f9406e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:05.649 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.648+0000 7f8f93400640 1 -- 172.21.15.145:0/1673707061 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3694592238 0 0) 0x7f8f9406ebd0 con 0x7f8f940ce060 2024-05-09T04:34:05.649 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.648+0000 7f8f93400640 1 -- 172.21.15.145:0/1673707061 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f78003520 con 0x7f8f940ce060 2024-05-09T04:34:05.649 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.649+0000 7f8f93400640 1 -- 172.21.15.145:0/1673707061 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1521897761 0 0) 0x7f8f78003520 con 0x7f8f940ce060 2024-05-09T04:34:05.649 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.649+0000 7f8f93400640 1 -- 172.21.15.145:0/1673707061 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8f940ce440 msgr2=0x7f8f9415d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:05.649 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.649+0000 7f8f93400640 1 --2- 172.21.15.145:0/1673707061 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8f940ce440 0x7f8f9415d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:05.649 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.649+0000 7f8f93400640 1 -- 172.21.15.145:0/1673707061 >> v2:172.21.15.192:3300/0 conn(0x7f8f94145ad0 msgr2=0x7f8f9406e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:05.649 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.649+0000 7f8f93400640 1 --2- 172.21.15.145:0/1673707061 >> v2:172.21.15.192:3300/0 conn(0x7f8f94145ad0 0x7f8f9406e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:05.649 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.649+0000 7f8f93400640 1 -- 172.21.15.145:0/1673707061 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f9406eeb0 con 0x7f8f940ce060 2024-05-09T04:34:05.649 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.649+0000 7f8f93400640 1 -- 172.21.15.145:0/1673707061 <== mon.0 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3474413261 0 0) 0x7f8f840016f0 con 0x7f8f940ce060 2024-05-09T04:34:05.649 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.649+0000 7f8f93400640 1 -- 172.21.15.145:0/1673707061 >> v1:172.21.15.145:6789/0 conn(0x7f8f940ce060 legacy=0x7f8f941453c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:05.649 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.649+0000 7f8f93400640 1 --2- 172.21.15.145:0/1673707061 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f78003a80 0x7f8f78003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:05.649 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.649+0000 7f8f93400640 1 --2- 172.21.15.145:0/1673707061 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8f940ce440 0x7f8f78004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:05.649 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.649+0000 7f8f93400640 1 --2- 172.21.15.145:0/1673707061 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8f78004d40 0x7f8f780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:05.649 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.649+0000 7f8f93400640 1 -- 172.21.15.145:0/1673707061 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9406f750 con 0x7f8f78004d40 2024-05-09T04:34:05.649 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.649+0000 7f8f93400640 1 -- 172.21.15.145:0/1673707061 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7800b9c0 con 0x7f8f78003a80 2024-05-09T04:34:05.650 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.649+0000 7f8f93400640 1 -- 172.21.15.145:0/1673707061 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8f7800ddd0 con 0x7f8f940ce440 2024-05-09T04:34:05.650 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.649+0000 7f8f99200640 1 --2- 172.21.15.145:0/1673707061 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8f78004d40 0x7f8f780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:05.650 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.649+0000 7f8f93e00640 1 --2- 172.21.15.145:0/1673707061 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f78003a80 0x7f8f78003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:05.650 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.649+0000 7f8f99200640 1 -- 172.21.15.145:0/1673707061 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8f940ce440 msgr2=0x7f8f78004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:05.650 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.649+0000 7f8f99200640 1 --2- 172.21.15.145:0/1673707061 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8f940ce440 0x7f8f78004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:05.650 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.649+0000 7f8f99200640 1 -- 172.21.15.145:0/1673707061 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f78003a80 msgr2=0x7f8f78003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:05.650 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.649+0000 7f8f99200640 1 --2- 172.21.15.145:0/1673707061 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f78003a80 0x7f8f78003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:05.650 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.649+0000 7f8f99200640 1 -- 172.21.15.145:0/1673707061 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f78003520 con 0x7f8f78004d40 2024-05-09T04:34:05.650 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.649+0000 7f8f93e00640 1 --2- 172.21.15.145:0/1673707061 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f78003a80 0x7f8f78003ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:34:05.662 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.650+0000 7f8f99200640 1 --2- 172.21.15.145:0/1673707061 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8f78004d40 0x7f8f780051c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c009ac0 tx=0x7f8f78005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81c66b300c36dc89 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:05.662 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.651+0000 7f8f93400640 1 -- 172.21.15.145:0/1673707061 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c036070 con 0x7f8f78004d40 2024-05-09T04:34:05.662 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.651+0000 7f8f93400640 1 -- 172.21.15.145:0/1673707061 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f8c031400 con 0x7f8f78004d40 2024-05-09T04:34:05.662 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.651+0000 7f8f93400640 1 -- 172.21.15.145:0/1673707061 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c009690 con 0x7f8f78004d40 2024-05-09T04:34:05.662 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.651+0000 7f8f9b200640 1 -- 172.21.15.145:0/1673707061 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8f78004d40 msgr2=0x7f8f780051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:05.662 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.651+0000 7f8f9b200640 1 --2- 172.21.15.145:0/1673707061 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8f78004d40 0x7f8f780051c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c009ac0 tx=0x7f8f78005830 comp rx=0 tx=0).stop 2024-05-09T04:34:05.662 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.651+0000 7f8f99200640 1 -- 172.21.15.145:0/1673707061 reap_dead start 2024-05-09T04:34:05.663 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.651+0000 7f8f9b200640 1 -- 172.21.15.145:0/1673707061 shutdown_connections 2024-05-09T04:34:05.663 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.651+0000 7f8f9b200640 1 -- 172.21.15.145:0/1673707061 >> 172.21.15.145:0/1673707061 conn(0x7f8f9406c8d0 msgr2=0x7f8f9406cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:05.663 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.651+0000 7f8f9b200640 1 -- 172.21.15.145:0/1673707061 shutdown_connections 2024-05-09T04:34:05.663 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.651+0000 7f8f9b200640 1 -- 172.21.15.145:0/1673707061 wait complete. 2024-05-09T04:34:05.663 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.653+0000 7f8f9b200640 1 Processor -- start 2024-05-09T04:34:05.663 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.653+0000 7f8f9b200640 1 -- start start 2024-05-09T04:34:05.663 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.654+0000 7f8f9b200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f78004d40 0x7f8f941f2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:05.663 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.654+0000 7f8f9b200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8f940ae880 0x7f8f940aed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:05.663 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.654+0000 7f8f9b200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8f94140b60 0x7f8f94140fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:05.663 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.654+0000 7f8f9b200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9415d8c0 con 0x7f8f940ae880 2024-05-09T04:34:05.663 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.654+0000 7f8f9b200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9406f470 con 0x7f8f78004d40 2024-05-09T04:34:05.663 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.654+0000 7f8f9b200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8f9415da40 con 0x7f8f94140b60 2024-05-09T04:34:05.663 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.655+0000 7f8f8b400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8f940ae880 0x7f8f940aed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:05.663 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.655+0000 7f8f8b400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8f940ae880 0x7f8f940aed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:37904/0 (socket says 172.21.15.145:37904) 2024-05-09T04:34:05.663 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.655+0000 7f8f8b400640 1 -- 172.21.15.145:0/1713197766 learned_addr learned my addr 172.21.15.145:0/1713197766 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:05.663 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.655+0000 7f8f93e00640 1 --2- 172.21.15.145:0/1713197766 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f78004d40 0x7f8f941f2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:05.664 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.655+0000 7f8f99200640 1 --2- 172.21.15.145:0/1713197766 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8f94140b60 0x7f8f94140fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:05.664 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.655+0000 7f8f8b400640 1 -- 172.21.15.145:0/1713197766 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8f94140b60 msgr2=0x7f8f94140fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:05.664 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.655+0000 7f8f8b400640 1 --2- 172.21.15.145:0/1713197766 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8f94140b60 0x7f8f94140fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:05.664 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.655+0000 7f8f8b400640 1 -- 172.21.15.145:0/1713197766 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f78004d40 msgr2=0x7f8f941f2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:05.664 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.655+0000 7f8f8b400640 1 --2- 172.21.15.145:0/1713197766 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f78004d40 0x7f8f941f2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:05.664 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.655+0000 7f8f8b400640 1 -- 172.21.15.145:0/1713197766 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f941427c0 con 0x7f8f940ae880 2024-05-09T04:34:05.664 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.655+0000 7f8f8b400640 1 --2- 172.21.15.145:0/1713197766 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8f940ae880 0x7f8f940aed00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c009d00 tx=0x7f8f7c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:05.664 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.655+0000 7f8f90c00640 1 -- 172.21.15.145:0/1713197766 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c003990 con 0x7f8f940ae880 2024-05-09T04:34:05.664 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.656+0000 7f8f90c00640 1 -- 172.21.15.145:0/1713197766 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f7c019070 con 0x7f8f940ae880 2024-05-09T04:34:05.664 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.656+0000 7f8f90c00640 1 -- 172.21.15.145:0/1713197766 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c003d70 con 0x7f8f940ae880 2024-05-09T04:34:05.664 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.656+0000 7f8f9b200640 1 -- 172.21.15.145:0/1713197766 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f940af3f0 con 0x7f8f940ae880 2024-05-09T04:34:05.664 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.656+0000 7f8f9b200640 1 -- 172.21.15.145:0/1713197766 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f9413e3c0 con 0x7f8f940ae880 2024-05-09T04:34:05.664 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.658+0000 7f8f90c00640 1 -- 172.21.15.145:0/1713197766 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8f7c005d10 con 0x7f8f940ae880 2024-05-09T04:34:05.664 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.659+0000 7f8f90c00640 1 --2- 172.21.15.145:0/1713197766 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f8f70076300 0x7f8f700787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:05.664 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.659+0000 7f8f9dffb740 1 Processor -- start 2024-05-09T04:34:05.665 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.659+0000 7f8f90c00640 1 -- 172.21.15.145:0/1713197766 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8f7c097de0 con 0x7f8f940ae880 2024-05-09T04:34:05.665 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.659+0000 7f8f9dffb740 1 -- start start 2024-05-09T04:34:05.665 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.659+0000 7f8f93e00640 1 --2- 172.21.15.145:0/1713197766 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f8f70076300 0x7f8f700787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:05.665 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.660+0000 7f8f93e00640 1 --2- 172.21.15.145:0/1713197766 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f8f70076300 0x7f8f700787c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8f9415cfb0 tx=0x7f8f84009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:05.665 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.660+0000 7f8f9dffb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55891cf24840 0x55891cf24c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:05.665 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.660+0000 7f8f9dffb740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55891cf1f6d0 0x55891cf23440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:05.665 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.660+0000 7f8f9dffb740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55891cf23980 0x55891cf23dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:05.665 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.660+0000 7f8f9dffb740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55891ce88790 con 0x55891cf1f6d0 2024-05-09T04:34:05.665 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.660+0000 7f8f9dffb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55891ce2df20 con 0x55891cf24840 2024-05-09T04:34:05.665 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.660+0000 7f8f9dffb740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x55891ce84750 con 0x55891cf23980 2024-05-09T04:34:05.665 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.661+0000 7f8f8b400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55891cf1f6d0 0x55891cf23440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:05.665 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.661+0000 7f8f8b400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55891cf1f6d0 0x55891cf23440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:37910/0 (socket says 172.21.15.145:37910) 2024-05-09T04:34:05.665 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.661+0000 7f8f8b400640 1 -- 172.21.15.145:0/2482685393 learned_addr learned my addr 172.21.15.145:0/2482685393 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:05.665 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.661+0000 7f8f99200640 1 --2- 172.21.15.145:0/2482685393 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55891cf23980 0x55891cf23dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:05.665 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.661+0000 7f8f93e00640 1 --2- 172.21.15.145:0/2482685393 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55891cf24840 0x55891cf24c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:05.666 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.661+0000 7f8f8b400640 1 -- 172.21.15.145:0/2482685393 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55891cf23980 msgr2=0x55891cf23dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:05.666 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.661+0000 7f8f8b400640 1 --2- 172.21.15.145:0/2482685393 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55891cf23980 0x55891cf23dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:05.666 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.661+0000 7f8f8b400640 1 -- 172.21.15.145:0/2482685393 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55891cf24840 msgr2=0x55891cf24c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:05.666 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.661+0000 7f8f8b400640 1 --2- 172.21.15.145:0/2482685393 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55891cf24840 0x55891cf24c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:05.666 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.661+0000 7f8f8b400640 1 -- 172.21.15.145:0/2482685393 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55891cb9a2d0 con 0x55891cf1f6d0 2024-05-09T04:34:05.666 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.661+0000 7f8f99200640 1 --2- 172.21.15.145:0/2482685393 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55891cf23980 0x55891cf23dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:34:05.666 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.661+0000 7f8f8b400640 1 --2- 172.21.15.145:0/2482685393 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55891cf1f6d0 0x55891cf23440 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c0093e0 tx=0x7f8f7c060710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74cdab5e8226a4df server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:05.666 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.661+0000 7f8f88c00640 1 -- 172.21.15.145:0/2482685393 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c060fa0 con 0x55891cf1f6d0 2024-05-09T04:34:05.666 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.661+0000 7f8f88c00640 1 -- 172.21.15.145:0/2482685393 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f7c065350 con 0x55891cf1f6d0 2024-05-09T04:34:05.666 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.661+0000 7f8f88c00640 1 -- 172.21.15.145:0/2482685393 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c060440 con 0x55891cf1f6d0 2024-05-09T04:34:05.666 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.662+0000 7f8f9dffb740 1 -- 172.21.15.145:0/2482685393 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55891cf1f6d0 msgr2=0x55891cf23440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:05.666 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.662+0000 7f8f9dffb740 1 --2- 172.21.15.145:0/2482685393 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55891cf1f6d0 0x55891cf23440 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c0093e0 tx=0x7f8f7c060710 comp rx=0 tx=0).stop 2024-05-09T04:34:05.666 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.662+0000 7f8f9dffb740 1 -- 172.21.15.145:0/2482685393 shutdown_connections 2024-05-09T04:34:05.666 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.662+0000 7f8f9dffb740 1 --2- 172.21.15.145:0/2482685393 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55891cf23980 0x55891cf23dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:05.666 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.662+0000 7f8f9dffb740 1 --2- 172.21.15.145:0/2482685393 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55891cf1f6d0 0x55891cf23440 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:05.667 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.662+0000 7f8f9dffb740 1 --2- 172.21.15.145:0/2482685393 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55891cf24840 0x55891cf24c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:05.667 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.662+0000 7f8f9dffb740 1 -- 172.21.15.145:0/2482685393 >> 172.21.15.145:0/2482685393 conn(0x55891cf21940 msgr2=0x55891cf6a400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:05.667 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.662+0000 7f8f9dffb740 1 -- 172.21.15.145:0/2482685393 shutdown_connections 2024-05-09T04:34:05.667 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.662+0000 7f8f9dffb740 1 -- 172.21.15.145:0/2482685393 wait complete. 2024-05-09T04:34:05.667 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.663+0000 7f8f9dffb740 1 -- start start 2024-05-09T04:34:05.667 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.663+0000 7f8f9dffb740 1 Processor -- start 2024-05-09T04:34:05.671 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.664+0000 7f8f9dffb740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55891cf23980 0x55891cf42190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:05.671 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.664+0000 7f8f9dffb740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55891cf24840 0x55891cf71560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:05.671 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.664+0000 7f8f9dffb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55891cf71aa0 0x55891cf6d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:05.671 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.664+0000 7f8f9dffb740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55891cf503d0 con 0x55891cf24840 2024-05-09T04:34:05.671 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.664+0000 7f8f9dffb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55891ce419c0 con 0x55891cf71aa0 2024-05-09T04:34:05.671 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.664+0000 7f8f9dffb740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x55891cf4f850 con 0x55891cf23980 2024-05-09T04:34:05.671 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.664+0000 7f8f9dffb740 10 client.0 _ll_register_callbacks cb 0x55891ce0e120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:34:05.671 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.664+0000 7f8f8b400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55891cf24840 0x55891cf71560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:05.671 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.664+0000 7f8f8b400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55891cf24840 0x55891cf71560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:37912/0 (socket says 172.21.15.145:37912) 2024-05-09T04:34:05.671 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.664+0000 7f8f8b400640 1 -- 172.21.15.145:0/1568982637 learned_addr learned my addr 172.21.15.145:0/1568982637 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:05.671 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.664+0000 7f8f93e00640 1 --2- 172.21.15.145:0/1568982637 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55891cf23980 0x55891cf42190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:05.671 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.664+0000 7f8f8b400640 1 -- 172.21.15.145:0/1568982637 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55891cf23980 msgr2=0x55891cf42190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:05.672 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.665+0000 7f8f8b400640 1 --2- 172.21.15.145:0/1568982637 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55891cf23980 0x55891cf42190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:05.672 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.665+0000 7f8f8b400640 1 -- 172.21.15.145:0/1568982637 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55891cf71aa0 msgr2=0x55891cf6d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:05.672 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.665+0000 7f8f99200640 1 --2- 172.21.15.145:0/1568982637 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55891cf71aa0 0x55891cf6d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:05.672 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.665+0000 7f8f8b400640 1 --2- 172.21.15.145:0/1568982637 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55891cf71aa0 0x55891cf6d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:05.672 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.665+0000 7f8f8b400640 1 -- 172.21.15.145:0/1568982637 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55891cc646b0 con 0x55891cf24840 2024-05-09T04:34:05.672 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.665+0000 7f8f99200640 1 --2- 172.21.15.145:0/1568982637 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55891cf71aa0 0x55891cf6d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:34:05.672 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.665+0000 7f8f8b400640 1 --2- 172.21.15.145:0/1568982637 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55891cf24840 0x55891cf71560 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c060c50 tx=0x7f8f7c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e77bc8914194a57 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:05.672 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.665+0000 7f8f81600640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-05-09T04:34:05.672 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.665+0000 7f8f81600640 1 -- 172.21.15.145:0/1568982637 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c07e020 con 0x55891cf24840 2024-05-09T04:34:05.672 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.665+0000 7f8f81600640 1 -- 172.21.15.145:0/1568982637 <== mon.0 v2:172.21.15.145:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8f7c0823a0 con 0x55891cf24840 2024-05-09T04:34:05.672 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.665+0000 7f8f9dffb740 1 -- 172.21.15.145:0/1568982637 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f7c065350 con 0x55891cf24840 2024-05-09T04:34:05.672 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.665+0000 7f8f81600640 1 -- 172.21.15.145:0/1568982637 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f8f7c069d70 con 0x55891cf24840 2024-05-09T04:34:05.672 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.665+0000 7f8f9dffb740 10 client.4560 fetch_fsmap learned FSMap version 8 2024-05-09T04:34:05.673 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.665+0000 7f8f9dffb740 1 -- 172.21.15.145:0/1568982637 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55891cf45600 con 0x55891cf24840 2024-05-09T04:34:05.673 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.665+0000 7f8f81600640 1 -- 172.21.15.145:0/1568982637 <== mon.0 v2:172.21.15.145:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f7c065350 con 0x55891cf24840 2024-05-09T04:34:05.673 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.666+0000 7f8f81600640 1 -- 172.21.15.145:0/1568982637 <== mon.0 v2:172.21.15.145:3300/0 5 ==== fsmap(e 8) ==== 3494+0+0 (secure 0 0 0) 0x7f8f7c05c910 con 0x55891cf24840 2024-05-09T04:34:05.673 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.666+0000 7f8f9dffb740 10 client.4560 fetch_fsmap finished waiting for FSMap version 8 2024-05-09T04:34:05.673 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.666+0000 7f8f9dffb740 10 client.4560 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-09T04:34:05.673 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.666+0000 7f8f9dffb740 20 client.4560 populate_metadata read hostname 'smithi145' 2024-05-09T04:34:05.673 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.666+0000 7f8f9dffb740 1 --2- 172.21.15.145:0/1568982637 >> [v2:172.21.15.192:6834/1113351558,v1:172.21.15.192:6835/1113351558] conn(0x55891cf44580 0x55891cf44960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:05.673 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.666+0000 7f8f9dffb740 4 client.4560 mds_command: new command op to 4270 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:34:05.673 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.666+0000 7f8f9dffb740 1 -- 172.21.15.145:0/1568982637 --> [v2:172.21.15.192:6834/1113351558,v1:172.21.15.192:6835/1113351558] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55891ce963a0 con 0x55891cf44580 2024-05-09T04:34:05.673 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.666+0000 7f8f93e00640 1 --2- 172.21.15.145:0/1568982637 >> [v2:172.21.15.192:6834/1113351558,v1:172.21.15.192:6835/1113351558] conn(0x55891cf44580 0x55891cf44960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:05.673 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.667+0000 7f8f93e00640 1 --2- 172.21.15.145:0/1568982637 >> [v2:172.21.15.192:6834/1113351558,v1:172.21.15.192:6835/1113351558] conn(0x55891cf44580 0x55891cf44960 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=382fec3743bb6b7a server_cookie=59055ab978fa8808 in_seq=0 out_seq=0 2024-05-09T04:34:05.673 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.667+0000 7f8f81600640 10 client.4560 ms_handle_connect on v2:172.21.15.192:6834/1113351558 2024-05-09T04:34:05.673 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.669+0000 7f8f81600640 1 -- 172.21.15.145:0/1568982637 <== mds.0 v2:172.21.15.192:6834/1113351558 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55891ce963a0 con 0x55891cf44580 2024-05-09T04:34:05.673 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.669+0000 7f8f81600640 10 client.4560 handle_command_reply: tid=0 2024-05-09T04:34:05.673 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.670+0000 7f8f9dffb740 1 client.4560 shutdown 2024-05-09T04:34:05.674 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.670+0000 7f8f9dffb740 1 -- 172.21.15.145:0/1568982637 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55891cf24840 msgr2=0x55891cf71560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:05.674 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.670+0000 7f8f9dffb740 1 --2- 172.21.15.145:0/1568982637 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55891cf24840 0x55891cf71560 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c060c50 tx=0x7f8f7c009750 comp rx=0 tx=0).stop 2024-05-09T04:34:05.674 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.670+0000 7f8f9dffb740 1 -- 172.21.15.145:0/1568982637 shutdown_connections 2024-05-09T04:34:05.674 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.670+0000 7f8f9dffb740 1 --2- 172.21.15.145:0/1568982637 >> [v2:172.21.15.192:6834/1113351558,v1:172.21.15.192:6835/1113351558] conn(0x55891cf44580 0x55891cf44960 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:05.674 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.670+0000 7f8f9dffb740 1 --2- 172.21.15.145:0/1568982637 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55891cf71aa0 0x55891cf6d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:05.674 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.670+0000 7f8f9dffb740 1 --2- 172.21.15.145:0/1568982637 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55891cf24840 0x55891cf71560 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:05.674 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.670+0000 7f8f81600640 0 client.4560 ms_handle_reset on v2:172.21.15.192:6834/1113351558 2024-05-09T04:34:05.674 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.670+0000 7f8f9dffb740 1 --2- 172.21.15.145:0/1568982637 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55891cf23980 0x55891cf42190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:05.674 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.670+0000 7f8f9dffb740 1 -- 172.21.15.145:0/1568982637 >> 172.21.15.145:0/1568982637 conn(0x55891cf1f6d0 msgr2=0x55891cf1edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:05.674 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.670+0000 7f8f9dffb740 1 -- 172.21.15.145:0/1568982637 shutdown_connections 2024-05-09T04:34:05.674 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.670+0000 7f8f9dffb740 1 -- 172.21.15.145:0/1568982637 wait complete. 2024-05-09T04:34:05.674 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.670+0000 7f8f9dffb740 20 client.4560 trim_cache size 0 max 16384 2024-05-09T04:34:05.689 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.690+0000 7f8f9dffb740 1 Processor -- start 2024-05-09T04:34:05.689 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.690+0000 7f8f9dffb740 1 -- start start 2024-05-09T04:34:05.690 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.691+0000 7f8f9dffb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55891cf23980 0x55891cf4fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:05.690 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.691+0000 7f8f9dffb740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55891cf24840 0x55891cfe2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:05.690 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.691+0000 7f8f9dffb740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55891cf71aa0 0x55891cfe3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:05.690 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.691+0000 7f8f9dffb740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8f94200f60 con 0x55891cf71aa0 2024-05-09T04:34:05.691 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.691+0000 7f8f9dffb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9413fb10 con 0x55891cf23980 2024-05-09T04:34:05.691 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.691+0000 7f8f9dffb740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8f94200940 con 0x55891cf24840 2024-05-09T04:34:05.691 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.692+0000 7f8f99200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55891cf71aa0 0x55891cfe3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:05.691 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.692+0000 7f8f99200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55891cf71aa0 0x55891cfe3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:37926/0 (socket says 172.21.15.145:37926) 2024-05-09T04:34:05.691 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.692+0000 7f8f99200640 1 -- 172.21.15.145:0/743529219 learned_addr learned my addr 172.21.15.145:0/743529219 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:05.691 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.692+0000 7f8f8b400640 1 --2- 172.21.15.145:0/743529219 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55891cf24840 0x55891cfe2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:05.691 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.692+0000 7f8f93e00640 1 --2- 172.21.15.145:0/743529219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55891cf23980 0x55891cf4fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:05.693 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.692+0000 7f8f99200640 1 -- 172.21.15.145:0/743529219 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55891cf24840 msgr2=0x55891cfe2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:05.693 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.692+0000 7f8f99200640 1 --2- 172.21.15.145:0/743529219 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55891cf24840 0x55891cfe2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:05.693 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.692+0000 7f8f99200640 1 -- 172.21.15.145:0/743529219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55891cf23980 msgr2=0x55891cf4fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:05.693 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.692+0000 7f8f99200640 1 --2- 172.21.15.145:0/743529219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55891cf23980 0x55891cf4fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:05.693 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.692+0000 7f8f99200640 1 -- 172.21.15.145:0/743529219 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55891cf71fc0 con 0x55891cf71aa0 2024-05-09T04:34:05.693 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.692+0000 7f8f99200640 1 --2- 172.21.15.145:0/743529219 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55891cf71aa0 0x55891cfe3140 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c006b00 tx=0x7f8f8c02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86ce33b8fc14f44c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:05.693 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.693+0000 7f8f83e00640 1 -- 172.21.15.145:0/743529219 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c042070 con 0x55891cf71aa0 2024-05-09T04:34:05.693 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.693+0000 7f8f83e00640 1 -- 172.21.15.145:0/743529219 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f8c004230 con 0x55891cf71aa0 2024-05-09T04:34:05.693 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.693+0000 7f8f83e00640 1 -- 172.21.15.145:0/743529219 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c036070 con 0x55891cf71aa0 2024-05-09T04:34:05.693 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.693+0000 7f8f9dffb740 1 -- 172.21.15.145:0/743529219 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55891cf71aa0 msgr2=0x55891cfe3140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:05.693 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.693+0000 7f8f9dffb740 1 --2- 172.21.15.145:0/743529219 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55891cf71aa0 0x55891cfe3140 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c006b00 tx=0x7f8f8c02e000 comp rx=0 tx=0).stop 2024-05-09T04:34:05.693 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.693+0000 7f8f9dffb740 1 -- 172.21.15.145:0/743529219 shutdown_connections 2024-05-09T04:34:05.693 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.693+0000 7f8f9dffb740 1 --2- 172.21.15.145:0/743529219 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55891cf71aa0 0x55891cfe3140 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:05.693 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.693+0000 7f8f9dffb740 1 --2- 172.21.15.145:0/743529219 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55891cf24840 0x55891cfe2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:05.694 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.693+0000 7f8f9dffb740 1 --2- 172.21.15.145:0/743529219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55891cf23980 0x55891cf4fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:05.694 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.693+0000 7f8f9dffb740 1 -- 172.21.15.145:0/743529219 >> 172.21.15.145:0/743529219 conn(0x55891cf1f6d0 msgr2=0x55891cf202d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:05.694 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.693+0000 7f8f9dffb740 1 -- 172.21.15.145:0/743529219 shutdown_connections 2024-05-09T04:34:05.694 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.693+0000 7f8f9dffb740 1 -- 172.21.15.145:0/743529219 wait complete. 2024-05-09T04:34:05.694 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.694+0000 7f8f9dffb740 1 -- start start 2024-05-09T04:34:05.694 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.694+0000 7f8f9dffb740 1 Processor -- start 2024-05-09T04:34:05.696 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.695+0000 7f8f9dffb740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55891cf23980 0x55891cfe7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:05.696 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.695+0000 7f8f9dffb740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55891cf24840 0x55891cf24f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:05.697 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.695+0000 7f8f9dffb740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55891cf71aa0 0x55891cf254d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:05.697 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.695+0000 7f8f9dffb740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55891cf503d0 con 0x55891cf23980 2024-05-09T04:34:05.697 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.695+0000 7f8f9dffb740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55891ce419c0 con 0x55891cf24840 2024-05-09T04:34:05.697 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.695+0000 7f8f9dffb740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x55891cf42e80 con 0x55891cf71aa0 2024-05-09T04:34:05.697 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.695+0000 7f8f9dffb740 10 client.0 _ll_register_callbacks cb 0x7f8f7c060e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:34:05.697 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.695+0000 7f8f93e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55891cf23980 0x55891cfe7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:05.697 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.695+0000 7f8f93e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55891cf23980 0x55891cfe7aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:37934/0 (socket says 172.21.15.145:37934) 2024-05-09T04:34:05.697 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.695+0000 7f8f93e00640 1 -- 172.21.15.145:0/3377359940 learned_addr learned my addr 172.21.15.145:0/3377359940 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:05.697 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.696+0000 7f8f8b400640 1 --2- 172.21.15.145:0/3377359940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55891cf24840 0x55891cf24f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:05.697 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.696+0000 7f8f99200640 1 --2- 172.21.15.145:0/3377359940 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55891cf71aa0 0x55891cf254d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:05.697 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.696+0000 7f8f93e00640 1 -- 172.21.15.145:0/3377359940 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55891cf71aa0 msgr2=0x55891cf254d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:05.697 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.696+0000 7f8f93e00640 1 --2- 172.21.15.145:0/3377359940 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55891cf71aa0 0x55891cf254d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:05.697 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.696+0000 7f8f93e00640 1 -- 172.21.15.145:0/3377359940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55891cf24840 msgr2=0x55891cf24f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:05.698 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.696+0000 7f8f93e00640 1 --2- 172.21.15.145:0/3377359940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55891cf24840 0x55891cf24f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:05.698 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.696+0000 7f8f93e00640 1 -- 172.21.15.145:0/3377359940 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55891cf450d0 con 0x55891cf23980 2024-05-09T04:34:05.698 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.696+0000 7f8f8b400640 1 --2- 172.21.15.145:0/3377359940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55891cf24840 0x55891cf24f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:34:05.698 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.696+0000 7f8f93e00640 1 --2- 172.21.15.145:0/3377359940 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55891cf23980 0x55891cfe7aa0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8f8400fe60 tx=0x7f8f9406f080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b83927137f87104b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:05.698 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.696+0000 7f8f81600640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-05-09T04:34:05.698 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.696+0000 7f8f81600640 1 -- 172.21.15.145:0/3377359940 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f84025b30 con 0x55891cf23980 2024-05-09T04:34:05.698 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.696+0000 7f8f81600640 1 -- 172.21.15.145:0/3377359940 <== mon.0 v2:172.21.15.145:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8f84025cd0 con 0x55891cf23980 2024-05-09T04:34:05.698 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.696+0000 7f8f9dffb740 1 -- 172.21.15.145:0/3377359940 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f8c004230 con 0x55891cf23980 2024-05-09T04:34:05.698 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.697+0000 7f8f81600640 1 -- 172.21.15.145:0/3377359940 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f8f84023430 con 0x55891cf23980 2024-05-09T04:34:05.698 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.697+0000 7f8f9dffb740 10 client.4566 fetch_fsmap learned FSMap version 8 2024-05-09T04:34:05.698 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.697+0000 7f8f9dffb740 1 -- 172.21.15.145:0/3377359940 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55891cf6a480 con 0x55891cf23980 2024-05-09T04:34:05.698 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.697+0000 7f8f81600640 1 -- 172.21.15.145:0/3377359940 <== mon.0 v2:172.21.15.145:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f8c004230 con 0x55891cf23980 2024-05-09T04:34:05.698 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.697+0000 7f8f81600640 1 -- 172.21.15.145:0/3377359940 <== mon.0 v2:172.21.15.145:3300/0 5 ==== fsmap(e 8) ==== 3494+0+0 (secure 0 0 0) 0x7f8f84039080 con 0x55891cf23980 2024-05-09T04:34:05.698 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.697+0000 7f8f9dffb740 10 client.4566 fetch_fsmap finished waiting for FSMap version 8 2024-05-09T04:34:05.698 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.697+0000 7f8f9dffb740 10 client.4566 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-09T04:34:05.699 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.697+0000 7f8f9dffb740 20 client.4566 populate_metadata read hostname 'smithi145' 2024-05-09T04:34:05.699 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.697+0000 7f8f9dffb740 1 --2- 172.21.15.145:0/3377359940 >> [v2:172.21.15.192:6834/1113351558,v1:172.21.15.192:6835/1113351558] conn(0x55891cf1e560 0x55891cf1e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:05.699 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.697+0000 7f8f9dffb740 4 client.4566 mds_command: new command op to 4270 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T04:34:05.699 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.697+0000 7f8f9dffb740 1 -- 172.21.15.145:0/3377359940 --> [v2:172.21.15.192:6834/1113351558,v1:172.21.15.192:6835/1113351558] -- command(tid 0: {"prefix": "status"}) -- 0x55891cf20ee0 con 0x55891cf1e560 2024-05-09T04:34:05.699 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.698+0000 7f8f8b400640 1 --2- 172.21.15.145:0/3377359940 >> [v2:172.21.15.192:6834/1113351558,v1:172.21.15.192:6835/1113351558] conn(0x55891cf1e560 0x55891cf1e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:05.700 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.698+0000 7f8f8b400640 1 --2- 172.21.15.145:0/3377359940 >> [v2:172.21.15.192:6834/1113351558,v1:172.21.15.192:6835/1113351558] conn(0x55891cf1e560 0x55891cf1e940 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66b5c219b2ded978 server_cookie=bdcc24d4d3de7472 in_seq=0 out_seq=0 2024-05-09T04:34:05.700 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.698+0000 7f8f81600640 10 client.4566 ms_handle_connect on v2:172.21.15.192:6834/1113351558 2024-05-09T04:34:05.700 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.699+0000 7f8f81600640 1 -- 172.21.15.145:0/3377359940 <== mds.0 v2:172.21.15.192:6834/1113351558 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55891cf20ee0 con 0x55891cf1e560 2024-05-09T04:34:05.700 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.699+0000 7f8f81600640 10 client.4566 handle_command_reply: tid=0 2024-05-09T04:34:05.700 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.699+0000 7f8f9dffb740 1 client.4566 shutdown 2024-05-09T04:34:05.700 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.699+0000 7f8f9dffb740 1 -- 172.21.15.145:0/3377359940 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55891cf23980 msgr2=0x55891cfe7aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:05.701 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.699+0000 7f8f9dffb740 1 --2- 172.21.15.145:0/3377359940 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55891cf23980 0x55891cfe7aa0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8f8400fe60 tx=0x7f8f9406f080 comp rx=0 tx=0).stop 2024-05-09T04:34:05.701 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.699+0000 7f8f9dffb740 1 -- 172.21.15.145:0/3377359940 shutdown_connections 2024-05-09T04:34:05.701 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.699+0000 7f8f9dffb740 1 --2- 172.21.15.145:0/3377359940 >> [v2:172.21.15.192:6834/1113351558,v1:172.21.15.192:6835/1113351558] conn(0x55891cf1e560 0x55891cf1e940 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:05.701 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.699+0000 7f8f9dffb740 1 --2- 172.21.15.145:0/3377359940 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55891cf71aa0 0x55891cf254d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:05.701 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.699+0000 7f8f9dffb740 1 --2- 172.21.15.145:0/3377359940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55891cf24840 0x55891cf24f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:05.701 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.699+0000 7f8f9dffb740 1 --2- 172.21.15.145:0/3377359940 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55891cf23980 0x55891cfe7aa0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:05.701 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.699+0000 7f8f9dffb740 1 -- 172.21.15.145:0/3377359940 >> 172.21.15.145:0/3377359940 conn(0x55891cf1f6d0 msgr2=0x55891cf202d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:05.701 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.699+0000 7f8f81600640 0 client.4566 ms_handle_reset on v2:172.21.15.192:6834/1113351558 2024-05-09T04:34:05.701 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.699+0000 7f8f9dffb740 1 -- 172.21.15.145:0/3377359940 shutdown_connections 2024-05-09T04:34:05.701 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.699+0000 7f8f9dffb740 1 -- 172.21.15.145:0/3377359940 wait complete. 2024-05-09T04:34:05.701 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.700+0000 7f8f9dffb740 20 client.4566 trim_cache size 0 max 16384 2024-05-09T04:34:05.701 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-05-09T04:34:05.702 INFO:teuthology.orchestra.run.smithi145.stdout: "cluster_fsid": "957ee6c1-073b-44cf-9b93-9491b3fdfc1d", 2024-05-09T04:34:05.702 INFO:teuthology.orchestra.run.smithi145.stdout: "whoami": 0, 2024-05-09T04:34:05.702 INFO:teuthology.orchestra.run.smithi145.stdout: "id": 4270, 2024-05-09T04:34:05.702 INFO:teuthology.orchestra.run.smithi145.stdout: "want_state": "up:active", 2024-05-09T04:34:05.702 INFO:teuthology.orchestra.run.smithi145.stdout: "state": "up:active", 2024-05-09T04:34:05.702 INFO:teuthology.orchestra.run.smithi145.stdout: "fs_name": "cephfs", 2024-05-09T04:34:05.702 INFO:teuthology.orchestra.run.smithi145.stdout: "rank_uptime": 13.534289221, 2024-05-09T04:34:05.702 INFO:teuthology.orchestra.run.smithi145.stdout: "mdsmap_epoch": 8, 2024-05-09T04:34:05.702 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_epoch": 20, 2024-05-09T04:34:05.702 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_epoch_barrier": 20, 2024-05-09T04:34:05.702 INFO:teuthology.orchestra.run.smithi145.stdout: "uptime": 19.741566924000001 2024-05-09T04:34:05.702 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-05-09T04:34:05.703 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.700+0000 7f8f9b200640 1 -- 172.21.15.145:0/1713197766 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f8f70076300 msgr2=0x7f8f700787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:05.703 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.700+0000 7f8f9b200640 1 --2- 172.21.15.145:0/1713197766 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f8f70076300 0x7f8f700787c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8f9415cfb0 tx=0x7f8f84009040 comp rx=0 tx=0).stop 2024-05-09T04:34:05.703 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.700+0000 7f8f9b200640 1 -- 172.21.15.145:0/1713197766 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8f940ae880 msgr2=0x7f8f940aed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:05.703 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.700+0000 7f8f9b200640 1 --2- 172.21.15.145:0/1713197766 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8f940ae880 0x7f8f940aed00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c009d00 tx=0x7f8f7c00c490 comp rx=0 tx=0).stop 2024-05-09T04:34:05.703 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.700+0000 7f8f9b200640 1 -- 172.21.15.145:0/1713197766 shutdown_connections 2024-05-09T04:34:05.703 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.700+0000 7f8f9b200640 1 --2- 172.21.15.145:0/1713197766 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f8f70076300 0x7f8f700787c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:05.703 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.700+0000 7f8f9b200640 1 --2- 172.21.15.145:0/1713197766 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8f94140b60 0x7f8f94140fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:05.704 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.700+0000 7f8f9b200640 1 --2- 172.21.15.145:0/1713197766 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8f940ae880 0x7f8f940aed00 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:05.704 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.700+0000 7f8f9b200640 1 --2- 172.21.15.145:0/1713197766 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f78004d40 0x7f8f941f2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:05.704 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.700+0000 7f8f9b200640 1 -- 172.21.15.145:0/1713197766 >> 172.21.15.145:0/1713197766 conn(0x7f8f9406c8d0 msgr2=0x7f8f940b1730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:05.704 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.700+0000 7f8f9b200640 1 -- 172.21.15.145:0/1713197766 shutdown_connections 2024-05-09T04:34:05.704 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:05.700+0000 7f8f9b200640 1 -- 172.21.15.145:0/1713197766 wait complete. 2024-05-09T04:34:05.715 INFO:teuthology.run_tasks:Running task sequential... 2024-05-09T04:34:05.726 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-09T04:34:05.726 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-09T04:34:05.726 INFO:teuthology.task.print:Enabling mgr modules 2024-05-09T04:34:05.727 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-05-09T04:34:05.795 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-05-09T04:34:05.795 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-05-09T04:34:05.795 INFO:tasks.ceph_fuse:client.0 config is {} 2024-05-09T04:34:05.795 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-09T04:34:05.795 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-09T04:34:05.795 INFO:tasks.ceph_fuse:client.2 config is {} 2024-05-09T04:34:05.795 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-09T04:34:05.795 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-05-09T04:34:05.796 INFO:tasks.ceph_fuse:client.1 config is {} 2024-05-09T04:34:05.796 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-09T04:34:05.796 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-09T04:34:05.796 INFO:tasks.ceph_fuse:client.3 config is {} 2024-05-09T04:34:05.796 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-09T04:34:05.796 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-05-09T04:34:05.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:05.796 DEBUG:teuthology.orchestra.run.smithi192:> ip netns list 2024-05-09T04:34:05.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:05.854 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link delete ceph-brx 2024-05-09T04:34:05.940 INFO:teuthology.orchestra.run.smithi192.stderr:Cannot find device "ceph-brx" 2024-05-09T04:34:05.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T04:34:05.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:05.945 DEBUG:teuthology.orchestra.run.smithi145:> ip netns list 2024-05-09T04:34:06.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:06.003 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip link delete ceph-brx 2024-05-09T04:34:06.093 INFO:teuthology.orchestra.run.smithi145.stderr:Cannot find device "ceph-brx" 2024-05-09T04:34:06.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T04:34:06.097 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-05-09T04:34:06.097 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-09T04:34:06.298 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.299+0000 7f61b1600640 1 Processor -- start 2024-05-09T04:34:06.298 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.299+0000 7f61b1600640 1 -- start start 2024-05-09T04:34:06.301 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.300+0000 7f61b1600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f61ac0ae710 0x7f61ac15deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:06.301 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.300+0000 7f61b1600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61ac0aecd0 con 0x7f61ac1572c0 2024-05-09T04:34:06.302 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.300+0000 7f61b1600640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61ac0aefb0 con 0x7f61ac0cff30 2024-05-09T04:34:06.302 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.300+0000 7f61b1600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f61ac0b6290 con 0x7f61ac0ae710 2024-05-09T04:34:06.302 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.301+0000 7f61abe00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f61ac1572c0 0x7f61ac161220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:37078/0 (socket says 172.21.15.145:37078) 2024-05-09T04:34:06.302 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.301+0000 7f61abe00640 1 -- 172.21.15.145:0/1873937287 learned_addr learned my addr 172.21.15.145:0/1873937287 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:34:06.302 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.301+0000 7f61aaa00640 1 -- 172.21.15.145:0/1873937287 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1376470174 0 0) 0x7f61ac0aecd0 con 0x7f61ac1572c0 2024-05-09T04:34:06.302 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.301+0000 7f61aaa00640 1 -- 172.21.15.145:0/1873937287 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6190003520 con 0x7f61ac1572c0 2024-05-09T04:34:06.302 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.301+0000 7f61ab400640 1 --2- 172.21.15.145:0/1873937287 >> v2:172.21.15.192:3300/0 conn(0x7f61ac0ae710 0x7f61ac15deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:06.302 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.301+0000 7f61aaa00640 1 -- 172.21.15.145:0/1873937287 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1364729359 0 0) 0x7f6190003520 con 0x7f61ac1572c0 2024-05-09T04:34:06.302 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.301+0000 7f61aaa00640 1 -- 172.21.15.145:0/1873937287 >> v2:172.21.15.192:3300/0 conn(0x7f61ac0ae710 msgr2=0x7f61ac15deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:06.302 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.301+0000 7f61aaa00640 1 --2- 172.21.15.145:0/1873937287 >> v2:172.21.15.192:3300/0 conn(0x7f61ac0ae710 0x7f61ac15deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:06.302 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.301+0000 7f61aaa00640 1 -- 172.21.15.145:0/1873937287 >> v1:172.21.15.192:6789/0 conn(0x7f61ac0cff30 legacy=0x7f61ac0d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:06.302 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.301+0000 7f61aaa00640 1 -- 172.21.15.145:0/1873937287 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61ac0af290 con 0x7f61ac1572c0 2024-05-09T04:34:06.303 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.302+0000 7f61aaa00640 1 -- 172.21.15.145:0/1873937287 <== mon.0 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3474413261 0 0) 0x7f619c0016f0 con 0x7f61ac1572c0 2024-05-09T04:34:06.303 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.302+0000 7f61aaa00640 1 -- 172.21.15.145:0/1873937287 >> v1:172.21.15.145:6789/0 conn(0x7f61ac1572c0 legacy=0x7f61ac161220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:06.303 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.302+0000 7f61aaa00640 1 --2- 172.21.15.145:0/1873937287 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6190003a00 0x7f6190003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:06.303 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.302+0000 7f61aaa00640 1 --2- 172.21.15.145:0/1873937287 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6190004800 0x7f6190004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:06.303 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.302+0000 7f61aaa00640 1 --2- 172.21.15.145:0/1873937287 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6190005210 0x7f619000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:06.303 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.302+0000 7f61aaa00640 1 -- 172.21.15.145:0/1873937287 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f61ac0b6290 con 0x7f6190004800 2024-05-09T04:34:06.303 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.302+0000 7f61aaa00640 1 -- 172.21.15.145:0/1873937287 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f619000c140 con 0x7f6190005210 2024-05-09T04:34:06.303 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.302+0000 7f61aaa00640 1 -- 172.21.15.145:0/1873937287 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f619000c340 con 0x7f6190003a00 2024-05-09T04:34:06.303 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.302+0000 7f61ab400640 1 --2- 172.21.15.145:0/1873937287 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6190004800 0x7f6190004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:06.303 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.302+0000 7f61b0c00640 1 --2- 172.21.15.145:0/1873937287 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6190005210 0x7f619000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:06.303 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.302+0000 7f61abe00640 1 --2- 172.21.15.145:0/1873937287 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6190003a00 0x7f6190003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:06.304 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.302+0000 7f61ab400640 1 -- 172.21.15.145:0/1873937287 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6190003a00 msgr2=0x7f6190003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:06.304 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.302+0000 7f61ab400640 1 --2- 172.21.15.145:0/1873937287 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6190003a00 0x7f6190003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:06.304 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.302+0000 7f61ab400640 1 -- 172.21.15.145:0/1873937287 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6190005210 msgr2=0x7f619000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:06.304 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.302+0000 7f61ab400640 1 --2- 172.21.15.145:0/1873937287 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6190005210 0x7f619000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:06.304 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.302+0000 7f61ab400640 1 -- 172.21.15.145:0/1873937287 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6190003520 con 0x7f6190004800 2024-05-09T04:34:06.304 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.302+0000 7f61b0c00640 1 -- 172.21.15.145:0/1873937287 reap_dead start 2024-05-09T04:34:06.305 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.303+0000 7f61ab400640 1 --2- 172.21.15.145:0/1873937287 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6190004800 0x7f6190004c80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f619400ba00 tx=0x7f6190005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99f1649986440a52 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:06.305 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.303+0000 7f61aaa00640 1 -- 172.21.15.145:0/1873937287 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6194030070 con 0x7f6190004800 2024-05-09T04:34:06.305 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.304+0000 7f61aaa00640 1 -- 172.21.15.145:0/1873937287 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f619402adb0 con 0x7f6190004800 2024-05-09T04:34:06.306 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.304+0000 7f61aaa00640 1 -- 172.21.15.145:0/1873937287 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f619400b600 con 0x7f6190004800 2024-05-09T04:34:06.306 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.304+0000 7f61b1600640 1 -- 172.21.15.145:0/1873937287 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6190004800 msgr2=0x7f6190004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:06.306 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.304+0000 7f61b1600640 1 --2- 172.21.15.145:0/1873937287 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6190004800 0x7f6190004c80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f619400ba00 tx=0x7f6190005830 comp rx=0 tx=0).stop 2024-05-09T04:34:06.306 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.304+0000 7f61b1600640 1 -- 172.21.15.145:0/1873937287 shutdown_connections 2024-05-09T04:34:06.306 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.304+0000 7f61b1600640 1 --2- 172.21.15.145:0/1873937287 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6190004800 0x7f6190004c80 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:06.306 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.304+0000 7f61b1600640 1 -- 172.21.15.145:0/1873937287 >> 172.21.15.145:0/1873937287 conn(0x7f61ac06c8a0 msgr2=0x7f61ac06ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:06.306 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.304+0000 7f61b1600640 1 -- 172.21.15.145:0/1873937287 shutdown_connections 2024-05-09T04:34:06.306 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.304+0000 7f61b1600640 1 -- 172.21.15.145:0/1873937287 wait complete. 2024-05-09T04:34:06.306 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.306+0000 7f61b1600640 1 Processor -- start 2024-05-09T04:34:06.306 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.306+0000 7f61b1600640 1 -- start start 2024-05-09T04:34:06.308 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.307+0000 7f61b1600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6190004800 0x7f61ac1420e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:06.308 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.307+0000 7f61b1600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f61ac09e2a0 0x7f61ac142720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:06.308 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.307+0000 7f61b1600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f61ac142c60 0x7f61ac0ce930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:06.308 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.307+0000 7f61b1600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f61ac1576a0 con 0x7f61ac142c60 2024-05-09T04:34:06.308 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.307+0000 7f61b1600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f61ac160d00 con 0x7f61ac09e2a0 2024-05-09T04:34:06.308 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.307+0000 7f61b1600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f61ac157820 con 0x7f6190004800 2024-05-09T04:34:06.308 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.307+0000 7f61b0c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f61ac142c60 0x7f61ac0ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:06.308 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.308+0000 7f61b0c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f61ac142c60 0x7f61ac0ce930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:37948/0 (socket says 172.21.15.145:37948) 2024-05-09T04:34:06.308 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.308+0000 7f61b0c00640 1 -- 172.21.15.145:0/2015299467 learned_addr learned my addr 172.21.15.145:0/2015299467 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:06.308 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.308+0000 7f61abe00640 1 --2- 172.21.15.145:0/2015299467 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6190004800 0x7f61ac1420e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:06.309 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.308+0000 7f61ab400640 1 --2- 172.21.15.145:0/2015299467 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f61ac09e2a0 0x7f61ac142720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:06.309 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.308+0000 7f61b0c00640 1 -- 172.21.15.145:0/2015299467 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6190004800 msgr2=0x7f61ac1420e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:06.309 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.308+0000 7f61b0c00640 1 --2- 172.21.15.145:0/2015299467 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6190004800 0x7f61ac1420e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:06.309 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.308+0000 7f61b0c00640 1 -- 172.21.15.145:0/2015299467 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f61ac09e2a0 msgr2=0x7f61ac142720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:06.309 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.308+0000 7f61b0c00640 1 --2- 172.21.15.145:0/2015299467 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f61ac09e2a0 0x7f61ac142720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:06.309 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.308+0000 7f61b0c00640 1 -- 172.21.15.145:0/2015299467 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61ac09e7d0 con 0x7f61ac142c60 2024-05-09T04:34:06.309 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.308+0000 7f61b0c00640 1 --2- 172.21.15.145:0/2015299467 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f61ac142c60 0x7f61ac0ce930 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f61a400e550 tx=0x7f61a400ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:06.309 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.308+0000 7f61a3e00640 1 -- 172.21.15.145:0/2015299467 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a4006d60 con 0x7f61ac142c60 2024-05-09T04:34:06.309 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.309+0000 7f61a3e00640 1 -- 172.21.15.145:0/2015299467 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61a4019070 con 0x7f61ac142c60 2024-05-09T04:34:06.309 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.309+0000 7f61a3e00640 1 -- 172.21.15.145:0/2015299467 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a40146c0 con 0x7f61ac142c60 2024-05-09T04:34:06.309 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.309+0000 7f61b1600640 1 -- 172.21.15.145:0/2015299467 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61ac0b3300 con 0x7f61ac142c60 2024-05-09T04:34:06.309 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.309+0000 7f61b1600640 1 -- 172.21.15.145:0/2015299467 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61ac0ce410 con 0x7f61ac142c60 2024-05-09T04:34:06.310 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.310+0000 7f61b1600640 1 -- 172.21.15.145:0/2015299467 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61ac06a310 con 0x7f61ac142c60 2024-05-09T04:34:06.310 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.311+0000 7f61a3e00640 1 -- 172.21.15.145:0/2015299467 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f61a4014860 con 0x7f61ac142c60 2024-05-09T04:34:06.311 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.312+0000 7f61a3e00640 1 --2- 172.21.15.145:0/2015299467 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f61880762d0 0x7f6188078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:06.311 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.312+0000 7f61a3e00640 1 -- 172.21.15.145:0/2015299467 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f61a4097580 con 0x7f61ac142c60 2024-05-09T04:34:06.311 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.312+0000 7f61abe00640 1 --2- 172.21.15.145:0/2015299467 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f61880762d0 0x7f6188078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:06.311 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.313+0000 7f61abe00640 1 --2- 172.21.15.145:0/2015299467 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f61880762d0 0x7f6188078790 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f61ac06be20 tx=0x7f619c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:06.315 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.316+0000 7f61a3e00640 1 -- 172.21.15.145:0/2015299467 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f61a4060ed0 con 0x7f61ac142c60 2024-05-09T04:34:06.571 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.572+0000 7f61b1600640 1 -- 172.21.15.145:0/2015299467 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f61ac0b45d0 con 0x7f61ac142c60 2024-05-09T04:34:06.572 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.573+0000 7f61a3e00640 1 -- 172.21.15.145:0/2015299467 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f61a40607d0 con 0x7f61ac142c60 2024-05-09T04:34:06.572 INFO:teuthology.orchestra.run.smithi145.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-09T04:34:06.575 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.576+0000 7f61b1600640 1 -- 172.21.15.145:0/2015299467 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f61880762d0 msgr2=0x7f6188078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:06.575 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.576+0000 7f61b1600640 1 --2- 172.21.15.145:0/2015299467 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f61880762d0 0x7f6188078790 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f61ac06be20 tx=0x7f619c009040 comp rx=0 tx=0).stop 2024-05-09T04:34:06.575 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.576+0000 7f61b1600640 1 -- 172.21.15.145:0/2015299467 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f61ac142c60 msgr2=0x7f61ac0ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:06.575 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.576+0000 7f61b1600640 1 --2- 172.21.15.145:0/2015299467 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f61ac142c60 0x7f61ac0ce930 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f61a400e550 tx=0x7f61a400ea10 comp rx=0 tx=0).stop 2024-05-09T04:34:06.575 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.576+0000 7f61b1600640 1 -- 172.21.15.145:0/2015299467 shutdown_connections 2024-05-09T04:34:06.575 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.576+0000 7f61b1600640 1 --2- 172.21.15.145:0/2015299467 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f61880762d0 0x7f6188078790 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:06.575 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.576+0000 7f61b1600640 1 --2- 172.21.15.145:0/2015299467 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f61ac142c60 0x7f61ac0ce930 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:06.575 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.576+0000 7f61b1600640 1 --2- 172.21.15.145:0/2015299467 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f61ac09e2a0 0x7f61ac142720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:06.575 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.576+0000 7f61b1600640 1 --2- 172.21.15.145:0/2015299467 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6190004800 0x7f61ac1420e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:06.576 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.576+0000 7f61b1600640 1 -- 172.21.15.145:0/2015299467 >> 172.21.15.145:0/2015299467 conn(0x7f61ac06c8a0 msgr2=0x7f61ac15c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:06.576 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.576+0000 7f61b1600640 1 -- 172.21.15.145:0/2015299467 shutdown_connections 2024-05-09T04:34:06.576 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06.576+0000 7f61b1600640 1 -- 172.21.15.145:0/2015299467 wait complete. 2024-05-09T04:34:06.594 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-09T04:34:06.594 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-09T04:34:06.595 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi145.front.sepia.ceph.com 2024-05-09T04:34:06.595 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-09T04:34:06.595 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-09T04:34:06.595 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-09T04:34:06.595 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-09T04:34:06.595 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-09T04:34:06.595 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-09T04:34:06.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:06.595 DEBUG:teuthology.orchestra.run.smithi145:> ip addr 2024-05-09T04:34:06.646 INFO:teuthology.orchestra.run.smithi145.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T04:34:06.646 INFO:teuthology.orchestra.run.smithi145.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T04:34:06.646 INFO:teuthology.orchestra.run.smithi145.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T04:34:06.646 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-05-09T04:34:06.646 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 ::1/128 scope host 2024-05-09T04:34:06.646 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-05-09T04:34:06.646 INFO:teuthology.orchestra.run.smithi145.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T04:34:06.646 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:d9:38:66 brd ff:ff:ff:ff:ff:ff 2024-05-09T04:34:06.646 INFO:teuthology.orchestra.run.smithi145.stdout: altname enp7s0f0 2024-05-09T04:34:06.646 INFO:teuthology.orchestra.run.smithi145.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T04:34:06.647 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:d9:38:67 brd ff:ff:ff:ff:ff:ff 2024-05-09T04:34:06.647 INFO:teuthology.orchestra.run.smithi145.stdout: altname enp7s0f1 2024-05-09T04:34:06.647 INFO:teuthology.orchestra.run.smithi145.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T04:34:06.647 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:88:77:90 brd ff:ff:ff:ff:ff:ff 2024-05-09T04:34:06.647 INFO:teuthology.orchestra.run.smithi145.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-09T04:34:06.647 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:88:77:91 brd ff:ff:ff:ff:ff:ff 2024-05-09T04:34:06.647 INFO:teuthology.orchestra.run.smithi145.stdout: inet 172.21.15.145/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-09T04:34:06.647 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft 42614sec preferred_lft 42614sec 2024-05-09T04:34:06.647 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::ec4:7aff:fe88:7791/64 scope link 2024-05-09T04:34:06.647 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-05-09T04:34:06.648 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-09T04:34:06.648 DEBUG:teuthology.orchestra.run.smithi145:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T04:34:06.648 DEBUG:teuthology.orchestra.run.smithi145:> set -e 2024-05-09T04:34:06.648 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip link add name ceph-brx type bridge 2024-05-09T04:34:06.648 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip addr flush dev ceph-brx 2024-05-09T04:34:06.648 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip link set ceph-brx up 2024-05-09T04:34:06.648 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-09T04:34:06.648 DEBUG:teuthology.orchestra.run.smithi145:> ') 2024-05-09T04:34:06.749 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T04:34:07.113 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T04:34:07.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:07.123 DEBUG:teuthology.orchestra.run.smithi145:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-09T04:34:07.187 INFO:teuthology.orchestra.run.smithi145.stdout:1 2024-05-09T04:34:07.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:07.191 DEBUG:teuthology.orchestra.run.smithi145:> route 2024-05-09T04:34:07.239 INFO:teuthology.orchestra.run.smithi145.stdout:Kernel IP routing table 2024-05-09T04:34:07.239 INFO:teuthology.orchestra.run.smithi145.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-09T04:34:07.239 INFO:teuthology.orchestra.run.smithi145.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-09T04:34:07.239 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-09T04:34:07.239 INFO:teuthology.orchestra.run.smithi145.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-09T04:34:07.239 DEBUG:teuthology.orchestra.run.smithi145:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T04:34:07.239 DEBUG:teuthology.orchestra.run.smithi145:> set -e 2024-05-09T04:34:07.240 DEBUG:teuthology.orchestra.run.smithi145:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-09T04:34:07.240 DEBUG:teuthology.orchestra.run.smithi145:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-09T04:34:07.240 DEBUG:teuthology.orchestra.run.smithi145:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-09T04:34:07.240 DEBUG:teuthology.orchestra.run.smithi145:> ') 2024-05-09T04:34:07.327 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T04:34:08.006 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T04:34:08.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:08.016 DEBUG:teuthology.orchestra.run.smithi145:> ip netns list 2024-05-09T04:34:08.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:08.056 DEBUG:teuthology.orchestra.run.smithi145:> ip netns list-id 2024-05-09T04:34:08.143 DEBUG:teuthology.orchestra.run.smithi145:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T04:34:08.144 DEBUG:teuthology.orchestra.run.smithi145:> set -e 2024-05-09T04:34:08.144 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T04:34:08.144 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-09T04:34:08.144 DEBUG:teuthology.orchestra.run.smithi145:> ') 2024-05-09T04:34:08.279 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T04:34:08.344 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T04:34:08.353 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-09T04:34:08.353 DEBUG:teuthology.orchestra.run.smithi145:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T04:34:08.353 DEBUG:teuthology.orchestra.run.smithi145:> set -e 2024-05-09T04:34:08.353 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-09T04:34:08.353 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-09T04:34:08.353 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-09T04:34:08.354 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-09T04:34:08.354 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-09T04:34:08.354 DEBUG:teuthology.orchestra.run.smithi145:> ') 2024-05-09T04:34:08.444 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T04:34:08.650 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T04:34:08.659 DEBUG:teuthology.orchestra.run.smithi145:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T04:34:08.659 DEBUG:teuthology.orchestra.run.smithi145:> set -e 2024-05-09T04:34:08.660 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip link set brx.0 up 2024-05-09T04:34:08.660 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip link set dev brx.0 master ceph-brx 2024-05-09T04:34:08.660 DEBUG:teuthology.orchestra.run.smithi145:> ') 2024-05-09T04:34:08.747 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T04:34:08.841 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T04:34:08.849 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-09T04:34:08.849 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T04:34:08.849 DEBUG:teuthology.orchestra.run.smithi145:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-09T04:34:08.887 INFO:teuthology.orchestra.run.smithi145.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-09T04:34:08.888 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T04:34:08.888 DEBUG:teuthology.orchestra.run.smithi145:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-09T04:34:08.971 DEBUG:teuthology.orchestra.run.smithi145:> sudo modprobe fuse 2024-05-09T04:34:09.088 DEBUG:teuthology.orchestra.run.smithi145:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T04:34:09.138 INFO:teuthology.orchestra.run.smithi145.stdout:/proc 2024-05-09T04:34:09.138 INFO:teuthology.orchestra.run.smithi145.stdout:/sys 2024-05-09T04:34:09.138 INFO:teuthology.orchestra.run.smithi145.stdout:/dev 2024-05-09T04:34:09.139 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/kernel/security 2024-05-09T04:34:09.139 INFO:teuthology.orchestra.run.smithi145.stdout:/dev/shm 2024-05-09T04:34:09.139 INFO:teuthology.orchestra.run.smithi145.stdout:/dev/pts 2024-05-09T04:34:09.139 INFO:teuthology.orchestra.run.smithi145.stdout:/run 2024-05-09T04:34:09.139 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/fs/cgroup 2024-05-09T04:34:09.139 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/fs/pstore 2024-05-09T04:34:09.139 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/fs/bpf 2024-05-09T04:34:09.139 INFO:teuthology.orchestra.run.smithi145.stdout:/ 2024-05-09T04:34:09.139 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/fs/selinux 2024-05-09T04:34:09.139 INFO:teuthology.orchestra.run.smithi145.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T04:34:09.139 INFO:teuthology.orchestra.run.smithi145.stdout:/dev/mqueue 2024-05-09T04:34:09.139 INFO:teuthology.orchestra.run.smithi145.stdout:/dev/hugepages 2024-05-09T04:34:09.139 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/kernel/debug 2024-05-09T04:34:09.139 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/kernel/tracing 2024-05-09T04:34:09.139 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/fs/fuse/connections 2024-05-09T04:34:09.140 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/kernel/config 2024-05-09T04:34:09.140 INFO:teuthology.orchestra.run.smithi145.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T04:34:09.140 INFO:teuthology.orchestra.run.smithi145.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T04:34:09.140 INFO:teuthology.orchestra.run.smithi145.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T04:34:09.140 INFO:teuthology.orchestra.run.smithi145.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T04:34:09.140 INFO:teuthology.orchestra.run.smithi145.stdout:/run/user/1000 2024-05-09T04:34:09.140 INFO:teuthology.orchestra.run.smithi145.stdout:/var/lib/ceph 2024-05-09T04:34:09.140 INFO:teuthology.orchestra.run.smithi145.stdout:/var/lib/ceph/osd/ceph-0 2024-05-09T04:34:09.140 INFO:teuthology.orchestra.run.smithi145.stdout:/var/lib/ceph/osd/ceph-1 2024-05-09T04:34:09.140 INFO:teuthology.orchestra.run.smithi145.stdout:/var/lib/ceph/osd/ceph-2 2024-05-09T04:34:09.140 INFO:teuthology.orchestra.run.smithi145.stdout:/var/lib/ceph/osd/ceph-3 2024-05-09T04:34:09.140 INFO:teuthology.orchestra.run.smithi145.stdout:/run/netns 2024-05-09T04:34:09.140 INFO:teuthology.orchestra.run.smithi145.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T04:34:09.140 INFO:teuthology.orchestra.run.smithi145.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T04:34:09.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:09.142 DEBUG:teuthology.orchestra.run.smithi145:> ls /sys/fs/fuse/connections 2024-05-09T04:34:09.187 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-09T04:34:09.187 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-09T04:34:09.233 DEBUG:teuthology.orchestra.run.smithi145:> sudo modprobe fuse 2024-05-09T04:34:09.300 DEBUG:teuthology.orchestra.run.smithi145:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T04:34:09.341 INFO:teuthology.orchestra.run.smithi145.stdout:/proc 2024-05-09T04:34:09.341 INFO:teuthology.orchestra.run.smithi145.stdout:/sys 2024-05-09T04:34:09.341 INFO:teuthology.orchestra.run.smithi145.stdout:/dev 2024-05-09T04:34:09.341 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/kernel/security 2024-05-09T04:34:09.341 INFO:teuthology.orchestra.run.smithi145.stdout:/dev/shm 2024-05-09T04:34:09.341 INFO:teuthology.orchestra.run.smithi145.stdout:/dev/pts 2024-05-09T04:34:09.342 INFO:teuthology.orchestra.run.smithi145.stdout:/run 2024-05-09T04:34:09.342 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/fs/cgroup 2024-05-09T04:34:09.342 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/fs/pstore 2024-05-09T04:34:09.342 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/fs/bpf 2024-05-09T04:34:09.342 INFO:teuthology.orchestra.run.smithi145.stdout:/ 2024-05-09T04:34:09.342 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/fs/selinux 2024-05-09T04:34:09.342 INFO:teuthology.orchestra.run.smithi145.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T04:34:09.342 INFO:teuthology.orchestra.run.smithi145.stdout:/dev/mqueue 2024-05-09T04:34:09.342 INFO:teuthology.orchestra.run.smithi145.stdout:/dev/hugepages 2024-05-09T04:34:09.346 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/kernel/debug 2024-05-09T04:34:09.346 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/kernel/tracing 2024-05-09T04:34:09.346 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/fs/fuse/connections 2024-05-09T04:34:09.346 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/kernel/config 2024-05-09T04:34:09.346 INFO:teuthology.orchestra.run.smithi145.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T04:34:09.346 INFO:teuthology.orchestra.run.smithi145.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T04:34:09.346 INFO:teuthology.orchestra.run.smithi145.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T04:34:09.346 INFO:teuthology.orchestra.run.smithi145.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T04:34:09.346 INFO:teuthology.orchestra.run.smithi145.stdout:/run/user/1000 2024-05-09T04:34:09.346 INFO:teuthology.orchestra.run.smithi145.stdout:/var/lib/ceph 2024-05-09T04:34:09.346 INFO:teuthology.orchestra.run.smithi145.stdout:/var/lib/ceph/osd/ceph-0 2024-05-09T04:34:09.346 INFO:teuthology.orchestra.run.smithi145.stdout:/var/lib/ceph/osd/ceph-1 2024-05-09T04:34:09.346 INFO:teuthology.orchestra.run.smithi145.stdout:/var/lib/ceph/osd/ceph-2 2024-05-09T04:34:09.346 INFO:teuthology.orchestra.run.smithi145.stdout:/var/lib/ceph/osd/ceph-3 2024-05-09T04:34:09.347 INFO:teuthology.orchestra.run.smithi145.stdout:/run/netns 2024-05-09T04:34:09.347 INFO:teuthology.orchestra.run.smithi145.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T04:34:09.347 INFO:teuthology.orchestra.run.smithi145.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T04:34:09.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:09.348 DEBUG:teuthology.orchestra.run.smithi145:> ls /sys/fs/fuse/connections 2024-05-09T04:34:09.406 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi145.stderr:2024-05-09T04:34:09.408+0000 7f82f00b2480 -1 init, newargv = 0x55adb933a350 newargc=15 2024-05-09T04:34:09.407 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi145.stderr:ceph-fuse[42977]: starting ceph client 2024-05-09T04:34:09.420 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi145.stderr:ceph-fuse[42977]: starting fuse 2024-05-09T04:34:09.434 INFO:teuthology.orchestra.run.smithi145.stdout:40 2024-05-09T04:34:09.434 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2024-05-09T04:34:09.434 DEBUG:teuthology.orchestra.run.smithi145:> sudo stdin-killer -- python3 -c ' 2024-05-09T04:34:09.434 DEBUG:teuthology.orchestra.run.smithi145:> import glob 2024-05-09T04:34:09.434 DEBUG:teuthology.orchestra.run.smithi145:> import re 2024-05-09T04:34:09.434 DEBUG:teuthology.orchestra.run.smithi145:> import os 2024-05-09T04:34:09.434 DEBUG:teuthology.orchestra.run.smithi145:> import subprocess 2024-05-09T04:34:09.435 DEBUG:teuthology.orchestra.run.smithi145:> 2024-05-09T04:34:09.435 DEBUG:teuthology.orchestra.run.smithi145:> def _find_admin_socket(client_name): 2024-05-09T04:34:09.435 DEBUG:teuthology.orchestra.run.smithi145:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-09T04:34:09.435 DEBUG:teuthology.orchestra.run.smithi145:> files = glob.glob(asok_path) 2024-05-09T04:34:09.435 DEBUG:teuthology.orchestra.run.smithi145:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-09T04:34:09.435 DEBUG:teuthology.orchestra.run.smithi145:> 2024-05-09T04:34:09.435 DEBUG:teuthology.orchestra.run.smithi145:> # Given a non-glob path, it better be there 2024-05-09T04:34:09.435 DEBUG:teuthology.orchestra.run.smithi145:> if "*" not in asok_path: 2024-05-09T04:34:09.435 DEBUG:teuthology.orchestra.run.smithi145:> assert(len(files) == 1) 2024-05-09T04:34:09.435 DEBUG:teuthology.orchestra.run.smithi145:> return files[0] 2024-05-09T04:34:09.435 DEBUG:teuthology.orchestra.run.smithi145:> 2024-05-09T04:34:09.435 DEBUG:teuthology.orchestra.run.smithi145:> for f in files: 2024-05-09T04:34:09.435 DEBUG:teuthology.orchestra.run.smithi145:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-09T04:34:09.435 DEBUG:teuthology.orchestra.run.smithi145:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-09T04:34:09.435 DEBUG:teuthology.orchestra.run.smithi145:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-09T04:34:09.435 DEBUG:teuthology.orchestra.run.smithi145:> contents = proc_f.read() 2024-05-09T04:34:09.436 DEBUG:teuthology.orchestra.run.smithi145:> if mountpoint in contents: 2024-05-09T04:34:09.436 DEBUG:teuthology.orchestra.run.smithi145:> return f 2024-05-09T04:34:09.436 DEBUG:teuthology.orchestra.run.smithi145:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-09T04:34:09.436 DEBUG:teuthology.orchestra.run.smithi145:> 2024-05-09T04:34:09.436 DEBUG:teuthology.orchestra.run.smithi145:> print(_find_admin_socket("client.0")) 2024-05-09T04:34:09.436 DEBUG:teuthology.orchestra.run.smithi145:> ' 2024-05-09T04:34:09.632 INFO:teuthology.orchestra.run.smithi145.stdout:/var/run/ceph/ceph-client.0.42977.asok 2024-05-09T04:34:09.636 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T04:34:09.649 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.42977.asok 2024-05-09T04:34:09.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:09.649 DEBUG:teuthology.orchestra.run.smithi145:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.42977.asok status 2024-05-09T04:34:09.874 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-05-09T04:34:09.875 INFO:teuthology.orchestra.run.smithi145.stdout: "metadata": { 2024-05-09T04:34:09.875 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_sha1": "165a15e51d08073b5317c443415b0d8a7141b1ab", 2024-05-09T04:34:09.875 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_version": "ceph version 19.0.0-3568-g165a15e5 (165a15e51d08073b5317c443415b0d8a7141b1ab) squid (dev)", 2024-05-09T04:34:09.875 INFO:teuthology.orchestra.run.smithi145.stdout: "entity_id": "0", 2024-05-09T04:34:09.875 INFO:teuthology.orchestra.run.smithi145.stdout: "hostname": "smithi145", 2024-05-09T04:34:09.875 INFO:teuthology.orchestra.run.smithi145.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-09T04:34:09.875 INFO:teuthology.orchestra.run.smithi145.stdout: "pid": "42977", 2024-05-09T04:34:09.875 INFO:teuthology.orchestra.run.smithi145.stdout: "root": "/" 2024-05-09T04:34:09.875 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:34:09.875 INFO:teuthology.orchestra.run.smithi145.stdout: "dentry_count": 0, 2024-05-09T04:34:09.875 INFO:teuthology.orchestra.run.smithi145.stdout: "dentry_pinned_count": 0, 2024-05-09T04:34:09.875 INFO:teuthology.orchestra.run.smithi145.stdout: "id": 4578, 2024-05-09T04:34:09.875 INFO:teuthology.orchestra.run.smithi145.stdout: "inst": { 2024-05-09T04:34:09.875 INFO:teuthology.orchestra.run.smithi145.stdout: "name": { 2024-05-09T04:34:09.876 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "client", 2024-05-09T04:34:09.876 INFO:teuthology.orchestra.run.smithi145.stdout: "num": 4578 2024-05-09T04:34:09.876 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:34:09.876 INFO:teuthology.orchestra.run.smithi145.stdout: "addr": { 2024-05-09T04:34:09.876 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "v1", 2024-05-09T04:34:09.876 INFO:teuthology.orchestra.run.smithi145.stdout: "addr": "192.168.0.1:0", 2024-05-09T04:34:09.876 INFO:teuthology.orchestra.run.smithi145.stdout: "nonce": 3880004281 2024-05-09T04:34:09.876 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-05-09T04:34:09.876 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:34:09.876 INFO:teuthology.orchestra.run.smithi145.stdout: "addr": { 2024-05-09T04:34:09.876 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "v1", 2024-05-09T04:34:09.876 INFO:teuthology.orchestra.run.smithi145.stdout: "addr": "192.168.0.1:0", 2024-05-09T04:34:09.876 INFO:teuthology.orchestra.run.smithi145.stdout: "nonce": 3880004281 2024-05-09T04:34:09.876 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:34:09.876 INFO:teuthology.orchestra.run.smithi145.stdout: "inst_str": "client.4578 192.168.0.1:0/3880004281", 2024-05-09T04:34:09.876 INFO:teuthology.orchestra.run.smithi145.stdout: "addr_str": "192.168.0.1:0/3880004281", 2024-05-09T04:34:09.877 INFO:teuthology.orchestra.run.smithi145.stdout: "inode_count": 1, 2024-05-09T04:34:09.877 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_epoch": 8, 2024-05-09T04:34:09.877 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_epoch": 20, 2024-05-09T04:34:09.877 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_epoch_barrier": 0, 2024-05-09T04:34:09.877 INFO:teuthology.orchestra.run.smithi145.stdout: "blocklisted": false, 2024-05-09T04:34:09.877 INFO:teuthology.orchestra.run.smithi145.stdout: "fs_name": "cephfs" 2024-05-09T04:34:09.877 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-05-09T04:34:09.889 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-09T04:34:10.084 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.085+0000 7fa8bf000640 1 Processor -- start 2024-05-09T04:34:10.084 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.085+0000 7fa8bf000640 1 -- start start 2024-05-09T04:34:10.086 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.087+0000 7fa8bf000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa8b806f0f0 0x7fa8b80b4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:10.086 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.087+0000 7fa8bf000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fa8b815bac0 0x7fa8b806f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:10.086 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.087+0000 7fa8bf000640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8b806fa10 con 0x7fa8b80738c0 2024-05-09T04:34:10.086 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.087+0000 7fa8bf000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fa8b80cf4b0 con 0x7fa8b815bac0 2024-05-09T04:34:10.086 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.087+0000 7fa8bf000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fa8b80b4ec0 con 0x7fa8b806f0f0 2024-05-09T04:34:10.086 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.087+0000 7fa8b7400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fa8b815bac0 0x7fa8b806f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:10.086 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.087+0000 7fa8b7400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fa8b815bac0 0x7fa8b806f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:37958/0 (socket says 172.21.15.145:37958) 2024-05-09T04:34:10.086 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.087+0000 7fa8b7400640 1 -- 172.21.15.145:0/4274421874 learned_addr learned my addr 172.21.15.145:0/4274421874 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:10.087 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.088+0000 7fa8b7400640 1 -- 172.21.15.145:0/4274421874 >> v2:172.21.15.192:3300/0 conn(0x7fa8b806f0f0 msgr2=0x7fa8b80b4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:10.087 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.088+0000 7fa8b7e00640 1 --2- 172.21.15.145:0/4274421874 >> v2:172.21.15.192:3300/0 conn(0x7fa8b806f0f0 0x7fa8b80b4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:10.087 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.088+0000 7fa8b7400640 1 --2- 172.21.15.145:0/4274421874 >> v2:172.21.15.192:3300/0 conn(0x7fa8b806f0f0 0x7fa8b80b4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:10.087 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.088+0000 7fa8b7400640 1 -- 172.21.15.145:0/4274421874 >> v1:172.21.15.192:6789/0 conn(0x7fa8b80738c0 legacy=0x7fa8b8072220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:10.087 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.088+0000 7fa8b7400640 1 -- 172.21.15.145:0/4274421874 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8b80b5af0 con 0x7fa8b815bac0 2024-05-09T04:34:10.087 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.088+0000 7fa8b7e00640 1 --2- 172.21.15.145:0/4274421874 >> v2:172.21.15.192:3300/0 conn(0x7fa8b806f0f0 0x7fa8b80b4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:34:10.088 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.089+0000 7fa8b7400640 1 --2- 172.21.15.145:0/4274421874 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8b815bac0 0x7fa8b806f4d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8071c10 tx=0x7fa8a002b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff34649e94a64ae0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:10.088 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.089+0000 7fa8b6a00640 1 -- 172.21.15.145:0/4274421874 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a000b040 con 0x7fa8b815bac0 2024-05-09T04:34:10.088 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.089+0000 7fa8b6a00640 1 -- 172.21.15.145:0/4274421874 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8a0030710 con 0x7fa8b815bac0 2024-05-09T04:34:10.088 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.089+0000 7fa8b6a00640 1 -- 172.21.15.145:0/4274421874 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a0030ae0 con 0x7fa8b815bac0 2024-05-09T04:34:10.089 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.090+0000 7fa8bf000640 1 -- 172.21.15.145:0/4274421874 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8b815bac0 msgr2=0x7fa8b806f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:10.089 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.090+0000 7fa8bf000640 1 --2- 172.21.15.145:0/4274421874 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8b815bac0 0x7fa8b806f4d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8071c10 tx=0x7fa8a002b5a0 comp rx=0 tx=0).stop 2024-05-09T04:34:10.089 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.090+0000 7fa8bf000640 1 -- 172.21.15.145:0/4274421874 shutdown_connections 2024-05-09T04:34:10.089 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.090+0000 7fa8bf000640 1 --2- 172.21.15.145:0/4274421874 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8b815bac0 0x7fa8b806f4d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:10.089 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.090+0000 7fa8bf000640 1 --2- 172.21.15.145:0/4274421874 >> v2:172.21.15.192:3300/0 conn(0x7fa8b806f0f0 0x7fa8b80b4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:10.089 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.090+0000 7fa8bf000640 1 -- 172.21.15.145:0/4274421874 >> 172.21.15.145:0/4274421874 conn(0x7fa8b806c870 msgr2=0x7fa8b806cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:10.089 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.090+0000 7fa8bf000640 1 -- 172.21.15.145:0/4274421874 shutdown_connections 2024-05-09T04:34:10.089 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.090+0000 7fa8bf000640 1 -- 172.21.15.145:0/4274421874 wait complete. 2024-05-09T04:34:10.090 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.092+0000 7fa8bf000640 1 Processor -- start 2024-05-09T04:34:10.090 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.092+0000 7fa8bf000640 1 -- start start 2024-05-09T04:34:10.092 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.093+0000 7fa8bf000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa8b806f0f0 0x7fa8b81f9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:10.092 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.093+0000 7fa8bf000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8b80738c0 0x7fa8b813f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:10.092 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.093+0000 7fa8bf000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8b815bac0 0x7fa8b80af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:10.092 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.093+0000 7fa8bf000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b80b6390 con 0x7fa8b815bac0 2024-05-09T04:34:10.092 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.093+0000 7fa8bf000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b80b60b0 con 0x7fa8b80738c0 2024-05-09T04:34:10.092 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.093+0000 7fa8bf000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa8b80cf140 con 0x7fa8b806f0f0 2024-05-09T04:34:10.092 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.093+0000 7fa8bd000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8b815bac0 0x7fa8b80af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:10.092 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.093+0000 7fa8bd000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8b815bac0 0x7fa8b80af8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:37970/0 (socket says 172.21.15.145:37970) 2024-05-09T04:34:10.092 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.093+0000 7fa8bd000640 1 -- 172.21.15.145:0/2173318888 learned_addr learned my addr 172.21.15.145:0/2173318888 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:10.094 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.094+0000 7fa8bd000640 1 -- 172.21.15.145:0/2173318888 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa8b806f0f0 msgr2=0x7fa8b81f9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:10.094 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.094+0000 7fa8b7e00640 1 --2- 172.21.15.145:0/2173318888 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa8b806f0f0 0x7fa8b81f9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:10.094 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.094+0000 7fa8b7400640 1 --2- 172.21.15.145:0/2173318888 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8b80738c0 0x7fa8b813f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:10.094 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.094+0000 7fa8bd000640 1 --2- 172.21.15.145:0/2173318888 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa8b806f0f0 0x7fa8b81f9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:10.094 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.094+0000 7fa8bd000640 1 -- 172.21.15.145:0/2173318888 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8b80738c0 msgr2=0x7fa8b813f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:10.094 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.094+0000 7fa8bd000640 1 --2- 172.21.15.145:0/2173318888 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8b80738c0 0x7fa8b813f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:10.094 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.094+0000 7fa8bd000640 1 -- 172.21.15.145:0/2173318888 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8b8072730 con 0x7fa8b815bac0 2024-05-09T04:34:10.094 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.094+0000 7fa8b7e00640 1 --2- 172.21.15.145:0/2173318888 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa8b806f0f0 0x7fa8b81f9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:34:10.094 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.094+0000 7fa8bd000640 1 --2- 172.21.15.145:0/2173318888 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8b815bac0 0x7fa8b80af8d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa8b000c3e0 tx=0x7fa8b000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:10.094 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.094+0000 7fa8afe00640 1 -- 172.21.15.145:0/2173318888 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b0011790 con 0x7fa8b815bac0 2024-05-09T04:34:10.094 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.094+0000 7fa8afe00640 1 -- 172.21.15.145:0/2173318888 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8b0017070 con 0x7fa8b815bac0 2024-05-09T04:34:10.094 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.094+0000 7fa8afe00640 1 -- 172.21.15.145:0/2173318888 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b0011bb0 con 0x7fa8b815bac0 2024-05-09T04:34:10.095 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.094+0000 7fa8bf000640 1 -- 172.21.15.145:0/2173318888 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8b8142a90 con 0x7fa8b815bac0 2024-05-09T04:34:10.095 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.094+0000 7fa8bf000640 1 -- 172.21.15.145:0/2173318888 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8b81f8580 con 0x7fa8b815bac0 2024-05-09T04:34:10.095 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.096+0000 7fa8bf000640 1 -- 172.21.15.145:0/2173318888 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8b806a310 con 0x7fa8b815bac0 2024-05-09T04:34:10.095 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.097+0000 7fa8afe00640 1 -- 172.21.15.145:0/2173318888 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa8b001c460 con 0x7fa8b815bac0 2024-05-09T04:34:10.096 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.098+0000 7fa8afe00640 1 --2- 172.21.15.145:0/2173318888 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fa894076300 0x7fa8940787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:10.096 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.098+0000 7fa8afe00640 1 -- 172.21.15.145:0/2173318888 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa8b0097430 con 0x7fa8b815bac0 2024-05-09T04:34:10.097 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.098+0000 7fa8b7e00640 1 --2- 172.21.15.145:0/2173318888 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fa894076300 0x7fa8940787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:10.097 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.099+0000 7fa8b7e00640 1 --2- 172.21.15.145:0/2173318888 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fa894076300 0x7fa8940787c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa8a800d430 tx=0x7fa8a8007530 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:10.101 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.102+0000 7fa8afe00640 1 -- 172.21.15.145:0/2173318888 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa8b0060cf0 con 0x7fa8b815bac0 2024-05-09T04:34:10.348 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.349+0000 7fa8bf000640 1 -- 172.21.15.145:0/2173318888 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa8b80d0b50 con 0x7fa8b815bac0 2024-05-09T04:34:10.349 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.350+0000 7fa8afe00640 1 -- 172.21.15.145:0/2173318888 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7fa8b00605f0 con 0x7fa8b815bac0 2024-05-09T04:34:10.349 INFO:teuthology.orchestra.run.smithi145.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-09T04:34:10.352 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.353+0000 7fa8bf000640 1 -- 172.21.15.145:0/2173318888 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fa894076300 msgr2=0x7fa8940787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:10.352 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.353+0000 7fa8bf000640 1 --2- 172.21.15.145:0/2173318888 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fa894076300 0x7fa8940787c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa8a800d430 tx=0x7fa8a8007530 comp rx=0 tx=0).stop 2024-05-09T04:34:10.352 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.353+0000 7fa8bf000640 1 -- 172.21.15.145:0/2173318888 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8b815bac0 msgr2=0x7fa8b80af8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:10.352 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.353+0000 7fa8bf000640 1 --2- 172.21.15.145:0/2173318888 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8b815bac0 0x7fa8b80af8d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa8b000c3e0 tx=0x7fa8b000c8a0 comp rx=0 tx=0).stop 2024-05-09T04:34:10.352 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.353+0000 7fa8bf000640 1 -- 172.21.15.145:0/2173318888 shutdown_connections 2024-05-09T04:34:10.352 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.353+0000 7fa8bf000640 1 --2- 172.21.15.145:0/2173318888 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fa894076300 0x7fa8940787c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:10.353 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.353+0000 7fa8bf000640 1 --2- 172.21.15.145:0/2173318888 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8b815bac0 0x7fa8b80af8d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:10.353 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.353+0000 7fa8bf000640 1 --2- 172.21.15.145:0/2173318888 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8b80738c0 0x7fa8b813f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:10.353 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.353+0000 7fa8bf000640 1 --2- 172.21.15.145:0/2173318888 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa8b806f0f0 0x7fa8b81f9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:10.353 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.353+0000 7fa8bf000640 1 -- 172.21.15.145:0/2173318888 >> 172.21.15.145:0/2173318888 conn(0x7fa8b806c870 msgr2=0x7fa8b8155c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:10.353 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.353+0000 7fa8bf000640 1 -- 172.21.15.145:0/2173318888 shutdown_connections 2024-05-09T04:34:10.353 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10.353+0000 7fa8bf000640 1 -- 172.21.15.145:0/2173318888 wait complete. 2024-05-09T04:34:10.371 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-09T04:34:10.371 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-09T04:34:10.371 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi145.front.sepia.ceph.com 2024-05-09T04:34:10.371 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-05-09T04:34:10.371 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-05-09T04:34:10.371 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-09T04:34:10.371 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-09T04:34:10.371 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-09T04:34:10.371 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-05-09T04:34:10.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:10.372 DEBUG:teuthology.orchestra.run.smithi145:> ip addr 2024-05-09T04:34:10.424 INFO:teuthology.orchestra.run.smithi145.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T04:34:10.424 INFO:teuthology.orchestra.run.smithi145.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T04:34:10.425 INFO:teuthology.orchestra.run.smithi145.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T04:34:10.425 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-05-09T04:34:10.425 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 ::1/128 scope host 2024-05-09T04:34:10.425 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-05-09T04:34:10.425 INFO:teuthology.orchestra.run.smithi145.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T04:34:10.425 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:d9:38:66 brd ff:ff:ff:ff:ff:ff 2024-05-09T04:34:10.425 INFO:teuthology.orchestra.run.smithi145.stdout: altname enp7s0f0 2024-05-09T04:34:10.425 INFO:teuthology.orchestra.run.smithi145.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T04:34:10.425 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:d9:38:67 brd ff:ff:ff:ff:ff:ff 2024-05-09T04:34:10.425 INFO:teuthology.orchestra.run.smithi145.stdout: altname enp7s0f1 2024-05-09T04:34:10.425 INFO:teuthology.orchestra.run.smithi145.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T04:34:10.425 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:88:77:90 brd ff:ff:ff:ff:ff:ff 2024-05-09T04:34:10.425 INFO:teuthology.orchestra.run.smithi145.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-09T04:34:10.425 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:88:77:91 brd ff:ff:ff:ff:ff:ff 2024-05-09T04:34:10.425 INFO:teuthology.orchestra.run.smithi145.stdout: inet 172.21.15.145/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-09T04:34:10.426 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft 42611sec preferred_lft 42611sec 2024-05-09T04:34:10.426 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::ec4:7aff:fe88:7791/64 scope link 2024-05-09T04:34:10.426 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-05-09T04:34:10.426 INFO:teuthology.orchestra.run.smithi145.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-09T04:34:10.426 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0e:0f:1c:f7:96:a9 brd ff:ff:ff:ff:ff:ff 2024-05-09T04:34:10.426 INFO:teuthology.orchestra.run.smithi145.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-09T04:34:10.426 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-05-09T04:34:10.426 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::e0a8:1bff:fe9d:6975/64 scope link 2024-05-09T04:34:10.426 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-05-09T04:34:10.426 INFO:teuthology.orchestra.run.smithi145.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-09T04:34:10.426 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0e:0f:1c:f7:96:a9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T04:34:10.426 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::c0f:1cff:fef7:96a9/64 scope link tentative 2024-05-09T04:34:10.426 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-05-09T04:34:10.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:10.427 DEBUG:teuthology.orchestra.run.smithi145:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-09T04:34:10.499 INFO:teuthology.orchestra.run.smithi145.stdout:1 2024-05-09T04:34:10.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:10.503 DEBUG:teuthology.orchestra.run.smithi145:> route 2024-05-09T04:34:10.552 INFO:teuthology.orchestra.run.smithi145.stdout:Kernel IP routing table 2024-05-09T04:34:10.552 INFO:teuthology.orchestra.run.smithi145.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-09T04:34:10.552 INFO:teuthology.orchestra.run.smithi145.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-09T04:34:10.552 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-09T04:34:10.552 INFO:teuthology.orchestra.run.smithi145.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-09T04:34:10.552 DEBUG:teuthology.orchestra.run.smithi145:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T04:34:10.553 DEBUG:teuthology.orchestra.run.smithi145:> set -e 2024-05-09T04:34:10.553 DEBUG:teuthology.orchestra.run.smithi145:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-09T04:34:10.553 DEBUG:teuthology.orchestra.run.smithi145:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-09T04:34:10.553 DEBUG:teuthology.orchestra.run.smithi145:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-09T04:34:10.553 DEBUG:teuthology.orchestra.run.smithi145:> ') 2024-05-09T04:34:10.647 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T04:34:10.744 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T04:34:10.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:10.755 DEBUG:teuthology.orchestra.run.smithi145:> ip netns list 2024-05-09T04:34:10.800 INFO:teuthology.orchestra.run.smithi145.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-09T04:34:10.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:10.801 DEBUG:teuthology.orchestra.run.smithi145:> ip netns list-id 2024-05-09T04:34:10.899 INFO:teuthology.orchestra.run.smithi145.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-05-09T04:34:10.899 DEBUG:teuthology.orchestra.run.smithi145:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T04:34:10.900 DEBUG:teuthology.orchestra.run.smithi145:> set -e 2024-05-09T04:34:10.900 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T04:34:10.900 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-05-09T04:34:10.900 DEBUG:teuthology.orchestra.run.smithi145:> ') 2024-05-09T04:34:11.005 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T04:34:11.072 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T04:34:11.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:11.082 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-05-09T04:34:11.161 INFO:teuthology.orchestra.run.smithi145.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T04:34:11.161 INFO:teuthology.orchestra.run.smithi145.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T04:34:11.161 INFO:teuthology.orchestra.run.smithi145.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T04:34:11.162 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-05-09T04:34:11.162 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 ::1/128 scope host 2024-05-09T04:34:11.162 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-05-09T04:34:11.162 INFO:teuthology.orchestra.run.smithi145.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-09T04:34:11.162 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 56:80:7e:77:ea:3f brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-09T04:34:11.162 INFO:teuthology.orchestra.run.smithi145.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-09T04:34:11.162 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-05-09T04:34:11.162 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::5480:7eff:fe77:ea3f/64 scope link 2024-05-09T04:34:11.162 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-05-09T04:34:11.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:11.165 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-05-09T04:34:11.250 INFO:teuthology.orchestra.run.smithi145.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T04:34:11.251 INFO:teuthology.orchestra.run.smithi145.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T04:34:11.251 INFO:teuthology.orchestra.run.smithi145.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T04:34:11.251 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-05-09T04:34:11.251 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 ::1/128 scope host 2024-05-09T04:34:11.251 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-05-09T04:34:11.251 INFO:teuthology.orchestra.run.smithi145.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-09T04:34:11.251 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 56:80:7e:77:ea:3f brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-09T04:34:11.251 INFO:teuthology.orchestra.run.smithi145.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-09T04:34:11.251 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-05-09T04:34:11.251 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::5480:7eff:fe77:ea3f/64 scope link 2024-05-09T04:34:11.251 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-05-09T04:34:11.254 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-05-09T04:34:11.255 DEBUG:teuthology.orchestra.run.smithi145:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T04:34:11.255 DEBUG:teuthology.orchestra.run.smithi145:> set -e 2024-05-09T04:34:11.255 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-05-09T04:34:11.255 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-05-09T04:34:11.255 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-05-09T04:34:11.255 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-05-09T04:34:11.255 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-05-09T04:34:11.255 DEBUG:teuthology.orchestra.run.smithi145:> ') 2024-05-09T04:34:11.361 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T04:34:11.522 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T04:34:11.531 DEBUG:teuthology.orchestra.run.smithi145:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T04:34:11.531 DEBUG:teuthology.orchestra.run.smithi145:> set -e 2024-05-09T04:34:11.531 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip link set brx.1 up 2024-05-09T04:34:11.531 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip link set dev brx.1 master ceph-brx 2024-05-09T04:34:11.531 DEBUG:teuthology.orchestra.run.smithi145:> ') 2024-05-09T04:34:11.626 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T04:34:11.725 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T04:34:11.734 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-05-09T04:34:11.734 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T04:34:11.734 DEBUG:teuthology.orchestra.run.smithi145:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-05-09T04:34:11.780 INFO:teuthology.orchestra.run.smithi145.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-05-09T04:34:11.781 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T04:34:11.781 DEBUG:teuthology.orchestra.run.smithi145:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-05-09T04:34:11.876 DEBUG:teuthology.orchestra.run.smithi145:> sudo modprobe fuse 2024-05-09T04:34:11.959 DEBUG:teuthology.orchestra.run.smithi145:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T04:34:12.015 INFO:teuthology.orchestra.run.smithi145.stdout:/proc 2024-05-09T04:34:12.016 INFO:teuthology.orchestra.run.smithi145.stdout:/sys 2024-05-09T04:34:12.016 INFO:teuthology.orchestra.run.smithi145.stdout:/dev 2024-05-09T04:34:12.016 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/kernel/security 2024-05-09T04:34:12.016 INFO:teuthology.orchestra.run.smithi145.stdout:/dev/shm 2024-05-09T04:34:12.016 INFO:teuthology.orchestra.run.smithi145.stdout:/dev/pts 2024-05-09T04:34:12.016 INFO:teuthology.orchestra.run.smithi145.stdout:/run 2024-05-09T04:34:12.017 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/fs/cgroup 2024-05-09T04:34:12.017 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/fs/pstore 2024-05-09T04:34:12.017 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/fs/bpf 2024-05-09T04:34:12.017 INFO:teuthology.orchestra.run.smithi145.stdout:/ 2024-05-09T04:34:12.017 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/fs/selinux 2024-05-09T04:34:12.017 INFO:teuthology.orchestra.run.smithi145.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T04:34:12.017 INFO:teuthology.orchestra.run.smithi145.stdout:/dev/mqueue 2024-05-09T04:34:12.018 INFO:teuthology.orchestra.run.smithi145.stdout:/dev/hugepages 2024-05-09T04:34:12.018 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/kernel/debug 2024-05-09T04:34:12.018 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/kernel/tracing 2024-05-09T04:34:12.018 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/fs/fuse/connections 2024-05-09T04:34:12.018 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/kernel/config 2024-05-09T04:34:12.018 INFO:teuthology.orchestra.run.smithi145.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T04:34:12.019 INFO:teuthology.orchestra.run.smithi145.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T04:34:12.019 INFO:teuthology.orchestra.run.smithi145.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T04:34:12.019 INFO:teuthology.orchestra.run.smithi145.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T04:34:12.019 INFO:teuthology.orchestra.run.smithi145.stdout:/run/user/1000 2024-05-09T04:34:12.019 INFO:teuthology.orchestra.run.smithi145.stdout:/var/lib/ceph 2024-05-09T04:34:12.019 INFO:teuthology.orchestra.run.smithi145.stdout:/var/lib/ceph/osd/ceph-0 2024-05-09T04:34:12.019 INFO:teuthology.orchestra.run.smithi145.stdout:/var/lib/ceph/osd/ceph-1 2024-05-09T04:34:12.019 INFO:teuthology.orchestra.run.smithi145.stdout:/var/lib/ceph/osd/ceph-2 2024-05-09T04:34:12.019 INFO:teuthology.orchestra.run.smithi145.stdout:/var/lib/ceph/osd/ceph-3 2024-05-09T04:34:12.019 INFO:teuthology.orchestra.run.smithi145.stdout:/run/netns 2024-05-09T04:34:12.019 INFO:teuthology.orchestra.run.smithi145.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T04:34:12.019 INFO:teuthology.orchestra.run.smithi145.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T04:34:12.019 INFO:teuthology.orchestra.run.smithi145.stdout:/home/ubuntu/cephtest/mnt.0 2024-05-09T04:34:12.019 INFO:teuthology.orchestra.run.smithi145.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T04:34:12.020 INFO:teuthology.orchestra.run.smithi145.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T04:34:12.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:12.020 DEBUG:teuthology.orchestra.run.smithi145:> ls /sys/fs/fuse/connections 2024-05-09T04:34:12.071 INFO:teuthology.orchestra.run.smithi145.stdout:40 2024-05-09T04:34:12.071 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [40] 2024-05-09T04:34:12.071 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-05-09T04:34:12.079 DEBUG:teuthology.orchestra.run.smithi145:> sudo modprobe fuse 2024-05-09T04:34:12.147 DEBUG:teuthology.orchestra.run.smithi145:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T04:34:12.187 INFO:teuthology.orchestra.run.smithi145.stdout:/proc 2024-05-09T04:34:12.187 INFO:teuthology.orchestra.run.smithi145.stdout:/sys 2024-05-09T04:34:12.187 INFO:teuthology.orchestra.run.smithi145.stdout:/dev 2024-05-09T04:34:12.187 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/kernel/security 2024-05-09T04:34:12.187 INFO:teuthology.orchestra.run.smithi145.stdout:/dev/shm 2024-05-09T04:34:12.187 INFO:teuthology.orchestra.run.smithi145.stdout:/dev/pts 2024-05-09T04:34:12.187 INFO:teuthology.orchestra.run.smithi145.stdout:/run 2024-05-09T04:34:12.187 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/fs/cgroup 2024-05-09T04:34:12.187 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/fs/pstore 2024-05-09T04:34:12.187 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/fs/bpf 2024-05-09T04:34:12.187 INFO:teuthology.orchestra.run.smithi145.stdout:/ 2024-05-09T04:34:12.187 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/fs/selinux 2024-05-09T04:34:12.187 INFO:teuthology.orchestra.run.smithi145.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T04:34:12.188 INFO:teuthology.orchestra.run.smithi145.stdout:/dev/mqueue 2024-05-09T04:34:12.188 INFO:teuthology.orchestra.run.smithi145.stdout:/dev/hugepages 2024-05-09T04:34:12.188 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/kernel/debug 2024-05-09T04:34:12.188 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/kernel/tracing 2024-05-09T04:34:12.188 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/fs/fuse/connections 2024-05-09T04:34:12.188 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/kernel/config 2024-05-09T04:34:12.188 INFO:teuthology.orchestra.run.smithi145.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T04:34:12.188 INFO:teuthology.orchestra.run.smithi145.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T04:34:12.188 INFO:teuthology.orchestra.run.smithi145.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T04:34:12.188 INFO:teuthology.orchestra.run.smithi145.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T04:34:12.188 INFO:teuthology.orchestra.run.smithi145.stdout:/run/user/1000 2024-05-09T04:34:12.188 INFO:teuthology.orchestra.run.smithi145.stdout:/var/lib/ceph 2024-05-09T04:34:12.188 INFO:teuthology.orchestra.run.smithi145.stdout:/var/lib/ceph/osd/ceph-0 2024-05-09T04:34:12.188 INFO:teuthology.orchestra.run.smithi145.stdout:/var/lib/ceph/osd/ceph-1 2024-05-09T04:34:12.188 INFO:teuthology.orchestra.run.smithi145.stdout:/var/lib/ceph/osd/ceph-2 2024-05-09T04:34:12.189 INFO:teuthology.orchestra.run.smithi145.stdout:/var/lib/ceph/osd/ceph-3 2024-05-09T04:34:12.189 INFO:teuthology.orchestra.run.smithi145.stdout:/run/netns 2024-05-09T04:34:12.189 INFO:teuthology.orchestra.run.smithi145.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T04:34:12.189 INFO:teuthology.orchestra.run.smithi145.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T04:34:12.189 INFO:teuthology.orchestra.run.smithi145.stdout:/home/ubuntu/cephtest/mnt.0 2024-05-09T04:34:12.189 INFO:teuthology.orchestra.run.smithi145.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T04:34:12.189 INFO:teuthology.orchestra.run.smithi145.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T04:34:12.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:12.190 DEBUG:teuthology.orchestra.run.smithi145:> ls /sys/fs/fuse/connections 2024-05-09T04:34:12.244 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi145.stderr:2024-05-09T04:34:12.246+0000 7f49bd13d480 -1 init, newargv = 0x5626825f1ff0 newargc=15 2024-05-09T04:34:12.245 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi145.stderr:ceph-fuse[43475]: starting ceph client 2024-05-09T04:34:12.255 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi145.stderr:ceph-fuse[43475]: starting fuse 2024-05-09T04:34:12.273 INFO:teuthology.orchestra.run.smithi145.stdout:40 2024-05-09T04:34:12.273 INFO:teuthology.orchestra.run.smithi145.stdout:43 2024-05-09T04:34:12.273 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40, 43] 2024-05-09T04:34:12.274 DEBUG:teuthology.orchestra.run.smithi145:> sudo stdin-killer -- python3 -c ' 2024-05-09T04:34:12.274 DEBUG:teuthology.orchestra.run.smithi145:> import glob 2024-05-09T04:34:12.274 DEBUG:teuthology.orchestra.run.smithi145:> import re 2024-05-09T04:34:12.274 DEBUG:teuthology.orchestra.run.smithi145:> import os 2024-05-09T04:34:12.274 DEBUG:teuthology.orchestra.run.smithi145:> import subprocess 2024-05-09T04:34:12.274 DEBUG:teuthology.orchestra.run.smithi145:> 2024-05-09T04:34:12.274 DEBUG:teuthology.orchestra.run.smithi145:> def _find_admin_socket(client_name): 2024-05-09T04:34:12.274 DEBUG:teuthology.orchestra.run.smithi145:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-05-09T04:34:12.274 DEBUG:teuthology.orchestra.run.smithi145:> files = glob.glob(asok_path) 2024-05-09T04:34:12.274 DEBUG:teuthology.orchestra.run.smithi145:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-05-09T04:34:12.274 DEBUG:teuthology.orchestra.run.smithi145:> 2024-05-09T04:34:12.275 DEBUG:teuthology.orchestra.run.smithi145:> # Given a non-glob path, it better be there 2024-05-09T04:34:12.275 DEBUG:teuthology.orchestra.run.smithi145:> if "*" not in asok_path: 2024-05-09T04:34:12.275 DEBUG:teuthology.orchestra.run.smithi145:> assert(len(files) == 1) 2024-05-09T04:34:12.275 DEBUG:teuthology.orchestra.run.smithi145:> return files[0] 2024-05-09T04:34:12.275 DEBUG:teuthology.orchestra.run.smithi145:> 2024-05-09T04:34:12.275 DEBUG:teuthology.orchestra.run.smithi145:> for f in files: 2024-05-09T04:34:12.275 DEBUG:teuthology.orchestra.run.smithi145:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-09T04:34:12.275 DEBUG:teuthology.orchestra.run.smithi145:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-09T04:34:12.275 DEBUG:teuthology.orchestra.run.smithi145:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-09T04:34:12.275 DEBUG:teuthology.orchestra.run.smithi145:> contents = proc_f.read() 2024-05-09T04:34:12.275 DEBUG:teuthology.orchestra.run.smithi145:> if mountpoint in contents: 2024-05-09T04:34:12.275 DEBUG:teuthology.orchestra.run.smithi145:> return f 2024-05-09T04:34:12.275 DEBUG:teuthology.orchestra.run.smithi145:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-09T04:34:12.275 DEBUG:teuthology.orchestra.run.smithi145:> 2024-05-09T04:34:12.276 DEBUG:teuthology.orchestra.run.smithi145:> print(_find_admin_socket("client.2")) 2024-05-09T04:34:12.276 DEBUG:teuthology.orchestra.run.smithi145:> ' 2024-05-09T04:34:12.460 INFO:teuthology.orchestra.run.smithi145.stdout:/var/run/ceph/ceph-client.2.43475.asok 2024-05-09T04:34:12.464 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T04:34:12.477 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.43475.asok 2024-05-09T04:34:12.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:12.477 DEBUG:teuthology.orchestra.run.smithi145:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.43475.asok status 2024-05-09T04:34:12.702 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-05-09T04:34:12.702 INFO:teuthology.orchestra.run.smithi145.stdout: "metadata": { 2024-05-09T04:34:12.702 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_sha1": "165a15e51d08073b5317c443415b0d8a7141b1ab", 2024-05-09T04:34:12.702 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_version": "ceph version 19.0.0-3568-g165a15e5 (165a15e51d08073b5317c443415b0d8a7141b1ab) squid (dev)", 2024-05-09T04:34:12.702 INFO:teuthology.orchestra.run.smithi145.stdout: "entity_id": "2", 2024-05-09T04:34:12.702 INFO:teuthology.orchestra.run.smithi145.stdout: "hostname": "smithi145", 2024-05-09T04:34:12.702 INFO:teuthology.orchestra.run.smithi145.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-05-09T04:34:12.703 INFO:teuthology.orchestra.run.smithi145.stdout: "pid": "43475", 2024-05-09T04:34:12.703 INFO:teuthology.orchestra.run.smithi145.stdout: "root": "/" 2024-05-09T04:34:12.703 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:34:12.703 INFO:teuthology.orchestra.run.smithi145.stdout: "dentry_count": 0, 2024-05-09T04:34:12.703 INFO:teuthology.orchestra.run.smithi145.stdout: "dentry_pinned_count": 0, 2024-05-09T04:34:12.703 INFO:teuthology.orchestra.run.smithi145.stdout: "id": 4590, 2024-05-09T04:34:12.703 INFO:teuthology.orchestra.run.smithi145.stdout: "inst": { 2024-05-09T04:34:12.703 INFO:teuthology.orchestra.run.smithi145.stdout: "name": { 2024-05-09T04:34:12.703 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "client", 2024-05-09T04:34:12.703 INFO:teuthology.orchestra.run.smithi145.stdout: "num": 4590 2024-05-09T04:34:12.703 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:34:12.703 INFO:teuthology.orchestra.run.smithi145.stdout: "addr": { 2024-05-09T04:34:12.703 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "v1", 2024-05-09T04:34:12.703 INFO:teuthology.orchestra.run.smithi145.stdout: "addr": "192.168.0.2:0", 2024-05-09T04:34:12.703 INFO:teuthology.orchestra.run.smithi145.stdout: "nonce": 3525027267 2024-05-09T04:34:12.704 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-05-09T04:34:12.704 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:34:12.704 INFO:teuthology.orchestra.run.smithi145.stdout: "addr": { 2024-05-09T04:34:12.704 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "v1", 2024-05-09T04:34:12.704 INFO:teuthology.orchestra.run.smithi145.stdout: "addr": "192.168.0.2:0", 2024-05-09T04:34:12.704 INFO:teuthology.orchestra.run.smithi145.stdout: "nonce": 3525027267 2024-05-09T04:34:12.704 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:34:12.704 INFO:teuthology.orchestra.run.smithi145.stdout: "inst_str": "client.4590 192.168.0.2:0/3525027267", 2024-05-09T04:34:12.704 INFO:teuthology.orchestra.run.smithi145.stdout: "addr_str": "192.168.0.2:0/3525027267", 2024-05-09T04:34:12.704 INFO:teuthology.orchestra.run.smithi145.stdout: "inode_count": 1, 2024-05-09T04:34:12.704 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_epoch": 8, 2024-05-09T04:34:12.704 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_epoch": 20, 2024-05-09T04:34:12.704 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_epoch_barrier": 0, 2024-05-09T04:34:12.704 INFO:teuthology.orchestra.run.smithi145.stdout: "blocklisted": false, 2024-05-09T04:34:12.704 INFO:teuthology.orchestra.run.smithi145.stdout: "fs_name": "cephfs" 2024-05-09T04:34:12.705 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-05-09T04:34:12.717 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-09T04:34:12.920 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:12.921+0000 7f62f1200640 1 Processor -- start 2024-05-09T04:34:12.920 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:12.921+0000 7f62f1200640 1 -- start start 2024-05-09T04:34:12.921 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:12.923+0000 7f62f1200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f62ec0af130 0x7f62ec16e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:12.922 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:12.923+0000 7f62f1200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62ec0af890 con 0x7f62ec06e320 2024-05-09T04:34:12.922 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:12.923+0000 7f62f1200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62ec0afb70 con 0x7f62ec163270 2024-05-09T04:34:12.922 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:12.923+0000 7f62f1200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f62ec072a80 con 0x7f62ec0af130 2024-05-09T04:34:12.922 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:12.923+0000 7f62ebe00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f62ec0af130 0x7f62ec16e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:12.922 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:12.923+0000 7f62e3e00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f62ec06e320 0x7f62ec073c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:34116/0 (socket says 172.21.15.145:34116) 2024-05-09T04:34:12.922 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:12.923+0000 7f62e3e00640 1 -- 172.21.15.145:0/1081766084 learned_addr learned my addr 172.21.15.145:0/1081766084 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:34:12.923 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:12.924+0000 7f62eaa00640 1 -- 172.21.15.145:0/1081766084 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 759063714 0 0) 0x7f62ec0af890 con 0x7f62ec06e320 2024-05-09T04:34:12.923 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:12.924+0000 7f62eaa00640 1 -- 172.21.15.145:0/1081766084 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f62d0003520 con 0x7f62ec06e320 2024-05-09T04:34:12.923 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:12.924+0000 7f62ebe00640 1 -- 172.21.15.145:0/1081766084 >> v1:172.21.15.192:6789/0 conn(0x7f62ec163270 legacy=0x7f62ec073560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:12.923 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:12.924+0000 7f62ebe00640 1 -- 172.21.15.145:0/1081766084 >> v1:172.21.15.145:6789/0 conn(0x7f62ec06e320 legacy=0x7f62ec073c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:12.923 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:12.924+0000 7f62ebe00640 1 -- 172.21.15.145:0/1081766084 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62ec072d60 con 0x7f62ec0af130 2024-05-09T04:34:12.924 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:12.925+0000 7f62ebe00640 1 --2- 172.21.15.145:0/1081766084 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62ec0af130 0x7f62ec16e0e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f62e4009d90 tx=0x7f62e402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f40a5f35d9c5711 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:12.924 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:12.925+0000 7f62eaa00640 1 -- 172.21.15.145:0/1081766084 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e4007740 con 0x7f62ec0af130 2024-05-09T04:34:12.924 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:12.926+0000 7f62eaa00640 1 -- 172.21.15.145:0/1081766084 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62e40078e0 con 0x7f62ec0af130 2024-05-09T04:34:12.924 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:12.926+0000 7f62eaa00640 1 -- 172.21.15.145:0/1081766084 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e4007c90 con 0x7f62ec0af130 2024-05-09T04:34:12.925 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:12.926+0000 7f62f1200640 1 -- 172.21.15.145:0/1081766084 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62ec0af130 msgr2=0x7f62ec16e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:12.925 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:12.926+0000 7f62f1200640 1 --2- 172.21.15.145:0/1081766084 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62ec0af130 0x7f62ec16e0e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f62e4009d90 tx=0x7f62e402a660 comp rx=0 tx=0).stop 2024-05-09T04:34:12.925 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:12.926+0000 7f62f1200640 1 -- 172.21.15.145:0/1081766084 shutdown_connections 2024-05-09T04:34:12.925 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:12.926+0000 7f62f1200640 1 --2- 172.21.15.145:0/1081766084 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62ec0af130 0x7f62ec16e0e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:12.925 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:12.926+0000 7f62f1200640 1 -- 172.21.15.145:0/1081766084 >> 172.21.15.145:0/1081766084 conn(0x7f62ec06c8a0 msgr2=0x7f62ec06ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:12.925 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:12.926+0000 7f62f1200640 1 -- 172.21.15.145:0/1081766084 shutdown_connections 2024-05-09T04:34:12.925 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:12.926+0000 7f62f1200640 1 -- 172.21.15.145:0/1081766084 wait complete. 2024-05-09T04:34:12.927 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:12.928+0000 7f62f1200640 1 Processor -- start 2024-05-09T04:34:12.927 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:12.928+0000 7f62f1200640 1 -- start start 2024-05-09T04:34:12.928 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:12.929+0000 7f62f1200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62ec06e320 0x7f62ec15e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:12.928 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:12.929+0000 7f62f1200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62ec0af130 0x7f62ec15e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:12.928 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:12.929+0000 7f62f1200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62ec163270 0x7f62ec164a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:12.928 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:12.929+0000 7f62f1200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f62ec163650 con 0x7f62ec0af130 2024-05-09T04:34:12.928 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:12.929+0000 7f62f1200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f62ec0af5a0 con 0x7f62ec163270 2024-05-09T04:34:12.928 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:12.929+0000 7f62f1200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f62ec1637d0 con 0x7f62ec06e320 2024-05-09T04:34:12.930 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:12.929+0000 7f62e3e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62ec0af130 0x7f62ec15e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:12.930 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:12.930+0000 7f62e3e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62ec0af130 0x7f62ec15e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:42264/0 (socket says 172.21.15.145:42264) 2024-05-09T04:34:12.930 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:12.930+0000 7f62e3e00640 1 -- 172.21.15.145:0/3833647863 learned_addr learned my addr 172.21.15.145:0/3833647863 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:12.930 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:12.930+0000 7f62eb400640 1 --2- 172.21.15.145:0/3833647863 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62ec06e320 0x7f62ec15e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:12.930 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:12.930+0000 7f62ebe00640 1 --2- 172.21.15.145:0/3833647863 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62ec163270 0x7f62ec164a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:12.930 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:12.930+0000 7f62e3e00640 1 -- 172.21.15.145:0/3833647863 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62ec06e320 msgr2=0x7f62ec15e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:12.930 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:12.930+0000 7f62e3e00640 1 --2- 172.21.15.145:0/3833647863 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62ec06e320 0x7f62ec15e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:12.930 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:12.930+0000 7f62e3e00640 1 -- 172.21.15.145:0/3833647863 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62ec163270 msgr2=0x7f62ec164a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:12.930 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:12.930+0000 7f62e3e00640 1 --2- 172.21.15.145:0/3833647863 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62ec163270 0x7f62ec164a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:12.931 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:12.930+0000 7f62e3e00640 1 -- 172.21.15.145:0/3833647863 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62ec0cd910 con 0x7f62ec0af130 2024-05-09T04:34:12.931 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:12.930+0000 7f62eb400640 1 --2- 172.21.15.145:0/3833647863 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62ec06e320 0x7f62ec15e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:34:12.931 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:12.930+0000 7f62ebe00640 1 --2- 172.21.15.145:0/3833647863 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62ec163270 0x7f62ec164a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:34:12.931 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:12.930+0000 7f62e3e00640 1 --2- 172.21.15.145:0/3833647863 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62ec0af130 0x7f62ec15e900 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f62d400c460 tx=0x7f62d400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:12.931 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:12.930+0000 7f62e3400640 1 -- 172.21.15.145:0/3833647863 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d4017070 con 0x7f62ec0af130 2024-05-09T04:34:12.931 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:12.931+0000 7f62e3400640 1 -- 172.21.15.145:0/3833647863 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62d4005d00 con 0x7f62ec0af130 2024-05-09T04:34:12.931 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:12.931+0000 7f62f1200640 1 -- 172.21.15.145:0/3833647863 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62ec0cdbf0 con 0x7f62ec0af130 2024-05-09T04:34:12.931 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:12.931+0000 7f62e3400640 1 -- 172.21.15.145:0/3833647863 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d40072b0 con 0x7f62ec0af130 2024-05-09T04:34:12.931 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:12.931+0000 7f62f1200640 1 -- 172.21.15.145:0/3833647863 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62ec0ce1b0 con 0x7f62ec0af130 2024-05-09T04:34:12.932 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:12.932+0000 7f62f1200640 1 -- 172.21.15.145:0/3833647863 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62ec06a310 con 0x7f62ec0af130 2024-05-09T04:34:12.932 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:12.933+0000 7f62e3400640 1 -- 172.21.15.145:0/3833647863 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f62d401a070 con 0x7f62ec0af130 2024-05-09T04:34:12.933 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:12.934+0000 7f62e3400640 1 --2- 172.21.15.145:0/3833647863 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f62c80762d0 0x7f62c8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:12.933 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:12.935+0000 7f62e3400640 1 -- 172.21.15.145:0/3833647863 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f62d4096940 con 0x7f62ec0af130 2024-05-09T04:34:12.933 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:12.935+0000 7f62eb400640 1 --2- 172.21.15.145:0/3833647863 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f62c80762d0 0x7f62c8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:12.934 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:12.935+0000 7f62eb400640 1 --2- 172.21.15.145:0/3833647863 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f62c80762d0 0x7f62c8078790 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f62dc000f60 tx=0x7f62dc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:12.938 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:12.939+0000 7f62e3400640 1 -- 172.21.15.145:0/3833647863 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f62d4060290 con 0x7f62ec0af130 2024-05-09T04:34:13.186 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:13.187+0000 7f62f1200640 1 -- 172.21.15.145:0/3833647863 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f62ec167ba0 con 0x7f62ec0af130 2024-05-09T04:34:13.187 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:13.188+0000 7f62e3400640 1 -- 172.21.15.145:0/3833647863 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f62d405fb90 con 0x7f62ec0af130 2024-05-09T04:34:13.187 INFO:teuthology.orchestra.run.smithi145.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-09T04:34:13.190 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:13.191+0000 7f62f1200640 1 -- 172.21.15.145:0/3833647863 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f62c80762d0 msgr2=0x7f62c8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:13.190 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:13.191+0000 7f62f1200640 1 --2- 172.21.15.145:0/3833647863 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f62c80762d0 0x7f62c8078790 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f62dc000f60 tx=0x7f62dc009040 comp rx=0 tx=0).stop 2024-05-09T04:34:13.190 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:13.191+0000 7f62f1200640 1 -- 172.21.15.145:0/3833647863 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62ec0af130 msgr2=0x7f62ec15e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:13.190 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:13.191+0000 7f62f1200640 1 --2- 172.21.15.145:0/3833647863 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62ec0af130 0x7f62ec15e900 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f62d400c460 tx=0x7f62d400c920 comp rx=0 tx=0).stop 2024-05-09T04:34:13.190 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:13.191+0000 7f62f1200640 1 -- 172.21.15.145:0/3833647863 shutdown_connections 2024-05-09T04:34:13.190 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:13.191+0000 7f62f1200640 1 --2- 172.21.15.145:0/3833647863 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f62c80762d0 0x7f62c8078790 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:13.190 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:13.191+0000 7f62f1200640 1 --2- 172.21.15.145:0/3833647863 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62ec163270 0x7f62ec164a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:13.190 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:13.191+0000 7f62f1200640 1 --2- 172.21.15.145:0/3833647863 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62ec0af130 0x7f62ec15e900 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:13.190 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:13.191+0000 7f62f1200640 1 --2- 172.21.15.145:0/3833647863 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62ec06e320 0x7f62ec15e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:13.191 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:13.191+0000 7f62f1200640 1 -- 172.21.15.145:0/3833647863 >> 172.21.15.145:0/3833647863 conn(0x7f62ec06c8a0 msgr2=0x7f62ec166e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:13.191 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:13.191+0000 7f62f1200640 1 -- 172.21.15.145:0/3833647863 shutdown_connections 2024-05-09T04:34:13.191 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:13.191+0000 7f62f1200640 1 -- 172.21.15.145:0/3833647863 wait complete. 2024-05-09T04:34:13.209 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-09T04:34:13.209 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-09T04:34:13.209 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi192.front.sepia.ceph.com 2024-05-09T04:34:13.209 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-09T04:34:13.209 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-09T04:34:13.209 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-09T04:34:13.209 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-09T04:34:13.209 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-09T04:34:13.210 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-09T04:34:13.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:13.210 DEBUG:teuthology.orchestra.run.smithi192:> ip addr 2024-05-09T04:34:13.267 INFO:teuthology.orchestra.run.smithi192.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T04:34:13.267 INFO:teuthology.orchestra.run.smithi192.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T04:34:13.267 INFO:teuthology.orchestra.run.smithi192.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T04:34:13.268 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2024-05-09T04:34:13.268 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 ::1/128 scope host 2024-05-09T04:34:13.268 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2024-05-09T04:34:13.268 INFO:teuthology.orchestra.run.smithi192.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T04:34:13.268 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:d9:38:4e brd ff:ff:ff:ff:ff:ff 2024-05-09T04:34:13.268 INFO:teuthology.orchestra.run.smithi192.stdout: altname enp7s0f0 2024-05-09T04:34:13.268 INFO:teuthology.orchestra.run.smithi192.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T04:34:13.268 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:d9:38:4f brd ff:ff:ff:ff:ff:ff 2024-05-09T04:34:13.268 INFO:teuthology.orchestra.run.smithi192.stdout: altname enp7s0f1 2024-05-09T04:34:13.268 INFO:teuthology.orchestra.run.smithi192.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T04:34:13.268 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:88:80:48 brd ff:ff:ff:ff:ff:ff 2024-05-09T04:34:13.268 INFO:teuthology.orchestra.run.smithi192.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-09T04:34:13.269 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:88:80:49 brd ff:ff:ff:ff:ff:ff 2024-05-09T04:34:13.269 INFO:teuthology.orchestra.run.smithi192.stdout: inet 172.21.15.192/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-09T04:34:13.269 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft 42592sec preferred_lft 42592sec 2024-05-09T04:34:13.269 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::ec4:7aff:fe88:8049/64 scope link 2024-05-09T04:34:13.269 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2024-05-09T04:34:13.270 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-09T04:34:13.270 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T04:34:13.270 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2024-05-09T04:34:13.270 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link add name ceph-brx type bridge 2024-05-09T04:34:13.270 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip addr flush dev ceph-brx 2024-05-09T04:34:13.270 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link set ceph-brx up 2024-05-09T04:34:13.270 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-09T04:34:13.270 DEBUG:teuthology.orchestra.run.smithi192:> ') 2024-05-09T04:34:13.372 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:34:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T04:34:13.730 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:34:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T04:34:13.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:13.743 DEBUG:teuthology.orchestra.run.smithi192:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-09T04:34:13.808 INFO:teuthology.orchestra.run.smithi192.stdout:1 2024-05-09T04:34:13.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:13.812 DEBUG:teuthology.orchestra.run.smithi192:> route 2024-05-09T04:34:13.863 INFO:teuthology.orchestra.run.smithi192.stdout:Kernel IP routing table 2024-05-09T04:34:13.863 INFO:teuthology.orchestra.run.smithi192.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-09T04:34:13.863 INFO:teuthology.orchestra.run.smithi192.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-09T04:34:13.863 INFO:teuthology.orchestra.run.smithi192.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-09T04:34:13.863 INFO:teuthology.orchestra.run.smithi192.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-09T04:34:13.863 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T04:34:13.863 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2024-05-09T04:34:13.863 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-09T04:34:13.863 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-09T04:34:13.864 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-09T04:34:13.864 DEBUG:teuthology.orchestra.run.smithi192:> ') 2024-05-09T04:34:13.957 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:34:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T04:34:14.595 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:34:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T04:34:14.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:14.603 DEBUG:teuthology.orchestra.run.smithi192:> ip netns list 2024-05-09T04:34:14.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:14.652 DEBUG:teuthology.orchestra.run.smithi192:> ip netns list-id 2024-05-09T04:34:14.693 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T04:34:14.693 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2024-05-09T04:34:14.693 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-09T04:34:14.693 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-05-09T04:34:14.694 DEBUG:teuthology.orchestra.run.smithi192:> ') 2024-05-09T04:34:14.831 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:34:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T04:34:14.897 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:34:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T04:34:14.906 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-05-09T04:34:14.906 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T04:34:14.906 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2024-05-09T04:34:14.906 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-05-09T04:34:14.906 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-09T04:34:14.906 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-05-09T04:34:14.906 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-05-09T04:34:14.907 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-05-09T04:34:14.907 DEBUG:teuthology.orchestra.run.smithi192:> ') 2024-05-09T04:34:14.997 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:34:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T04:34:15.179 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:34:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T04:34:15.188 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T04:34:15.188 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2024-05-09T04:34:15.188 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link set brx.0 up 2024-05-09T04:34:15.188 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link set dev brx.0 master ceph-brx 2024-05-09T04:34:15.188 DEBUG:teuthology.orchestra.run.smithi192:> ') 2024-05-09T04:34:15.274 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:34:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T04:34:15.363 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:34:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T04:34:15.372 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-05-09T04:34:15.372 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T04:34:15.372 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-09T04:34:15.416 INFO:teuthology.orchestra.run.smithi192.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-09T04:34:15.416 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T04:34:15.416 DEBUG:teuthology.orchestra.run.smithi192:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-09T04:34:15.505 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2024-05-09T04:34:15.585 DEBUG:teuthology.orchestra.run.smithi192:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T04:34:15.637 INFO:teuthology.orchestra.run.smithi192.stdout:/proc 2024-05-09T04:34:15.637 INFO:teuthology.orchestra.run.smithi192.stdout:/sys 2024-05-09T04:34:15.637 INFO:teuthology.orchestra.run.smithi192.stdout:/dev 2024-05-09T04:34:15.637 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/security 2024-05-09T04:34:15.637 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/shm 2024-05-09T04:34:15.638 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/pts 2024-05-09T04:34:15.638 INFO:teuthology.orchestra.run.smithi192.stdout:/run 2024-05-09T04:34:15.638 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/cgroup 2024-05-09T04:34:15.638 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/pstore 2024-05-09T04:34:15.638 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/bpf 2024-05-09T04:34:15.638 INFO:teuthology.orchestra.run.smithi192.stdout:/ 2024-05-09T04:34:15.638 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/selinux 2024-05-09T04:34:15.638 INFO:teuthology.orchestra.run.smithi192.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T04:34:15.638 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/hugepages 2024-05-09T04:34:15.638 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/mqueue 2024-05-09T04:34:15.638 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/debug 2024-05-09T04:34:15.638 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/tracing 2024-05-09T04:34:15.638 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/fuse/connections 2024-05-09T04:34:15.638 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T04:34:15.638 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T04:34:15.639 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/config 2024-05-09T04:34:15.639 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T04:34:15.639 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T04:34:15.639 INFO:teuthology.orchestra.run.smithi192.stdout:/run/user/1000 2024-05-09T04:34:15.639 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph 2024-05-09T04:34:15.639 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-4 2024-05-09T04:34:15.639 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-5 2024-05-09T04:34:15.639 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-6 2024-05-09T04:34:15.639 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-7 2024-05-09T04:34:15.639 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns 2024-05-09T04:34:15.639 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-09T04:34:15.639 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-09T04:34:15.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:15.640 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2024-05-09T04:34:15.686 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-09T04:34:15.686 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-05-09T04:34:15.735 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2024-05-09T04:34:15.806 DEBUG:teuthology.orchestra.run.smithi192:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T04:34:15.848 INFO:teuthology.orchestra.run.smithi192.stdout:/proc 2024-05-09T04:34:15.849 INFO:teuthology.orchestra.run.smithi192.stdout:/sys 2024-05-09T04:34:15.849 INFO:teuthology.orchestra.run.smithi192.stdout:/dev 2024-05-09T04:34:15.849 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/security 2024-05-09T04:34:15.849 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/shm 2024-05-09T04:34:15.849 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/pts 2024-05-09T04:34:15.849 INFO:teuthology.orchestra.run.smithi192.stdout:/run 2024-05-09T04:34:15.849 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/cgroup 2024-05-09T04:34:15.849 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/pstore 2024-05-09T04:34:15.849 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/bpf 2024-05-09T04:34:15.849 INFO:teuthology.orchestra.run.smithi192.stdout:/ 2024-05-09T04:34:15.849 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/selinux 2024-05-09T04:34:15.849 INFO:teuthology.orchestra.run.smithi192.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T04:34:15.849 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/hugepages 2024-05-09T04:34:15.849 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/mqueue 2024-05-09T04:34:15.849 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/debug 2024-05-09T04:34:15.850 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/tracing 2024-05-09T04:34:15.850 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/fuse/connections 2024-05-09T04:34:15.850 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T04:34:15.850 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T04:34:15.850 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/config 2024-05-09T04:34:15.850 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T04:34:15.850 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T04:34:15.850 INFO:teuthology.orchestra.run.smithi192.stdout:/run/user/1000 2024-05-09T04:34:15.850 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph 2024-05-09T04:34:15.850 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-4 2024-05-09T04:34:15.850 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-5 2024-05-09T04:34:15.850 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-6 2024-05-09T04:34:15.850 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-7 2024-05-09T04:34:15.850 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns 2024-05-09T04:34:15.850 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-09T04:34:15.851 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-09T04:34:15.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:15.852 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2024-05-09T04:34:15.946 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:2024-05-09T04:34:15.946+0000 7faf26cf5480 -1 init, newargv = 0x55ec51bd94a0 newargc=15 2024-05-09T04:34:15.946 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[40596]: starting ceph client 2024-05-09T04:34:15.957 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[40596]: starting fuse 2024-05-09T04:34:17.040 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2024-05-09T04:34:17.129 DEBUG:teuthology.orchestra.run.smithi192:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T04:34:17.187 INFO:teuthology.orchestra.run.smithi192.stdout:/proc 2024-05-09T04:34:17.187 INFO:teuthology.orchestra.run.smithi192.stdout:/sys 2024-05-09T04:34:17.187 INFO:teuthology.orchestra.run.smithi192.stdout:/dev 2024-05-09T04:34:17.187 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/security 2024-05-09T04:34:17.187 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/shm 2024-05-09T04:34:17.187 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/pts 2024-05-09T04:34:17.187 INFO:teuthology.orchestra.run.smithi192.stdout:/run 2024-05-09T04:34:17.187 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/cgroup 2024-05-09T04:34:17.188 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/pstore 2024-05-09T04:34:17.188 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/bpf 2024-05-09T04:34:17.188 INFO:teuthology.orchestra.run.smithi192.stdout:/ 2024-05-09T04:34:17.188 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/selinux 2024-05-09T04:34:17.188 INFO:teuthology.orchestra.run.smithi192.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T04:34:17.188 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/hugepages 2024-05-09T04:34:17.188 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/mqueue 2024-05-09T04:34:17.188 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/debug 2024-05-09T04:34:17.188 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/tracing 2024-05-09T04:34:17.188 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/fuse/connections 2024-05-09T04:34:17.188 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T04:34:17.188 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T04:34:17.188 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/config 2024-05-09T04:34:17.188 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T04:34:17.188 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T04:34:17.189 INFO:teuthology.orchestra.run.smithi192.stdout:/run/user/1000 2024-05-09T04:34:17.189 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph 2024-05-09T04:34:17.189 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-4 2024-05-09T04:34:17.189 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-5 2024-05-09T04:34:17.189 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-6 2024-05-09T04:34:17.189 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-7 2024-05-09T04:34:17.189 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns 2024-05-09T04:34:17.189 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-09T04:34:17.189 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-09T04:34:17.189 INFO:teuthology.orchestra.run.smithi192.stdout:/home/ubuntu/cephtest/mnt.1 2024-05-09T04:34:17.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:17.190 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2024-05-09T04:34:17.242 INFO:teuthology.orchestra.run.smithi192.stdout:40 2024-05-09T04:34:17.242 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2024-05-09T04:34:17.242 DEBUG:teuthology.orchestra.run.smithi192:> sudo stdin-killer -- python3 -c ' 2024-05-09T04:34:17.242 DEBUG:teuthology.orchestra.run.smithi192:> import glob 2024-05-09T04:34:17.242 DEBUG:teuthology.orchestra.run.smithi192:> import re 2024-05-09T04:34:17.242 DEBUG:teuthology.orchestra.run.smithi192:> import os 2024-05-09T04:34:17.242 DEBUG:teuthology.orchestra.run.smithi192:> import subprocess 2024-05-09T04:34:17.243 DEBUG:teuthology.orchestra.run.smithi192:> 2024-05-09T04:34:17.243 DEBUG:teuthology.orchestra.run.smithi192:> def _find_admin_socket(client_name): 2024-05-09T04:34:17.243 DEBUG:teuthology.orchestra.run.smithi192:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-05-09T04:34:17.243 DEBUG:teuthology.orchestra.run.smithi192:> files = glob.glob(asok_path) 2024-05-09T04:34:17.243 DEBUG:teuthology.orchestra.run.smithi192:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-05-09T04:34:17.243 DEBUG:teuthology.orchestra.run.smithi192:> 2024-05-09T04:34:17.243 DEBUG:teuthology.orchestra.run.smithi192:> # Given a non-glob path, it better be there 2024-05-09T04:34:17.243 DEBUG:teuthology.orchestra.run.smithi192:> if "*" not in asok_path: 2024-05-09T04:34:17.243 DEBUG:teuthology.orchestra.run.smithi192:> assert(len(files) == 1) 2024-05-09T04:34:17.243 DEBUG:teuthology.orchestra.run.smithi192:> return files[0] 2024-05-09T04:34:17.243 DEBUG:teuthology.orchestra.run.smithi192:> 2024-05-09T04:34:17.243 DEBUG:teuthology.orchestra.run.smithi192:> for f in files: 2024-05-09T04:34:17.243 DEBUG:teuthology.orchestra.run.smithi192:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-09T04:34:17.243 DEBUG:teuthology.orchestra.run.smithi192:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-09T04:34:17.243 DEBUG:teuthology.orchestra.run.smithi192:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-09T04:34:17.243 DEBUG:teuthology.orchestra.run.smithi192:> contents = proc_f.read() 2024-05-09T04:34:17.243 DEBUG:teuthology.orchestra.run.smithi192:> if mountpoint in contents: 2024-05-09T04:34:17.244 DEBUG:teuthology.orchestra.run.smithi192:> return f 2024-05-09T04:34:17.244 DEBUG:teuthology.orchestra.run.smithi192:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-09T04:34:17.244 DEBUG:teuthology.orchestra.run.smithi192:> 2024-05-09T04:34:17.244 DEBUG:teuthology.orchestra.run.smithi192:> print(_find_admin_socket("client.1")) 2024-05-09T04:34:17.244 DEBUG:teuthology.orchestra.run.smithi192:> ' 2024-05-09T04:34:17.413 INFO:teuthology.orchestra.run.smithi192.stdout:/var/run/ceph/ceph-client.1.40596.asok 2024-05-09T04:34:17.417 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:34:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T04:34:17.430 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.40596.asok 2024-05-09T04:34:17.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:17.430 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.40596.asok status 2024-05-09T04:34:17.660 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-09T04:34:17.660 INFO:teuthology.orchestra.run.smithi192.stdout: "metadata": { 2024-05-09T04:34:17.660 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_sha1": "165a15e51d08073b5317c443415b0d8a7141b1ab", 2024-05-09T04:34:17.660 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 19.0.0-3568-g165a15e5 (165a15e51d08073b5317c443415b0d8a7141b1ab) squid (dev)", 2024-05-09T04:34:17.660 INFO:teuthology.orchestra.run.smithi192.stdout: "entity_id": "1", 2024-05-09T04:34:17.660 INFO:teuthology.orchestra.run.smithi192.stdout: "hostname": "smithi192", 2024-05-09T04:34:17.660 INFO:teuthology.orchestra.run.smithi192.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-05-09T04:34:17.660 INFO:teuthology.orchestra.run.smithi192.stdout: "pid": "40596", 2024-05-09T04:34:17.661 INFO:teuthology.orchestra.run.smithi192.stdout: "root": "/" 2024-05-09T04:34:17.661 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-09T04:34:17.661 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_count": 0, 2024-05-09T04:34:17.661 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_pinned_count": 0, 2024-05-09T04:34:17.661 INFO:teuthology.orchestra.run.smithi192.stdout: "id": 4421, 2024-05-09T04:34:17.661 INFO:teuthology.orchestra.run.smithi192.stdout: "inst": { 2024-05-09T04:34:17.661 INFO:teuthology.orchestra.run.smithi192.stdout: "name": { 2024-05-09T04:34:17.661 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "client", 2024-05-09T04:34:17.661 INFO:teuthology.orchestra.run.smithi192.stdout: "num": 4421 2024-05-09T04:34:17.661 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-09T04:34:17.661 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2024-05-09T04:34:17.661 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2024-05-09T04:34:17.661 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "192.168.0.1:0", 2024-05-09T04:34:17.661 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 1809114053 2024-05-09T04:34:17.661 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-09T04:34:17.662 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-09T04:34:17.662 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2024-05-09T04:34:17.662 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2024-05-09T04:34:17.662 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "192.168.0.1:0", 2024-05-09T04:34:17.662 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 1809114053 2024-05-09T04:34:17.662 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-09T04:34:17.662 INFO:teuthology.orchestra.run.smithi192.stdout: "inst_str": "client.4421 192.168.0.1:0/1809114053", 2024-05-09T04:34:17.662 INFO:teuthology.orchestra.run.smithi192.stdout: "addr_str": "192.168.0.1:0/1809114053", 2024-05-09T04:34:17.662 INFO:teuthology.orchestra.run.smithi192.stdout: "inode_count": 1, 2024-05-09T04:34:17.662 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_epoch": 8, 2024-05-09T04:34:17.662 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch": 20, 2024-05-09T04:34:17.662 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch_barrier": 0, 2024-05-09T04:34:17.662 INFO:teuthology.orchestra.run.smithi192.stdout: "blocklisted": false, 2024-05-09T04:34:17.662 INFO:teuthology.orchestra.run.smithi192.stdout: "fs_name": "cephfs" 2024-05-09T04:34:17.663 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-09T04:34:17.675 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-09T04:34:17.884 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.885+0000 7fa4b5400640 1 Processor -- start 2024-05-09T04:34:17.885 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.885+0000 7fa4b5400640 1 -- start start 2024-05-09T04:34:17.886 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.886+0000 7fa4b5400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa4b006e860 0x7fa4b006ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:17.886 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.886+0000 7fa4b5400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa4b015d780 0x7fa4b0160c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:17.886 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.886+0000 7fa4b5400640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4b006f180 con 0x7fa4b00ce620 2024-05-09T04:34:17.886 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.886+0000 7fa4b5400640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fa4b01615c0 con 0x7fa4b015d780 2024-05-09T04:34:17.886 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.886+0000 7fa4b5400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa4b01618a0 con 0x7fa4b006e860 2024-05-09T04:34:17.886 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.886+0000 7fa4af200640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7fa4b00ce620 0x7fa4b00ceab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:34132/0 (socket says 172.21.15.145:34132) 2024-05-09T04:34:17.886 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.886+0000 7fa4af200640 1 -- 172.21.15.145:0/2636045477 learned_addr learned my addr 172.21.15.145:0/2636045477 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:34:17.888 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.887+0000 7fa4a5e00640 1 --2- 172.21.15.145:0/2636045477 >> v2:172.21.15.192:3300/0 conn(0x7fa4b015d780 0x7fa4b0160c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:17.888 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.887+0000 7fa4ae800640 1 --2- 172.21.15.145:0/2636045477 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa4b006e860 0x7fa4b006ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:17.888 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.887+0000 7fa4ade00640 1 -- 172.21.15.145:0/2636045477 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 488042690 0 0) 0x7fa4b006f180 con 0x7fa4b00ce620 2024-05-09T04:34:17.888 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.887+0000 7fa4ade00640 1 -- 172.21.15.145:0/2636045477 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa494003520 con 0x7fa4b00ce620 2024-05-09T04:34:17.888 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.887+0000 7fa4ade00640 1 -- 172.21.15.145:0/2636045477 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 676704136 0 0) 0x7fa494003520 con 0x7fa4b00ce620 2024-05-09T04:34:17.889 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.887+0000 7fa4ade00640 1 -- 172.21.15.145:0/2636045477 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa4b006e860 msgr2=0x7fa4b006ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:17.889 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.887+0000 7fa4ade00640 1 --2- 172.21.15.145:0/2636045477 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa4b006e860 0x7fa4b006ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:17.889 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.887+0000 7fa4ade00640 1 -- 172.21.15.145:0/2636045477 >> v2:172.21.15.192:3300/0 conn(0x7fa4b015d780 msgr2=0x7fa4b0160c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:17.889 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.887+0000 7fa4ade00640 1 --2- 172.21.15.145:0/2636045477 >> v2:172.21.15.192:3300/0 conn(0x7fa4b015d780 0x7fa4b0160c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:17.889 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.887+0000 7fa4ade00640 1 -- 172.21.15.145:0/2636045477 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4b00b4110 con 0x7fa4b00ce620 2024-05-09T04:34:17.889 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.887+0000 7fa4ade00640 1 -- 172.21.15.145:0/2636045477 <== mon.0 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3474413261 0 0) 0x7fa4a80016f0 con 0x7fa4b00ce620 2024-05-09T04:34:17.889 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.887+0000 7fa4ade00640 1 -- 172.21.15.145:0/2636045477 >> v1:172.21.15.145:6789/0 conn(0x7fa4b00ce620 legacy=0x7fa4b00ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:17.889 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.887+0000 7fa4ade00640 1 --2- 172.21.15.145:0/2636045477 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa4b006e860 0x7fa494003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:17.889 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.887+0000 7fa4ade00640 1 --2- 172.21.15.145:0/2636045477 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa494003fd0 0x7fa494004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:17.889 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.887+0000 7fa4ade00640 1 --2- 172.21.15.145:0/2636045477 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa494004d40 0x7fa4940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:17.889 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.887+0000 7fa4ade00640 1 -- 172.21.15.145:0/2636045477 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b01615c0 con 0x7fa494003fd0 2024-05-09T04:34:17.889 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.887+0000 7fa4ade00640 1 -- 172.21.15.145:0/2636045477 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b01618a0 con 0x7fa494004d40 2024-05-09T04:34:17.889 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.887+0000 7fa4ade00640 1 -- 172.21.15.145:0/2636045477 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa49400bac0 con 0x7fa4b006e860 2024-05-09T04:34:17.889 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.888+0000 7fa4a5e00640 1 --2- 172.21.15.145:0/2636045477 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa494003fd0 0x7fa494004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:17.889 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.888+0000 7fa4af200640 1 --2- 172.21.15.145:0/2636045477 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa494004d40 0x7fa4940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:17.890 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.888+0000 7fa4ae800640 1 --2- 172.21.15.145:0/2636045477 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa4b006e860 0x7fa494003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:17.890 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.888+0000 7fa4a5e00640 1 -- 172.21.15.145:0/2636045477 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa4b006e860 msgr2=0x7fa494003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:17.890 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.888+0000 7fa4a5e00640 1 --2- 172.21.15.145:0/2636045477 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa4b006e860 0x7fa494003a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:17.890 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.888+0000 7fa4a5e00640 1 -- 172.21.15.145:0/2636045477 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa494004d40 msgr2=0x7fa4940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:17.890 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.888+0000 7fa4a5e00640 1 --2- 172.21.15.145:0/2636045477 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa494004d40 0x7fa4940051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:17.890 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.888+0000 7fa4a5e00640 1 -- 172.21.15.145:0/2636045477 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa494003520 con 0x7fa494003fd0 2024-05-09T04:34:17.890 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.888+0000 7fa4af200640 1 --2- 172.21.15.145:0/2636045477 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa494004d40 0x7fa4940051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:34:17.891 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.889+0000 7fa4a5e00640 1 --2- 172.21.15.145:0/2636045477 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa494003fd0 0x7fa494004800 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa498009fc0 tx=0x7fa494005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5a7da6de74e3552 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:17.891 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.889+0000 7fa4ade00640 1 -- 172.21.15.145:0/2636045477 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa498009c10 con 0x7fa494003fd0 2024-05-09T04:34:17.891 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.889+0000 7fa4ade00640 1 -- 172.21.15.145:0/2636045477 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa498036070 con 0x7fa494003fd0 2024-05-09T04:34:17.891 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.889+0000 7fa4ade00640 1 -- 172.21.15.145:0/2636045477 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49802d070 con 0x7fa494003fd0 2024-05-09T04:34:17.891 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.890+0000 7fa4b5400640 1 -- 172.21.15.145:0/2636045477 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa494003fd0 msgr2=0x7fa494004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:17.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.890+0000 7fa4b5400640 1 --2- 172.21.15.145:0/2636045477 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa494003fd0 0x7fa494004800 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa498009fc0 tx=0x7fa494005830 comp rx=0 tx=0).stop 2024-05-09T04:34:17.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.890+0000 7fa4af200640 1 -- 172.21.15.145:0/2636045477 reap_dead start 2024-05-09T04:34:17.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.890+0000 7fa4b5400640 1 -- 172.21.15.145:0/2636045477 shutdown_connections 2024-05-09T04:34:17.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.890+0000 7fa4b5400640 1 -- 172.21.15.145:0/2636045477 >> 172.21.15.145:0/2636045477 conn(0x7fa4b006c870 msgr2=0x7fa4b006cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:17.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.890+0000 7fa4b5400640 1 -- 172.21.15.145:0/2636045477 shutdown_connections 2024-05-09T04:34:17.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.890+0000 7fa4b5400640 1 -- 172.21.15.145:0/2636045477 wait complete. 2024-05-09T04:34:17.893 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.892+0000 7fa4b5400640 1 Processor -- start 2024-05-09T04:34:17.893 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.892+0000 7fa4b5400640 1 -- start start 2024-05-09T04:34:17.893 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.893+0000 7fa4b5400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa494003fd0 0x7fa4b013bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:17.893 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.893+0000 7fa4b5400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa4b0139ee0 0x7fa4b013a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:17.893 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.893+0000 7fa4b5400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa4b0150bc0 0x7fa4b0151040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:17.893 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.893+0000 7fa4b5400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b0146d80 con 0x7fa494003fd0 2024-05-09T04:34:17.893 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.893+0000 7fa4b5400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b0146c00 con 0x7fa4b0150bc0 2024-05-09T04:34:17.893 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.893+0000 7fa4b5400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa4b0161190 con 0x7fa4b0139ee0 2024-05-09T04:34:17.893 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.893+0000 7fa4ae800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa494003fd0 0x7fa4b013bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:17.893 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.893+0000 7fa4ae800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa494003fd0 0x7fa4b013bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:42276/0 (socket says 172.21.15.145:42276) 2024-05-09T04:34:17.894 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.893+0000 7fa4ae800640 1 -- 172.21.15.145:0/314523657 learned_addr learned my addr 172.21.15.145:0/314523657 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:17.895 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.894+0000 7fa4ae800640 1 -- 172.21.15.145:0/314523657 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa4b0139ee0 msgr2=0x7fa4b013a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:17.895 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.894+0000 7fa4ae800640 1 --2- 172.21.15.145:0/314523657 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa4b0139ee0 0x7fa4b013a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:17.895 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.894+0000 7fa4af200640 1 --2- 172.21.15.145:0/314523657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa4b0150bc0 0x7fa4b0151040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:17.895 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.894+0000 7fa4ae800640 1 -- 172.21.15.145:0/314523657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa4b0150bc0 msgr2=0x7fa4b0151040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:17.895 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.894+0000 7fa4ae800640 1 --2- 172.21.15.145:0/314523657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa4b0150bc0 0x7fa4b0151040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:17.895 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.894+0000 7fa4ae800640 1 -- 172.21.15.145:0/314523657 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4b013c410 con 0x7fa494003fd0 2024-05-09T04:34:17.895 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.894+0000 7fa4ae800640 1 --2- 172.21.15.145:0/314523657 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa494003fd0 0x7fa4b013bed0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa4a000e7e0 tx=0x7fa4a000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:17.895 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.894+0000 7fa4a7400640 1 -- 172.21.15.145:0/314523657 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a0017070 con 0x7fa494003fd0 2024-05-09T04:34:17.895 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.894+0000 7fa4a7400640 1 -- 172.21.15.145:0/314523657 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4a000c6d0 con 0x7fa494003fd0 2024-05-09T04:34:17.895 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.894+0000 7fa4a7400640 1 -- 172.21.15.145:0/314523657 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a000f040 con 0x7fa494003fd0 2024-05-09T04:34:17.895 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.894+0000 7fa4b5400640 1 -- 172.21.15.145:0/314523657 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4b013c6d0 con 0x7fa494003fd0 2024-05-09T04:34:17.895 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.894+0000 7fa4b5400640 1 -- 172.21.15.145:0/314523657 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4b013cc90 con 0x7fa494003fd0 2024-05-09T04:34:17.896 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.895+0000 7fa4b5400640 1 -- 172.21.15.145:0/314523657 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4b006a310 con 0x7fa494003fd0 2024-05-09T04:34:17.896 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.897+0000 7fa4a7400640 1 -- 172.21.15.145:0/314523657 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa4a001a070 con 0x7fa494003fd0 2024-05-09T04:34:17.897 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.898+0000 7fa4a7400640 1 --2- 172.21.15.145:0/314523657 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fa48c0762d0 0x7fa48c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:17.897 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.898+0000 7fa4a7400640 1 -- 172.21.15.145:0/314523657 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa4a00971a0 con 0x7fa494003fd0 2024-05-09T04:34:17.897 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.898+0000 7fa4a5e00640 1 --2- 172.21.15.145:0/314523657 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fa48c0762d0 0x7fa48c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:17.898 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.899+0000 7fa4a5e00640 1 --2- 172.21.15.145:0/314523657 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fa48c0762d0 0x7fa48c078790 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa498035000 tx=0x7fa49802d000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:17.902 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:17.902+0000 7fa4a7400640 1 -- 172.21.15.145:0/314523657 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa4a0060af0 con 0x7fa494003fd0 2024-05-09T04:34:18.156 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:18.156+0000 7fa4b5400640 1 -- 172.21.15.145:0/314523657 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa4b00b0680 con 0x7fa494003fd0 2024-05-09T04:34:18.156 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:18.157+0000 7fa4a7400640 1 -- 172.21.15.145:0/314523657 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7fa4a00603f0 con 0x7fa494003fd0 2024-05-09T04:34:18.157 INFO:teuthology.orchestra.run.smithi145.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-09T04:34:18.160 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:18.159+0000 7fa4b5400640 1 -- 172.21.15.145:0/314523657 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fa48c0762d0 msgr2=0x7fa48c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:18.160 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:18.159+0000 7fa4b5400640 1 --2- 172.21.15.145:0/314523657 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fa48c0762d0 0x7fa48c078790 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa498035000 tx=0x7fa49802d000 comp rx=0 tx=0).stop 2024-05-09T04:34:18.160 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:18.159+0000 7fa4b5400640 1 -- 172.21.15.145:0/314523657 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa494003fd0 msgr2=0x7fa4b013bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:18.160 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:18.159+0000 7fa4b5400640 1 --2- 172.21.15.145:0/314523657 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa494003fd0 0x7fa4b013bed0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa4a000e7e0 tx=0x7fa4a000eca0 comp rx=0 tx=0).stop 2024-05-09T04:34:18.160 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:18.160+0000 7fa4b5400640 1 -- 172.21.15.145:0/314523657 shutdown_connections 2024-05-09T04:34:18.160 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:18.160+0000 7fa4b5400640 1 --2- 172.21.15.145:0/314523657 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fa48c0762d0 0x7fa48c078790 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:18.160 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:18.160+0000 7fa4b5400640 1 --2- 172.21.15.145:0/314523657 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa4b0150bc0 0x7fa4b0151040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:18.160 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:18.160+0000 7fa4b5400640 1 --2- 172.21.15.145:0/314523657 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa4b0139ee0 0x7fa4b013a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:18.160 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:18.160+0000 7fa4b5400640 1 --2- 172.21.15.145:0/314523657 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa494003fd0 0x7fa4b013bed0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:18.160 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:18.160+0000 7fa4b5400640 1 -- 172.21.15.145:0/314523657 >> 172.21.15.145:0/314523657 conn(0x7fa4b006c870 msgr2=0x7fa4b00b4760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:18.160 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:18.160+0000 7fa4b5400640 1 -- 172.21.15.145:0/314523657 shutdown_connections 2024-05-09T04:34:18.160 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:18.160+0000 7fa4b5400640 1 -- 172.21.15.145:0/314523657 wait complete. 2024-05-09T04:34:18.180 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-09T04:34:18.180 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-09T04:34:18.180 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi192.front.sepia.ceph.com 2024-05-09T04:34:18.180 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-05-09T04:34:18.180 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-05-09T04:34:18.180 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-09T04:34:18.180 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-09T04:34:18.180 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-09T04:34:18.180 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-05-09T04:34:18.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:18.181 DEBUG:teuthology.orchestra.run.smithi192:> ip addr 2024-05-09T04:34:18.238 INFO:teuthology.orchestra.run.smithi192.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T04:34:18.238 INFO:teuthology.orchestra.run.smithi192.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T04:34:18.238 INFO:teuthology.orchestra.run.smithi192.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T04:34:18.238 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2024-05-09T04:34:18.238 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 ::1/128 scope host 2024-05-09T04:34:18.238 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2024-05-09T04:34:18.238 INFO:teuthology.orchestra.run.smithi192.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T04:34:18.238 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:d9:38:4e brd ff:ff:ff:ff:ff:ff 2024-05-09T04:34:18.238 INFO:teuthology.orchestra.run.smithi192.stdout: altname enp7s0f0 2024-05-09T04:34:18.238 INFO:teuthology.orchestra.run.smithi192.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T04:34:18.239 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:d9:38:4f brd ff:ff:ff:ff:ff:ff 2024-05-09T04:34:18.239 INFO:teuthology.orchestra.run.smithi192.stdout: altname enp7s0f1 2024-05-09T04:34:18.239 INFO:teuthology.orchestra.run.smithi192.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T04:34:18.239 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:88:80:48 brd ff:ff:ff:ff:ff:ff 2024-05-09T04:34:18.239 INFO:teuthology.orchestra.run.smithi192.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-09T04:34:18.239 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:88:80:49 brd ff:ff:ff:ff:ff:ff 2024-05-09T04:34:18.239 INFO:teuthology.orchestra.run.smithi192.stdout: inet 172.21.15.192/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-09T04:34:18.239 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft 42587sec preferred_lft 42587sec 2024-05-09T04:34:18.239 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::ec4:7aff:fe88:8049/64 scope link 2024-05-09T04:34:18.239 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2024-05-09T04:34:18.239 INFO:teuthology.orchestra.run.smithi192.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-09T04:34:18.239 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 7e:17:2e:99:fa:e6 brd ff:ff:ff:ff:ff:ff 2024-05-09T04:34:18.239 INFO:teuthology.orchestra.run.smithi192.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-09T04:34:18.239 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2024-05-09T04:34:18.239 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::6887:29ff:fec1:7e22/64 scope link 2024-05-09T04:34:18.240 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2024-05-09T04:34:18.240 INFO:teuthology.orchestra.run.smithi192.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-09T04:34:18.240 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 7e:17:2e:99:fa:e6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-09T04:34:18.240 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::7c17:2eff:fe99:fae6/64 scope link 2024-05-09T04:34:18.240 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2024-05-09T04:34:18.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:18.241 DEBUG:teuthology.orchestra.run.smithi192:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-09T04:34:18.321 INFO:teuthology.orchestra.run.smithi192.stdout:1 2024-05-09T04:34:18.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:18.326 DEBUG:teuthology.orchestra.run.smithi192:> route 2024-05-09T04:34:18.377 INFO:teuthology.orchestra.run.smithi192.stdout:Kernel IP routing table 2024-05-09T04:34:18.377 INFO:teuthology.orchestra.run.smithi192.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-09T04:34:18.377 INFO:teuthology.orchestra.run.smithi192.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-09T04:34:18.377 INFO:teuthology.orchestra.run.smithi192.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-09T04:34:18.377 INFO:teuthology.orchestra.run.smithi192.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-09T04:34:18.378 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T04:34:18.378 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2024-05-09T04:34:18.378 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-09T04:34:18.378 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-09T04:34:18.378 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-09T04:34:18.378 DEBUG:teuthology.orchestra.run.smithi192:> ') 2024-05-09T04:34:18.480 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:34:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T04:34:18.575 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:34:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T04:34:18.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:18.585 DEBUG:teuthology.orchestra.run.smithi192:> ip netns list 2024-05-09T04:34:18.632 INFO:teuthology.orchestra.run.smithi192.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-09T04:34:18.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:18.632 DEBUG:teuthology.orchestra.run.smithi192:> ip netns list-id 2024-05-09T04:34:18.682 INFO:teuthology.orchestra.run.smithi192.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-05-09T04:34:18.682 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T04:34:18.683 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2024-05-09T04:34:18.683 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-09T04:34:18.683 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-05-09T04:34:18.683 DEBUG:teuthology.orchestra.run.smithi192:> ') 2024-05-09T04:34:18.789 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:34:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T04:34:18.857 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:34:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T04:34:18.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:18.867 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-05-09T04:34:18.942 INFO:teuthology.orchestra.run.smithi192.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T04:34:18.942 INFO:teuthology.orchestra.run.smithi192.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T04:34:18.942 INFO:teuthology.orchestra.run.smithi192.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T04:34:18.942 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2024-05-09T04:34:18.942 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 ::1/128 scope host 2024-05-09T04:34:18.942 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2024-05-09T04:34:18.942 INFO:teuthology.orchestra.run.smithi192.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-09T04:34:18.942 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 8a:ec:6c:bc:bc:c5 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-09T04:34:18.943 INFO:teuthology.orchestra.run.smithi192.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-09T04:34:18.943 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2024-05-09T04:34:18.943 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::88ec:6cff:febc:bcc5/64 scope link 2024-05-09T04:34:18.943 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2024-05-09T04:34:18.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:18.946 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-05-09T04:34:19.028 INFO:teuthology.orchestra.run.smithi192.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T04:34:19.029 INFO:teuthology.orchestra.run.smithi192.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T04:34:19.029 INFO:teuthology.orchestra.run.smithi192.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T04:34:19.029 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2024-05-09T04:34:19.029 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 ::1/128 scope host 2024-05-09T04:34:19.029 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2024-05-09T04:34:19.029 INFO:teuthology.orchestra.run.smithi192.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-09T04:34:19.029 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 8a:ec:6c:bc:bc:c5 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-05-09T04:34:19.029 INFO:teuthology.orchestra.run.smithi192.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-05-09T04:34:19.029 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2024-05-09T04:34:19.029 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::88ec:6cff:febc:bcc5/64 scope link 2024-05-09T04:34:19.029 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2024-05-09T04:34:19.033 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-05-09T04:34:19.033 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T04:34:19.033 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2024-05-09T04:34:19.033 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-05-09T04:34:19.033 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-05-09T04:34:19.033 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-05-09T04:34:19.034 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-05-09T04:34:19.034 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-05-09T04:34:19.034 DEBUG:teuthology.orchestra.run.smithi192:> ') 2024-05-09T04:34:19.137 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:34:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T04:34:19.291 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:34:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T04:34:19.300 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T04:34:19.300 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2024-05-09T04:34:19.300 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link set brx.1 up 2024-05-09T04:34:19.300 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link set dev brx.1 master ceph-brx 2024-05-09T04:34:19.300 DEBUG:teuthology.orchestra.run.smithi192:> ') 2024-05-09T04:34:19.391 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:34:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T04:34:19.486 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:34:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T04:34:19.495 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-05-09T04:34:19.495 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T04:34:19.495 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-05-09T04:34:19.544 INFO:teuthology.orchestra.run.smithi192.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-05-09T04:34:19.544 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T04:34:19.544 DEBUG:teuthology.orchestra.run.smithi192:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-05-09T04:34:19.639 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2024-05-09T04:34:19.717 DEBUG:teuthology.orchestra.run.smithi192:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T04:34:19.774 INFO:teuthology.orchestra.run.smithi192.stdout:/proc 2024-05-09T04:34:19.774 INFO:teuthology.orchestra.run.smithi192.stdout:/sys 2024-05-09T04:34:19.774 INFO:teuthology.orchestra.run.smithi192.stdout:/dev 2024-05-09T04:34:19.774 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/security 2024-05-09T04:34:19.774 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/shm 2024-05-09T04:34:19.774 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/pts 2024-05-09T04:34:19.774 INFO:teuthology.orchestra.run.smithi192.stdout:/run 2024-05-09T04:34:19.774 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/cgroup 2024-05-09T04:34:19.774 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/pstore 2024-05-09T04:34:19.774 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/bpf 2024-05-09T04:34:19.775 INFO:teuthology.orchestra.run.smithi192.stdout:/ 2024-05-09T04:34:19.775 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/selinux 2024-05-09T04:34:19.775 INFO:teuthology.orchestra.run.smithi192.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T04:34:19.775 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/hugepages 2024-05-09T04:34:19.775 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/mqueue 2024-05-09T04:34:19.775 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/debug 2024-05-09T04:34:19.775 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/tracing 2024-05-09T04:34:19.775 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/fuse/connections 2024-05-09T04:34:19.775 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T04:34:19.775 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T04:34:19.775 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/config 2024-05-09T04:34:19.775 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T04:34:19.775 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T04:34:19.775 INFO:teuthology.orchestra.run.smithi192.stdout:/run/user/1000 2024-05-09T04:34:19.776 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph 2024-05-09T04:34:19.776 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-4 2024-05-09T04:34:19.776 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-5 2024-05-09T04:34:19.776 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-6 2024-05-09T04:34:19.776 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-7 2024-05-09T04:34:19.776 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns 2024-05-09T04:34:19.776 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-09T04:34:19.776 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-09T04:34:19.776 INFO:teuthology.orchestra.run.smithi192.stdout:/home/ubuntu/cephtest/mnt.1 2024-05-09T04:34:19.776 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-09T04:34:19.776 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-09T04:34:19.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:19.777 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2024-05-09T04:34:19.833 INFO:teuthology.orchestra.run.smithi192.stdout:40 2024-05-09T04:34:19.833 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [40] 2024-05-09T04:34:19.833 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-05-09T04:34:19.838 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2024-05-09T04:34:19.908 DEBUG:teuthology.orchestra.run.smithi192:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T04:34:19.948 INFO:teuthology.orchestra.run.smithi192.stdout:/proc 2024-05-09T04:34:19.949 INFO:teuthology.orchestra.run.smithi192.stdout:/sys 2024-05-09T04:34:19.949 INFO:teuthology.orchestra.run.smithi192.stdout:/dev 2024-05-09T04:34:19.949 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/security 2024-05-09T04:34:19.949 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/shm 2024-05-09T04:34:19.949 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/pts 2024-05-09T04:34:19.949 INFO:teuthology.orchestra.run.smithi192.stdout:/run 2024-05-09T04:34:19.949 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/cgroup 2024-05-09T04:34:19.949 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/pstore 2024-05-09T04:34:19.949 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/bpf 2024-05-09T04:34:19.949 INFO:teuthology.orchestra.run.smithi192.stdout:/ 2024-05-09T04:34:19.949 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/selinux 2024-05-09T04:34:19.949 INFO:teuthology.orchestra.run.smithi192.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T04:34:19.949 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/hugepages 2024-05-09T04:34:19.950 INFO:teuthology.orchestra.run.smithi192.stdout:/dev/mqueue 2024-05-09T04:34:19.950 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/debug 2024-05-09T04:34:19.950 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/tracing 2024-05-09T04:34:19.950 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/fs/fuse/connections 2024-05-09T04:34:19.950 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T04:34:19.950 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T04:34:19.950 INFO:teuthology.orchestra.run.smithi192.stdout:/sys/kernel/config 2024-05-09T04:34:19.950 INFO:teuthology.orchestra.run.smithi192.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T04:34:19.950 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T04:34:19.950 INFO:teuthology.orchestra.run.smithi192.stdout:/run/user/1000 2024-05-09T04:34:19.950 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph 2024-05-09T04:34:19.950 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-4 2024-05-09T04:34:19.950 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-5 2024-05-09T04:34:19.950 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-6 2024-05-09T04:34:19.951 INFO:teuthology.orchestra.run.smithi192.stdout:/var/lib/ceph/osd/ceph-7 2024-05-09T04:34:19.951 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns 2024-05-09T04:34:19.951 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-09T04:34:19.951 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-09T04:34:19.951 INFO:teuthology.orchestra.run.smithi192.stdout:/home/ubuntu/cephtest/mnt.1 2024-05-09T04:34:19.951 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-09T04:34:19.951 INFO:teuthology.orchestra.run.smithi192.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-09T04:34:19.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:19.952 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2024-05-09T04:34:20.007 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi192.stderr:2024-05-09T04:34:20.007+0000 7efd43fdc480 -1 init, newargv = 0x55b003f41170 newargc=15 2024-05-09T04:34:20.007 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi192.stderr:ceph-fuse[41116]: starting ceph client 2024-05-09T04:34:20.017 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi192.stderr:ceph-fuse[41116]: starting fuse 2024-05-09T04:34:20.036 INFO:teuthology.orchestra.run.smithi192.stdout:40 2024-05-09T04:34:20.036 INFO:teuthology.orchestra.run.smithi192.stdout:43 2024-05-09T04:34:20.037 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40, 43] 2024-05-09T04:34:20.037 DEBUG:teuthology.orchestra.run.smithi192:> sudo stdin-killer -- python3 -c ' 2024-05-09T04:34:20.037 DEBUG:teuthology.orchestra.run.smithi192:> import glob 2024-05-09T04:34:20.037 DEBUG:teuthology.orchestra.run.smithi192:> import re 2024-05-09T04:34:20.037 DEBUG:teuthology.orchestra.run.smithi192:> import os 2024-05-09T04:34:20.037 DEBUG:teuthology.orchestra.run.smithi192:> import subprocess 2024-05-09T04:34:20.037 DEBUG:teuthology.orchestra.run.smithi192:> 2024-05-09T04:34:20.037 DEBUG:teuthology.orchestra.run.smithi192:> def _find_admin_socket(client_name): 2024-05-09T04:34:20.037 DEBUG:teuthology.orchestra.run.smithi192:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-05-09T04:34:20.037 DEBUG:teuthology.orchestra.run.smithi192:> files = glob.glob(asok_path) 2024-05-09T04:34:20.037 DEBUG:teuthology.orchestra.run.smithi192:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-05-09T04:34:20.038 DEBUG:teuthology.orchestra.run.smithi192:> 2024-05-09T04:34:20.038 DEBUG:teuthology.orchestra.run.smithi192:> # Given a non-glob path, it better be there 2024-05-09T04:34:20.038 DEBUG:teuthology.orchestra.run.smithi192:> if "*" not in asok_path: 2024-05-09T04:34:20.038 DEBUG:teuthology.orchestra.run.smithi192:> assert(len(files) == 1) 2024-05-09T04:34:20.038 DEBUG:teuthology.orchestra.run.smithi192:> return files[0] 2024-05-09T04:34:20.038 DEBUG:teuthology.orchestra.run.smithi192:> 2024-05-09T04:34:20.038 DEBUG:teuthology.orchestra.run.smithi192:> for f in files: 2024-05-09T04:34:20.038 DEBUG:teuthology.orchestra.run.smithi192:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-09T04:34:20.038 DEBUG:teuthology.orchestra.run.smithi192:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-09T04:34:20.038 DEBUG:teuthology.orchestra.run.smithi192:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-09T04:34:20.038 DEBUG:teuthology.orchestra.run.smithi192:> contents = proc_f.read() 2024-05-09T04:34:20.038 DEBUG:teuthology.orchestra.run.smithi192:> if mountpoint in contents: 2024-05-09T04:34:20.038 DEBUG:teuthology.orchestra.run.smithi192:> return f 2024-05-09T04:34:20.038 DEBUG:teuthology.orchestra.run.smithi192:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-09T04:34:20.038 DEBUG:teuthology.orchestra.run.smithi192:> 2024-05-09T04:34:20.039 DEBUG:teuthology.orchestra.run.smithi192:> print(_find_admin_socket("client.3")) 2024-05-09T04:34:20.039 DEBUG:teuthology.orchestra.run.smithi192:> ' 2024-05-09T04:34:20.226 INFO:teuthology.orchestra.run.smithi192.stdout:/var/run/ceph/ceph-client.3.41116.asok 2024-05-09T04:34:20.230 INFO:teuthology.orchestra.run.smithi192.stderr:2024-05-09T04:34:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T04:34:20.243 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.41116.asok 2024-05-09T04:34:20.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:20.244 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.41116.asok status 2024-05-09T04:34:20.473 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-05-09T04:34:20.474 INFO:teuthology.orchestra.run.smithi192.stdout: "metadata": { 2024-05-09T04:34:20.474 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_sha1": "165a15e51d08073b5317c443415b0d8a7141b1ab", 2024-05-09T04:34:20.474 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 19.0.0-3568-g165a15e5 (165a15e51d08073b5317c443415b0d8a7141b1ab) squid (dev)", 2024-05-09T04:34:20.474 INFO:teuthology.orchestra.run.smithi192.stdout: "entity_id": "3", 2024-05-09T04:34:20.474 INFO:teuthology.orchestra.run.smithi192.stdout: "hostname": "smithi192", 2024-05-09T04:34:20.474 INFO:teuthology.orchestra.run.smithi192.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-05-09T04:34:20.474 INFO:teuthology.orchestra.run.smithi192.stdout: "pid": "41116", 2024-05-09T04:34:20.475 INFO:teuthology.orchestra.run.smithi192.stdout: "root": "/" 2024-05-09T04:34:20.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-09T04:34:20.475 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_count": 0, 2024-05-09T04:34:20.475 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_pinned_count": 0, 2024-05-09T04:34:20.475 INFO:teuthology.orchestra.run.smithi192.stdout: "id": 4549, 2024-05-09T04:34:20.475 INFO:teuthology.orchestra.run.smithi192.stdout: "inst": { 2024-05-09T04:34:20.475 INFO:teuthology.orchestra.run.smithi192.stdout: "name": { 2024-05-09T04:34:20.475 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "client", 2024-05-09T04:34:20.475 INFO:teuthology.orchestra.run.smithi192.stdout: "num": 4549 2024-05-09T04:34:20.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-09T04:34:20.475 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2024-05-09T04:34:20.476 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2024-05-09T04:34:20.476 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "192.168.0.2:0", 2024-05-09T04:34:20.476 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 947037602 2024-05-09T04:34:20.476 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-05-09T04:34:20.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-09T04:34:20.476 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2024-05-09T04:34:20.476 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2024-05-09T04:34:20.476 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "192.168.0.2:0", 2024-05-09T04:34:20.476 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 947037602 2024-05-09T04:34:20.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-05-09T04:34:20.476 INFO:teuthology.orchestra.run.smithi192.stdout: "inst_str": "client.4549 192.168.0.2:0/947037602", 2024-05-09T04:34:20.476 INFO:teuthology.orchestra.run.smithi192.stdout: "addr_str": "192.168.0.2:0/947037602", 2024-05-09T04:34:20.476 INFO:teuthology.orchestra.run.smithi192.stdout: "inode_count": 1, 2024-05-09T04:34:20.476 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_epoch": 8, 2024-05-09T04:34:20.476 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch": 20, 2024-05-09T04:34:20.476 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch_barrier": 0, 2024-05-09T04:34:20.477 INFO:teuthology.orchestra.run.smithi192.stdout: "blocklisted": false, 2024-05-09T04:34:20.477 INFO:teuthology.orchestra.run.smithi192.stdout: "fs_name": "cephfs" 2024-05-09T04:34:20.477 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-05-09T04:34:20.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:20.488 DEBUG:teuthology.orchestra.run.smithi145:> stat --file-system '--printf=%T 2024-05-09T04:34:20.488 DEBUG:teuthology.orchestra.run.smithi145:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-09T04:34:20.548 INFO:teuthology.orchestra.run.smithi145.stdout:fuseblk 2024-05-09T04:34:20.548 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-09T04:34:20.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:20.548 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-09T04:34:20.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:20.636 DEBUG:teuthology.orchestra.run.smithi145:> stat --file-system '--printf=%T 2024-05-09T04:34:20.636 DEBUG:teuthology.orchestra.run.smithi145:> ' -- /home/ubuntu/cephtest/mnt.2 2024-05-09T04:34:20.690 INFO:teuthology.orchestra.run.smithi145.stdout:fuseblk 2024-05-09T04:34:20.690 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-05-09T04:34:20.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:20.691 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-05-09T04:34:20.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:20.770 DEBUG:teuthology.orchestra.run.smithi192:> stat --file-system '--printf=%T 2024-05-09T04:34:20.770 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1 2024-05-09T04:34:20.828 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2024-05-09T04:34:20.828 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-05-09T04:34:20.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:20.829 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-09T04:34:20.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:20.912 DEBUG:teuthology.orchestra.run.smithi192:> stat --file-system '--printf=%T 2024-05-09T04:34:20.912 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.3 2024-05-09T04:34:20.964 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2024-05-09T04:34:20.965 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-05-09T04:34:20.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:20.965 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-05-09T04:34:21.037 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-05-09T04:34:21.062 INFO:tasks.cephfs_test_runner:Starting test: test_backtrace (tasks.cephfs.test_backtrace.TestBacktrace) That the 'parent' 'layout' and 'symlink' xattrs on the head objects of files 2024-05-09T04:34:21.062 INFO:tasks.cephfs_test_runner:test_backtrace (tasks.cephfs.test_backtrace.TestBacktrace) 2024-05-09T04:34:21.063 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T04:34:21.274 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.274+0000 7f2a0a800640 1 Processor -- start 2024-05-09T04:34:21.274 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.274+0000 7f2a0a800640 1 -- start start 2024-05-09T04:34:21.276 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.276+0000 7f2a0a800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2a0415fbc0 0x7f2a04142200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:21.276 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.276+0000 7f2a0a800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f2a040726b0 0x7f2a04072b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:21.276 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.276+0000 7f2a0a800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a0413f340 con 0x7f2a0415ffa0 2024-05-09T04:34:21.276 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.276+0000 7f2a0a800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f2a04072220 con 0x7f2a040726b0 2024-05-09T04:34:21.276 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.276+0000 7f2a0a800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f2a040cffe0 con 0x7f2a0415fbc0 2024-05-09T04:34:21.276 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.276+0000 7f2a02a00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f2a0415ffa0 0x7f2a040cf8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:44834/0 (socket says 172.21.15.145:44834) 2024-05-09T04:34:21.277 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.276+0000 7f2a02a00640 1 -- 172.21.15.145:0/2468686405 learned_addr learned my addr 172.21.15.145:0/2468686405 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:34:21.277 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.276+0000 7f2a03400640 1 --2- 172.21.15.145:0/2468686405 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2a0415fbc0 0x7f2a04142200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:21.277 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.276+0000 7f2a03e00640 1 --2- 172.21.15.145:0/2468686405 >> v2:172.21.15.192:3300/0 conn(0x7f2a040726b0 0x7f2a04072b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:21.277 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.276+0000 7f2a02000640 1 -- 172.21.15.145:0/2468686405 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2134469258 0 0) 0x7f29f0001110 con 0x7f2a0415ffa0 2024-05-09T04:34:21.277 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.277+0000 7f2a02000640 1 -- 172.21.15.145:0/2468686405 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f29e8003520 con 0x7f2a0415ffa0 2024-05-09T04:34:21.279 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.277+0000 7f2a02000640 1 -- 172.21.15.145:0/2468686405 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 573935574 0 0) 0x7f29e8003520 con 0x7f2a0415ffa0 2024-05-09T04:34:21.279 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.277+0000 7f2a02000640 1 -- 172.21.15.145:0/2468686405 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2a0415fbc0 msgr2=0x7f2a04142200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:21.279 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.277+0000 7f2a02000640 1 --2- 172.21.15.145:0/2468686405 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2a0415fbc0 0x7f2a04142200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:21.279 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.277+0000 7f2a02000640 1 -- 172.21.15.145:0/2468686405 >> v2:172.21.15.192:3300/0 conn(0x7f2a040726b0 msgr2=0x7f2a04072b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:21.279 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.277+0000 7f2a02000640 1 --2- 172.21.15.145:0/2468686405 >> v2:172.21.15.192:3300/0 conn(0x7f2a040726b0 0x7f2a04072b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:21.279 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.277+0000 7f2a02000640 1 -- 172.21.15.145:0/2468686405 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a040d02b0 con 0x7f2a0415ffa0 2024-05-09T04:34:21.279 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.277+0000 7f2a02000640 1 -- 172.21.15.145:0/2468686405 <== mon.0 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3474413261 0 0) 0x7f29f00018c0 con 0x7f2a0415ffa0 2024-05-09T04:34:21.279 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.277+0000 7f2a02000640 1 -- 172.21.15.145:0/2468686405 >> v1:172.21.15.145:6789/0 conn(0x7f2a0415ffa0 legacy=0x7f2a040cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:21.280 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.277+0000 7f2a02000640 1 --2- 172.21.15.145:0/2468686405 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2a0415fbc0 0x7f29e8003a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:21.280 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.277+0000 7f2a02000640 1 --2- 172.21.15.145:0/2468686405 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29e8004020 0x7f29e8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:21.280 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.277+0000 7f2a02000640 1 --2- 172.21.15.145:0/2468686405 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f29e8004d40 0x7f29e80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:21.280 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.277+0000 7f2a02000640 1 -- 172.21.15.145:0/2468686405 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2a04072220 con 0x7f29e8004d40 2024-05-09T04:34:21.280 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.277+0000 7f2a02000640 1 -- 172.21.15.145:0/2468686405 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2a040cffe0 con 0x7f29e8004020 2024-05-09T04:34:21.280 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.277+0000 7f2a02000640 1 -- 172.21.15.145:0/2468686405 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f29e800baf0 con 0x7f2a0415fbc0 2024-05-09T04:34:21.280 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.277+0000 7f2a03e00640 1 --2- 172.21.15.145:0/2468686405 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f29e8004d40 0x7f29e80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:21.280 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.278+0000 7f2a02a00640 1 --2- 172.21.15.145:0/2468686405 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29e8004020 0x7f29e8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:21.280 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.278+0000 7f2a03400640 1 --2- 172.21.15.145:0/2468686405 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2a0415fbc0 0x7f29e8003a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:21.280 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.278+0000 7f2a03e00640 1 -- 172.21.15.145:0/2468686405 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2a0415fbc0 msgr2=0x7f29e8003a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:21.280 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.278+0000 7f2a03e00640 1 --2- 172.21.15.145:0/2468686405 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2a0415fbc0 0x7f29e8003a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:21.280 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.278+0000 7f2a03e00640 1 -- 172.21.15.145:0/2468686405 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29e8004020 msgr2=0x7f29e8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:21.280 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.278+0000 7f2a03e00640 1 --2- 172.21.15.145:0/2468686405 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29e8004020 0x7f29e8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:21.280 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.278+0000 7f2a03e00640 1 -- 172.21.15.145:0/2468686405 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29e8003520 con 0x7f29e8004d40 2024-05-09T04:34:21.280 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.278+0000 7f2a02a00640 1 --2- 172.21.15.145:0/2468686405 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29e8004020 0x7f29e8004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:34:21.281 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.279+0000 7f2a03e00640 1 --2- 172.21.15.145:0/2468686405 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f29e8004d40 0x7f29e80051c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f29fc009fc0 tx=0x7f29e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29284c04848cf451 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:21.281 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.279+0000 7f2a02000640 1 -- 172.21.15.145:0/2468686405 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29fc009c10 con 0x7f29e8004d40 2024-05-09T04:34:21.281 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.279+0000 7f2a02000640 1 -- 172.21.15.145:0/2468686405 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29fc036070 con 0x7f29e8004d40 2024-05-09T04:34:21.281 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.279+0000 7f2a02000640 1 -- 172.21.15.145:0/2468686405 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29fc02d070 con 0x7f29e8004d40 2024-05-09T04:34:21.282 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.280+0000 7f2a0a800640 1 -- 172.21.15.145:0/2468686405 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f29e8004d40 msgr2=0x7f29e80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:21.282 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.280+0000 7f2a0a800640 1 --2- 172.21.15.145:0/2468686405 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f29e8004d40 0x7f29e80051c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f29fc009fc0 tx=0x7f29e8005830 comp rx=0 tx=0).stop 2024-05-09T04:34:21.282 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.280+0000 7f2a03e00640 1 -- 172.21.15.145:0/2468686405 reap_dead start 2024-05-09T04:34:21.282 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.280+0000 7f2a0a800640 1 -- 172.21.15.145:0/2468686405 shutdown_connections 2024-05-09T04:34:21.282 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.280+0000 7f2a0a800640 1 -- 172.21.15.145:0/2468686405 >> 172.21.15.145:0/2468686405 conn(0x7f2a0406c870 msgr2=0x7f2a0406cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:21.282 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.280+0000 7f2a0a800640 1 -- 172.21.15.145:0/2468686405 shutdown_connections 2024-05-09T04:34:21.282 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.280+0000 7f2a0a800640 1 -- 172.21.15.145:0/2468686405 wait complete. 2024-05-09T04:34:21.282 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.282+0000 7f2a0a800640 1 Processor -- start 2024-05-09T04:34:21.282 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.282+0000 7f2a0a800640 1 -- start start 2024-05-09T04:34:21.283 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.283+0000 7f2a0a800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2a04072d00 0x7f2a04136f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:21.283 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.283+0000 7f2a0a800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a040b5190 0x7f2a041374b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:21.283 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.283+0000 7f2a0a800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2a040730e0 0x7f2a041379f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:21.283 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.283+0000 7f2a0a800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2a040d0b50 con 0x7f2a04072d00 2024-05-09T04:34:21.283 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.283+0000 7f2a0a800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2a040d0870 con 0x7f2a040b5190 2024-05-09T04:34:21.283 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.283+0000 7f2a0a800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f2a04160400 con 0x7f2a040730e0 2024-05-09T04:34:21.285 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.284+0000 7f2a03400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2a04072d00 0x7f2a04136f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:21.285 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.284+0000 7f2a03400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2a04072d00 0x7f2a04136f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:40432/0 (socket says 172.21.15.145:40432) 2024-05-09T04:34:21.285 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.284+0000 7f2a03400640 1 -- 172.21.15.145:0/2537697159 learned_addr learned my addr 172.21.15.145:0/2537697159 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:21.285 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.284+0000 7f2a02a00640 1 --2- 172.21.15.145:0/2537697159 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a040b5190 0x7f2a041374b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:21.285 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.284+0000 7f2a03e00640 1 --2- 172.21.15.145:0/2537697159 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2a040730e0 0x7f2a041379f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:21.285 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.284+0000 7f2a03400640 1 -- 172.21.15.145:0/2537697159 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2a040730e0 msgr2=0x7f2a041379f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:21.285 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.284+0000 7f2a03400640 1 --2- 172.21.15.145:0/2537697159 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2a040730e0 0x7f2a041379f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:21.285 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.284+0000 7f2a03400640 1 -- 172.21.15.145:0/2537697159 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a040b5190 msgr2=0x7f2a041374b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:21.286 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.284+0000 7f2a03400640 1 --2- 172.21.15.145:0/2537697159 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a040b5190 0x7f2a041374b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:21.286 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.284+0000 7f2a03400640 1 -- 172.21.15.145:0/2537697159 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a040b5700 con 0x7f2a04072d00 2024-05-09T04:34:21.286 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.285+0000 7f2a03400640 1 --2- 172.21.15.145:0/2537697159 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2a04072d00 0x7f2a04136f70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f29f4011630 tx=0x7f29f4011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:21.286 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.285+0000 7f29fb400640 1 -- 172.21.15.145:0/2537697159 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f400f920 con 0x7f2a04072d00 2024-05-09T04:34:21.286 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.285+0000 7f29fb400640 1 -- 172.21.15.145:0/2537697159 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29f400fac0 con 0x7f2a04072d00 2024-05-09T04:34:21.286 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.285+0000 7f29fb400640 1 -- 172.21.15.145:0/2537697159 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f400fe70 con 0x7f2a04072d00 2024-05-09T04:34:21.286 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.285+0000 7f2a0a800640 1 -- 172.21.15.145:0/2537697159 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a040b59e0 con 0x7f2a04072d00 2024-05-09T04:34:21.286 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.285+0000 7f2a0a800640 1 -- 172.21.15.145:0/2537697159 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a0415d7a0 con 0x7f2a04072d00 2024-05-09T04:34:21.287 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.286+0000 7f2a0a800640 1 -- 172.21.15.145:0/2537697159 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a0406a310 con 0x7f2a04072d00 2024-05-09T04:34:21.287 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.287+0000 7f29fb400640 1 -- 172.21.15.145:0/2537697159 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f29f4015460 con 0x7f2a04072d00 2024-05-09T04:34:21.288 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.288+0000 7f29fb400640 1 --2- 172.21.15.145:0/2537697159 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f29e4076300 0x7f29e40787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:21.288 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.288+0000 7f29fb400640 1 -- 172.21.15.145:0/2537697159 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f29f4097be0 con 0x7f2a04072d00 2024-05-09T04:34:21.288 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.289+0000 7f2a02a00640 1 --2- 172.21.15.145:0/2537697159 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f29e4076300 0x7f29e40787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:21.289 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.289+0000 7f2a02a00640 1 --2- 172.21.15.145:0/2537697159 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f29e4076300 0x7f29e40787c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2a0413be20 tx=0x7f29f0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:21.292 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.293+0000 7f29fb400640 1 -- 172.21.15.145:0/2537697159 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f29f4019030 con 0x7f2a04072d00 2024-05-09T04:34:21.501 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.501+0000 7f2a0a800640 1 -- 172.21.15.145:0/2537697159 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2a040ce390 con 0x7f2a04072d00 2024-05-09T04:34:21.504 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.505+0000 7f29fb400640 1 -- 172.21.15.145:0/2537697159 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16653 (secure 0 0 0) 0x7f29f40614a0 con 0x7f2a04072d00 2024-05-09T04:34:21.505 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:34:21.505 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":20,"fsid":"957ee6c1-073b-44cf-9b93-9491b3fdfc1d","created":"2024-05-09T04:33:26.544367+0000","modified":"2024-05-09T04:33:53.178020+0000","last_up_change":"2024-05-09T04:33:39.039318+0000","last_in_change":"2024-05-09T04:33:29.902361+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T04:33:39.795344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T04:33:49.651743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T04:33:50.596065+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22e550ad-847b-4952-80e6-63d859ed1a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6801","nonce":3182572912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6803","nonce":3182572912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6807","nonce":3182572912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6805","nonce":3182572912}]},"public_addr":"172.21.15.145:6801/3182572912","cluster_addr":"172.21.15.145:6803/3182572912","heartbeat_back_addr":"172.21.15.145:6807/3182572912","heartbeat_front_addr":"172.21.15.145:6805/3182572912","state":["exists","up"]},{"osd":1,"uuid":"0f6b9fca-0c41-4145-8d74-598ddb42f2a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6825","nonce":1845225555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6827","nonce":1845225555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6831","nonce":1845225555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6829","nonce":1845225555}]},"public_addr":"172.21.15.145:6825/1845225555","cluster_addr":"172.21.15.145:6827/1845225555","heartbeat_back_addr":"172.21.15.145:6831/1845225555","heartbeat_front_addr":"172.21.15.145:6829/1845225555","state":["exists","up"]},{"osd":2,"uuid":"b3b1f67e-5790-483f-8610-e05a5bce4c30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6817","nonce":2168958993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6819","nonce":2168958993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6823","nonce":2168958993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6821","nonce":2168958993}]},"public_addr":"172.21.15.145:6817/2168958993","cluster_addr":"172.21.15.145:6819/2168958993","heartbeat_back_addr":"172.21.15.145:6823/2168958993","heartbeat_front_addr":"172.21.15.145:6821/2168958993","state":["exists","up"]},{"osd":3,"uuid":"884787a5-e037-4d31-9aae-743030eba4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6809","nonce":337009826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6811","nonce":337009826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6815","nonce":337009826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6813","nonce":337009826}]},"public_addr":"172.21.15.145:6809/337009826","cluster_addr":"172.21.15.145:6811/337009826","heartbeat_back_addr":"172.21.15.145:6815/337009826","heartbeat_front_addr":"172.21.15.145:6813/337009826","state":["exists","up"]},{"osd":4,"uuid":"0759fc1e-c5fc-4a2f-86bb-e8eccd83f833","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6818","nonce":3117700995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6821","nonce":3117700995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6827","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6829","nonce":3117700995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6823","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6825","nonce":3117700995}]},"public_addr":"172.21.15.192:6818/3117700995","cluster_addr":"172.21.15.192:6821/3117700995","heartbeat_back_addr":"172.21.15.192:6829/3117700995","heartbeat_front_addr":"172.21.15.192:6825/3117700995","state":["exists","up"]},{"osd":5,"uuid":"9443cc22-9379-46e3-9173-98b592bf2c16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6801","nonce":1328373268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6803","nonce":1328373268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6807","nonce":1328373268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6805","nonce":1328373268}]},"public_addr":"172.21.15.192:6801/1328373268","cluster_addr":"172.21.15.192:6803/1328373268","heartbeat_back_addr":"172.21.15.192:6807/1328373268","heartbeat_front_addr":"172.21.15.192:6805/1328373268","state":["exists","up"]},{"osd":6,"uuid":"d05a3311-505a-45b5-85b9-aa3113e96987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6817","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6820","nonce":1868011057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6824","nonce":1868011057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6831","nonce":1868011057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6828","nonce":1868011057}]},"public_addr":"172.21.15.192:6820/1868011057","cluster_addr":"172.21.15.192:6824/1868011057","heartbeat_back_addr":"172.21.15.192:6831/1868011057","heartbeat_front_addr":"172.21.15.192:6828/1868011057","state":["exists","up"]},{"osd":7,"uuid":"c7c23dbc-2029-43ec-9f53-25f3d58019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6809","nonce":3368512157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6811","nonce":3368512157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6815","nonce":3368512157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6813","nonce":3368512157}]},"public_addr":"172.21.15.192:6809/3368512157","cluster_addr":"172.21.15.192:6811/3368512157","heartbeat_back_addr":"172.21.15.192:6815/3368512157","heartbeat_front_addr":"172.21.15.192:6813/3368512157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.091188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.032567+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:34.797468+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.228839+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.622400+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.473103+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.566092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.349395+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T04:34:21.507 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.507+0000 7f2a0a800640 1 -- 172.21.15.145:0/2537697159 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f29e4076300 msgr2=0x7f29e40787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:21.508 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.507+0000 7f2a0a800640 1 --2- 172.21.15.145:0/2537697159 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f29e4076300 0x7f29e40787c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2a0413be20 tx=0x7f29f0009040 comp rx=0 tx=0).stop 2024-05-09T04:34:21.508 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.507+0000 7f2a0a800640 1 -- 172.21.15.145:0/2537697159 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2a04072d00 msgr2=0x7f2a04136f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:21.508 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.507+0000 7f2a0a800640 1 --2- 172.21.15.145:0/2537697159 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2a04072d00 0x7f2a04136f70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f29f4011630 tx=0x7f29f4011af0 comp rx=0 tx=0).stop 2024-05-09T04:34:21.508 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.507+0000 7f2a0a800640 1 -- 172.21.15.145:0/2537697159 shutdown_connections 2024-05-09T04:34:21.508 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.507+0000 7f2a0a800640 1 --2- 172.21.15.145:0/2537697159 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f29e4076300 0x7f29e40787c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:21.508 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.508+0000 7f2a0a800640 1 --2- 172.21.15.145:0/2537697159 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2a040730e0 0x7f2a041379f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:21.508 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.508+0000 7f2a0a800640 1 --2- 172.21.15.145:0/2537697159 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a040b5190 0x7f2a041374b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:21.508 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.508+0000 7f2a0a800640 1 --2- 172.21.15.145:0/2537697159 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2a04072d00 0x7f2a04136f70 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:21.508 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.508+0000 7f2a0a800640 1 -- 172.21.15.145:0/2537697159 >> 172.21.15.145:0/2537697159 conn(0x7f2a0406c870 msgr2=0x7f2a040cf8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:21.508 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.508+0000 7f2a0a800640 1 -- 172.21.15.145:0/2537697159 shutdown_connections 2024-05-09T04:34:21.508 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.508+0000 7f2a0a800640 1 -- 172.21.15.145:0/2537697159 wait complete. 2024-05-09T04:34:21.528 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T04:33:39.795344+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T04:33:49.651743+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T04:33:50.596065+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T04:34:21.528 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T04:34:21.730 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.730+0000 7f3723e00640 1 Processor -- start 2024-05-09T04:34:21.730 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.730+0000 7f3723e00640 1 -- start start 2024-05-09T04:34:21.732 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.732+0000 7f3723e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f372415d0b0 0x7f37240b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:21.732 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.732+0000 7f3723e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f37240cfef0 0x7f37240d0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:21.732 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.732+0000 7f3723e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f372406fcf0 con 0x7f37240add40 2024-05-09T04:34:21.732 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.732+0000 7f3723e00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f3724072850 con 0x7f372415d0b0 2024-05-09T04:34:21.732 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.732+0000 7f3723e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f372406ff90 con 0x7f37240cfef0 2024-05-09T04:34:21.733 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.732+0000 7f3722a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f372415d0b0 0x7f37240b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:21.733 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.732+0000 7f3722a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f372415d0b0 0x7f37240b02b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:40436/0 (socket says 172.21.15.145:40436) 2024-05-09T04:34:21.733 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.732+0000 7f3722a00640 1 -- 172.21.15.145:0/4276829912 learned_addr learned my addr 172.21.15.145:0/4276829912 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:21.733 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.732+0000 7f3722a00640 1 -- 172.21.15.145:0/4276829912 >> v2:172.21.15.192:3300/0 conn(0x7f37240cfef0 msgr2=0x7f37240d0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:21.733 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.732+0000 7f3722a00640 1 --2- 172.21.15.145:0/4276829912 >> v2:172.21.15.192:3300/0 conn(0x7f37240cfef0 0x7f37240d0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:21.733 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.733+0000 7f3722a00640 1 -- 172.21.15.145:0/4276829912 >> v1:172.21.15.192:6789/0 conn(0x7f37240add40 legacy=0x7f3724072140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:21.733 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.733+0000 7f3722a00640 1 -- 172.21.15.145:0/4276829912 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3724070270 con 0x7f372415d0b0 2024-05-09T04:34:21.734 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.734+0000 7f3722a00640 1 --2- 172.21.15.145:0/4276829912 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f372415d0b0 0x7f37240b02b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3724070cb0 tx=0x7f371402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38227119d9d84cf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:21.734 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.734+0000 7f3721600640 1 -- 172.21.15.145:0/4276829912 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3714037070 con 0x7f372415d0b0 2024-05-09T04:34:21.734 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.734+0000 7f3721600640 1 -- 172.21.15.145:0/4276829912 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3714032440 con 0x7f372415d0b0 2024-05-09T04:34:21.734 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.734+0000 7f3721600640 1 -- 172.21.15.145:0/4276829912 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3714032810 con 0x7f372415d0b0 2024-05-09T04:34:21.734 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.734+0000 7f3723e00640 1 -- 172.21.15.145:0/4276829912 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f372415d0b0 msgr2=0x7f37240b02b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:21.735 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.734+0000 7f3723e00640 1 --2- 172.21.15.145:0/4276829912 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f372415d0b0 0x7f37240b02b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3724070cb0 tx=0x7f371402a140 comp rx=0 tx=0).stop 2024-05-09T04:34:21.735 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.734+0000 7f3723e00640 1 -- 172.21.15.145:0/4276829912 shutdown_connections 2024-05-09T04:34:21.735 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.734+0000 7f3723e00640 1 --2- 172.21.15.145:0/4276829912 >> v2:172.21.15.192:3300/0 conn(0x7f37240cfef0 0x7f37240d0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:21.735 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.734+0000 7f3723e00640 1 --2- 172.21.15.145:0/4276829912 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f372415d0b0 0x7f37240b02b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:21.735 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.734+0000 7f3723e00640 1 -- 172.21.15.145:0/4276829912 >> 172.21.15.145:0/4276829912 conn(0x7f37240b28f0 msgr2=0x7f37240b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:21.735 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.735+0000 7f3723e00640 1 -- 172.21.15.145:0/4276829912 shutdown_connections 2024-05-09T04:34:21.735 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.735+0000 7f3723e00640 1 -- 172.21.15.145:0/4276829912 wait complete. 2024-05-09T04:34:21.736 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.736+0000 7f3723e00640 1 Processor -- start 2024-05-09T04:34:21.736 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.736+0000 7f3723e00640 1 -- start start 2024-05-09T04:34:21.738 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.737+0000 7f3723e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f37240add40 0x7f3724147980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:21.738 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.737+0000 7f3723e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f37240cfef0 0x7f37240c5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:21.738 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.737+0000 7f3723e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f372415d0b0 0x7f37240c61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:21.738 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.737+0000 7f3723e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f372406eae0 con 0x7f37240add40 2024-05-09T04:34:21.738 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.737+0000 7f3723e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f372406e5c0 con 0x7f37240cfef0 2024-05-09T04:34:21.738 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.737+0000 7f3723e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f372406ec60 con 0x7f372415d0b0 2024-05-09T04:34:21.738 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.737+0000 7f3722a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f37240add40 0x7f3724147980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:21.738 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.738+0000 7f3722a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f37240add40 0x7f3724147980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:40440/0 (socket says 172.21.15.145:40440) 2024-05-09T04:34:21.738 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.738+0000 7f3722a00640 1 -- 172.21.15.145:0/1160565565 learned_addr learned my addr 172.21.15.145:0/1160565565 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:21.738 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.738+0000 7f3722a00640 1 -- 172.21.15.145:0/1160565565 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f372415d0b0 msgr2=0x7f37240c61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:21.739 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.738+0000 7f3722a00640 1 --2- 172.21.15.145:0/1160565565 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f372415d0b0 0x7f37240c61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:21.739 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.738+0000 7f3722a00640 1 -- 172.21.15.145:0/1160565565 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f37240cfef0 msgr2=0x7f37240c5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:21.739 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.738+0000 7f3722000640 1 --2- 172.21.15.145:0/1160565565 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f37240cfef0 0x7f37240c5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:21.739 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.738+0000 7f3722a00640 1 --2- 172.21.15.145:0/1160565565 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f37240cfef0 0x7f37240c5c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:21.739 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.738+0000 7f3722a00640 1 -- 172.21.15.145:0/1160565565 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3724148390 con 0x7f37240add40 2024-05-09T04:34:21.739 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.738+0000 7f3722a00640 1 --2- 172.21.15.145:0/1160565565 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f37240add40 0x7f3724147980 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3714000c00 tx=0x7f37140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:21.740 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.739+0000 7f371aa00640 1 -- 172.21.15.145:0/1160565565 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3714041070 con 0x7f37240add40 2024-05-09T04:34:21.740 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.739+0000 7f371aa00640 1 -- 172.21.15.145:0/1160565565 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f371403c950 con 0x7f37240add40 2024-05-09T04:34:21.740 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.739+0000 7f371aa00640 1 -- 172.21.15.145:0/1160565565 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3714037040 con 0x7f37240add40 2024-05-09T04:34:21.740 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.739+0000 7f3723e00640 1 -- 172.21.15.145:0/1160565565 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3724148670 con 0x7f37240add40 2024-05-09T04:34:21.740 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.739+0000 7f3723e00640 1 -- 172.21.15.145:0/1160565565 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3724148c30 con 0x7f37240add40 2024-05-09T04:34:21.740 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.740+0000 7f3723e00640 1 -- 172.21.15.145:0/1160565565 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f372406a310 con 0x7f37240add40 2024-05-09T04:34:21.741 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.741+0000 7f371aa00640 1 -- 172.21.15.145:0/1160565565 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3714044070 con 0x7f37240add40 2024-05-09T04:34:21.742 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.743+0000 7f371aa00640 1 --2- 172.21.15.145:0/1160565565 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f37040762d0 0x7f3704078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:21.742 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.743+0000 7f371aa00640 1 -- 172.21.15.145:0/1160565565 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f37140b8160 con 0x7f37240add40 2024-05-09T04:34:21.743 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.743+0000 7f3722000640 1 --2- 172.21.15.145:0/1160565565 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f37040762d0 0x7f3704078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:21.743 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.743+0000 7f3722000640 1 --2- 172.21.15.145:0/1160565565 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f37040762d0 0x7f3704078790 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f37241587d0 tx=0x7f3710009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:21.747 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.748+0000 7f371aa00640 1 -- 172.21.15.145:0/1160565565 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3714081ab0 con 0x7f37240add40 2024-05-09T04:34:21.956 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.956+0000 7f3723e00640 1 -- 172.21.15.145:0/1160565565 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3724072350 con 0x7f37240add40 2024-05-09T04:34:21.957 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.957+0000 7f371aa00640 1 -- 172.21.15.145:0/1160565565 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f37140813b0 con 0x7f37240add40 2024-05-09T04:34:21.957 INFO:teuthology.orchestra.run.smithi145.stdout:pg_num: 1 2024-05-09T04:34:21.960 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.960+0000 7f3723e00640 1 -- 172.21.15.145:0/1160565565 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f37040762d0 msgr2=0x7f3704078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:21.960 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.960+0000 7f3723e00640 1 --2- 172.21.15.145:0/1160565565 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f37040762d0 0x7f3704078790 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f37241587d0 tx=0x7f3710009040 comp rx=0 tx=0).stop 2024-05-09T04:34:21.960 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.960+0000 7f3723e00640 1 -- 172.21.15.145:0/1160565565 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f37240add40 msgr2=0x7f3724147980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:21.960 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.960+0000 7f3723e00640 1 --2- 172.21.15.145:0/1160565565 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f37240add40 0x7f3724147980 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3714000c00 tx=0x7f37140046c0 comp rx=0 tx=0).stop 2024-05-09T04:34:21.960 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.960+0000 7f3723e00640 1 -- 172.21.15.145:0/1160565565 shutdown_connections 2024-05-09T04:34:21.960 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.960+0000 7f3723e00640 1 --2- 172.21.15.145:0/1160565565 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f37040762d0 0x7f3704078790 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:21.961 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.960+0000 7f3723e00640 1 --2- 172.21.15.145:0/1160565565 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f372415d0b0 0x7f37240c61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:21.961 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.960+0000 7f3723e00640 1 --2- 172.21.15.145:0/1160565565 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f37240cfef0 0x7f37240c5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:21.961 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.960+0000 7f3723e00640 1 --2- 172.21.15.145:0/1160565565 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f37240add40 0x7f3724147980 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:21.961 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.960+0000 7f3723e00640 1 -- 172.21.15.145:0/1160565565 >> 172.21.15.145:0/1160565565 conn(0x7f37240b28f0 msgr2=0x7f37240b1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:21.961 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.960+0000 7f3723e00640 1 -- 172.21.15.145:0/1160565565 shutdown_connections 2024-05-09T04:34:21.961 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:21.961+0000 7f3723e00640 1 -- 172.21.15.145:0/1160565565 wait complete. 2024-05-09T04:34:21.980 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T04:34:22.180 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.180+0000 7f631aa00640 1 Processor -- start 2024-05-09T04:34:22.180 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.180+0000 7f631aa00640 1 -- start start 2024-05-09T04:34:22.182 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.182+0000 7f631aa00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f6314163380 0x7f6314073870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:22.182 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.182+0000 7f631aa00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6314160530 con 0x7f631415f140 2024-05-09T04:34:22.182 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.182+0000 7f631aa00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6314073fa0 con 0x7f6314070040 2024-05-09T04:34:22.182 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.182+0000 7f631aa00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f6314072350 con 0x7f6314163380 2024-05-09T04:34:22.184 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.182+0000 7f6319600640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f631415f140 0x7f6314072da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:44842/0 (socket says 172.21.15.145:44842) 2024-05-09T04:34:22.184 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.182+0000 7f6319600640 1 -- 172.21.15.145:0/442102510 learned_addr learned my addr 172.21.15.145:0/442102510 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:34:22.184 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.183+0000 7f631a000640 1 --2- 172.21.15.145:0/442102510 >> v2:172.21.15.192:3300/0 conn(0x7f6314163380 0x7f6314073870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:22.184 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.183+0000 7f6313e00640 1 -- 172.21.15.145:0/442102510 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3350099683 0 0) 0x7f6314160530 con 0x7f631415f140 2024-05-09T04:34:22.184 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.183+0000 7f6313e00640 1 -- 172.21.15.145:0/442102510 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f62f8003520 con 0x7f631415f140 2024-05-09T04:34:22.185 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.183+0000 7f6313e00640 1 -- 172.21.15.145:0/442102510 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4224177461 0 0) 0x7f62f8003520 con 0x7f631415f140 2024-05-09T04:34:22.185 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.183+0000 7f6313e00640 1 -- 172.21.15.145:0/442102510 >> v2:172.21.15.192:3300/0 conn(0x7f6314163380 msgr2=0x7f6314073870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:22.185 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.183+0000 7f6313e00640 1 --2- 172.21.15.145:0/442102510 >> v2:172.21.15.192:3300/0 conn(0x7f6314163380 0x7f6314073870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:22.185 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.183+0000 7f6313e00640 1 -- 172.21.15.145:0/442102510 >> v1:172.21.15.192:6789/0 conn(0x7f6314070040 legacy=0x7f6314162c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:22.185 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.183+0000 7f6313e00640 1 -- 172.21.15.145:0/442102510 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6314074280 con 0x7f631415f140 2024-05-09T04:34:22.185 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.183+0000 7f6313e00640 1 -- 172.21.15.145:0/442102510 <== mon.0 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3474413261 0 0) 0x7f6304001a70 con 0x7f631415f140 2024-05-09T04:34:22.185 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.183+0000 7f6313e00640 1 -- 172.21.15.145:0/442102510 >> v1:172.21.15.145:6789/0 conn(0x7f631415f140 legacy=0x7f6314072da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:22.185 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.183+0000 7f6313e00640 1 --2- 172.21.15.145:0/442102510 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62f8003aa0 0x7f62f8003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:22.185 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.184+0000 7f6313e00640 1 --2- 172.21.15.145:0/442102510 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62f8004800 0x7f62f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:22.185 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.184+0000 7f6313e00640 1 --2- 172.21.15.145:0/442102510 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62f80053c0 0x7f62f800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:22.185 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.184+0000 7f6313e00640 1 -- 172.21.15.145:0/442102510 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6314072350 con 0x7f62f8003aa0 2024-05-09T04:34:22.185 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.184+0000 7f6313e00640 1 -- 172.21.15.145:0/442102510 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f62f800c2a0 con 0x7f62f8004800 2024-05-09T04:34:22.185 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.184+0000 7f6313e00640 1 -- 172.21.15.145:0/442102510 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f62f800c4a0 con 0x7f62f80053c0 2024-05-09T04:34:22.185 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.184+0000 7f6319600640 1 --2- 172.21.15.145:0/442102510 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62f8003aa0 0x7f62f8003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:22.185 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.184+0000 7f6318c00640 1 --2- 172.21.15.145:0/442102510 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62f8004800 0x7f62f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:22.186 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.184+0000 7f6319600640 1 -- 172.21.15.145:0/442102510 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62f80053c0 msgr2=0x7f62f800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:22.186 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.184+0000 7f6319600640 1 --2- 172.21.15.145:0/442102510 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62f80053c0 0x7f62f800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:22.186 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.184+0000 7f6319600640 1 -- 172.21.15.145:0/442102510 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62f8004800 msgr2=0x7f62f8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:22.186 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.184+0000 7f6319600640 1 --2- 172.21.15.145:0/442102510 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62f8004800 0x7f62f8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:22.186 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.184+0000 7f6319600640 1 -- 172.21.15.145:0/442102510 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62f8003520 con 0x7f62f8003aa0 2024-05-09T04:34:22.186 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.184+0000 7f631a000640 1 -- 172.21.15.145:0/442102510 reap_dead start 2024-05-09T04:34:22.187 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.185+0000 7f6319600640 1 --2- 172.21.15.145:0/442102510 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62f8003aa0 0x7f62f8003ec0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f630400a800 tx=0x7f62f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=283cabe2ad7f438b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:22.187 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.185+0000 7f6313e00640 1 -- 172.21.15.145:0/442102510 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630400a450 con 0x7f62f8003aa0 2024-05-09T04:34:22.187 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.186+0000 7f6313e00640 1 -- 172.21.15.145:0/442102510 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f630402f070 con 0x7f62f8003aa0 2024-05-09T04:34:22.187 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.186+0000 7f6313e00640 1 -- 172.21.15.145:0/442102510 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630402bbe0 con 0x7f62f8003aa0 2024-05-09T04:34:22.187 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.186+0000 7f631aa00640 1 -- 172.21.15.145:0/442102510 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62f8003aa0 msgr2=0x7f62f8003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:22.187 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.186+0000 7f631aa00640 1 --2- 172.21.15.145:0/442102510 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62f8003aa0 0x7f62f8003ec0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f630400a800 tx=0x7f62f8005830 comp rx=0 tx=0).stop 2024-05-09T04:34:22.187 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.186+0000 7f631aa00640 1 -- 172.21.15.145:0/442102510 shutdown_connections 2024-05-09T04:34:22.188 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.186+0000 7f631aa00640 1 --2- 172.21.15.145:0/442102510 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62f8003aa0 0x7f62f8003ec0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:22.188 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.186+0000 7f631aa00640 1 -- 172.21.15.145:0/442102510 >> 172.21.15.145:0/442102510 conn(0x7f63140b28f0 msgr2=0x7f63140b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:22.188 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.186+0000 7f631aa00640 1 -- 172.21.15.145:0/442102510 shutdown_connections 2024-05-09T04:34:22.188 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.186+0000 7f631aa00640 1 -- 172.21.15.145:0/442102510 wait complete. 2024-05-09T04:34:22.188 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.188+0000 7f631aa00640 1 Processor -- start 2024-05-09T04:34:22.188 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.188+0000 7f631aa00640 1 -- start start 2024-05-09T04:34:22.190 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.189+0000 7f631aa00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62f8003aa0 0x7f631414b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:22.190 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.189+0000 7f631aa00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f631409ba10 0x7f631409e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:22.190 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.189+0000 7f631aa00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6314161870 0x7f6314161cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:22.190 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.189+0000 7f631aa00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6314072880 con 0x7f6314161870 2024-05-09T04:34:22.190 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.189+0000 7f631aa00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f63140704d0 con 0x7f631409ba10 2024-05-09T04:34:22.190 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.189+0000 7f631aa00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f6314073870 con 0x7f62f8003aa0 2024-05-09T04:34:22.190 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.189+0000 7f631a000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6314161870 0x7f6314161cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:22.190 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.190+0000 7f631a000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6314161870 0x7f6314161cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:40462/0 (socket says 172.21.15.145:40462) 2024-05-09T04:34:22.190 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.190+0000 7f631a000640 1 -- 172.21.15.145:0/1228917 learned_addr learned my addr 172.21.15.145:0/1228917 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:22.190 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.190+0000 7f6319600640 1 --2- 172.21.15.145:0/1228917 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62f8003aa0 0x7f631414b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:22.190 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.190+0000 7f6318c00640 1 --2- 172.21.15.145:0/1228917 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f631409ba10 0x7f631409e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:22.190 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.190+0000 7f631a000640 1 -- 172.21.15.145:0/1228917 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62f8003aa0 msgr2=0x7f631414b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:22.190 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.190+0000 7f631a000640 1 --2- 172.21.15.145:0/1228917 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62f8003aa0 0x7f631414b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:22.191 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.190+0000 7f631a000640 1 -- 172.21.15.145:0/1228917 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f631409ba10 msgr2=0x7f631409e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:22.191 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.190+0000 7f631a000640 1 --2- 172.21.15.145:0/1228917 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f631409ba10 0x7f631409e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:22.191 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.190+0000 7f631a000640 1 -- 172.21.15.145:0/1228917 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f631409bf40 con 0x7f6314161870 2024-05-09T04:34:22.191 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.190+0000 7f631a000640 1 --2- 172.21.15.145:0/1228917 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6314161870 0x7f6314161cd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f63141607b0 tx=0x7f630c00d820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:22.192 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.191+0000 7f6311600640 1 -- 172.21.15.145:0/1228917 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630c004020 con 0x7f6314161870 2024-05-09T04:34:22.192 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.191+0000 7f6311600640 1 -- 172.21.15.145:0/1228917 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f630c0041e0 con 0x7f6314161870 2024-05-09T04:34:22.192 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.191+0000 7f6311600640 1 -- 172.21.15.145:0/1228917 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630c0043d0 con 0x7f6314161870 2024-05-09T04:34:22.192 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.191+0000 7f631aa00640 1 -- 172.21.15.145:0/1228917 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f631415c1e0 con 0x7f6314161870 2024-05-09T04:34:22.192 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.191+0000 7f631aa00640 1 -- 172.21.15.145:0/1228917 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f631414c210 con 0x7f6314161870 2024-05-09T04:34:22.192 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.192+0000 7f631aa00640 1 -- 172.21.15.145:0/1228917 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6314162210 con 0x7f6314161870 2024-05-09T04:34:22.192 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.193+0000 7f6311600640 1 -- 172.21.15.145:0/1228917 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f630c01a020 con 0x7f6314161870 2024-05-09T04:34:22.194 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.194+0000 7f6311600640 1 --2- 172.21.15.145:0/1228917 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f62f40762d0 0x7f62f4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:22.194 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.194+0000 7f6311600640 1 -- 172.21.15.145:0/1228917 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f630c096d70 con 0x7f6314161870 2024-05-09T04:34:22.194 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.194+0000 7f6319600640 1 --2- 172.21.15.145:0/1228917 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f62f40762d0 0x7f62f4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:22.194 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.195+0000 7f6319600640 1 --2- 172.21.15.145:0/1228917 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f62f40762d0 0x7f62f4078790 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6304033000 tx=0x7f6304007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:22.198 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.198+0000 7f6311600640 1 -- 172.21.15.145:0/1228917 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f630c060630 con 0x7f6314161870 2024-05-09T04:34:22.409 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.409+0000 7f631aa00640 1 -- 172.21.15.145:0/1228917 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f631406a310 con 0x7f6314161870 2024-05-09T04:34:22.409 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.409+0000 7f6311600640 1 -- 172.21.15.145:0/1228917 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f630c05ff30 con 0x7f6314161870 2024-05-09T04:34:22.409 INFO:teuthology.orchestra.run.smithi145.stdout:pg_num: 32 2024-05-09T04:34:22.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.412+0000 7f631aa00640 1 -- 172.21.15.145:0/1228917 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f62f40762d0 msgr2=0x7f62f4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:22.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.412+0000 7f631aa00640 1 --2- 172.21.15.145:0/1228917 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f62f40762d0 0x7f62f4078790 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6304033000 tx=0x7f6304007000 comp rx=0 tx=0).stop 2024-05-09T04:34:22.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.412+0000 7f631aa00640 1 -- 172.21.15.145:0/1228917 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6314161870 msgr2=0x7f6314161cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:22.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.412+0000 7f631aa00640 1 --2- 172.21.15.145:0/1228917 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6314161870 0x7f6314161cd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f63141607b0 tx=0x7f630c00d820 comp rx=0 tx=0).stop 2024-05-09T04:34:22.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.412+0000 7f631aa00640 1 -- 172.21.15.145:0/1228917 shutdown_connections 2024-05-09T04:34:22.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.412+0000 7f631aa00640 1 --2- 172.21.15.145:0/1228917 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f62f40762d0 0x7f62f4078790 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:22.413 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.412+0000 7f631aa00640 1 --2- 172.21.15.145:0/1228917 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6314161870 0x7f6314161cd0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:22.413 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.412+0000 7f631aa00640 1 --2- 172.21.15.145:0/1228917 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f631409ba10 0x7f631409e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:22.413 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.412+0000 7f631aa00640 1 --2- 172.21.15.145:0/1228917 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62f8003aa0 0x7f631414b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:22.413 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.412+0000 7f631aa00640 1 -- 172.21.15.145:0/1228917 >> 172.21.15.145:0/1228917 conn(0x7f63140b28f0 msgr2=0x7f631406e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:22.413 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.412+0000 7f631aa00640 1 -- 172.21.15.145:0/1228917 shutdown_connections 2024-05-09T04:34:22.413 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.413+0000 7f631aa00640 1 -- 172.21.15.145:0/1228917 wait complete. 2024-05-09T04:34:22.430 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T04:34:22.625 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.625+0000 7f713d600640 1 Processor -- start 2024-05-09T04:34:22.625 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.625+0000 7f713d600640 1 -- start start 2024-05-09T04:34:22.627 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.627+0000 7f713d600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f7138144c00 0x7f71380ce640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:22.627 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.627+0000 7f713d600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f7138145440 0x7f71381458d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:22.628 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.627+0000 7f713d600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f713806f5d0 0x7f713806fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:22.628 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.627+0000 7f713d600640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f713806ff60 con 0x7f7138144c00 2024-05-09T04:34:22.628 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.627+0000 7f713d600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f713815ffa0 con 0x7f7138145440 2024-05-09T04:34:22.628 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.627+0000 7f713d600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f7138160280 con 0x7f713806f5d0 2024-05-09T04:34:22.628 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.627+0000 7f7136a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f7138144c00 0x7f71380ce640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:22.628 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.627+0000 7f7136a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f7138144c00 0x7f71380ce640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:40464/0 (socket says 172.21.15.145:40464) 2024-05-09T04:34:22.628 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.627+0000 7f7136a00640 1 -- 172.21.15.145:0/962867904 learned_addr learned my addr 172.21.15.145:0/962867904 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:22.628 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.628+0000 7f7137400640 1 --2- 172.21.15.145:0/962867904 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f713806f5d0 0x7f713806fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:22.628 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.628+0000 7f7136000640 1 --2- 172.21.15.145:0/962867904 >> v2:172.21.15.192:3300/0 conn(0x7f7138145440 0x7f71381458d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:22.628 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.628+0000 7f7136a00640 1 -- 172.21.15.145:0/962867904 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f713806f5d0 msgr2=0x7f713806fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:22.628 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.628+0000 7f7136a00640 1 --2- 172.21.15.145:0/962867904 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f713806f5d0 0x7f713806fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:22.628 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.628+0000 7f7136a00640 1 -- 172.21.15.145:0/962867904 >> v2:172.21.15.192:3300/0 conn(0x7f7138145440 msgr2=0x7f71381458d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:22.628 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.628+0000 7f7136a00640 1 --2- 172.21.15.145:0/962867904 >> v2:172.21.15.192:3300/0 conn(0x7f7138145440 0x7f71381458d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:22.628 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.628+0000 7f7136a00640 1 -- 172.21.15.145:0/962867904 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7138160560 con 0x7f7138144c00 2024-05-09T04:34:22.630 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.629+0000 7f7136a00640 1 --2- 172.21.15.145:0/962867904 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7138144c00 0x7f71380ce640 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f713806d440 tx=0x7f712802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc462744f2630af8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:22.630 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.629+0000 7f7135600640 1 -- 172.21.15.145:0/962867904 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7128037070 con 0x7f7138144c00 2024-05-09T04:34:22.630 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.629+0000 7f7135600640 1 -- 172.21.15.145:0/962867904 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7128032440 con 0x7f7138144c00 2024-05-09T04:34:22.630 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.629+0000 7f7135600640 1 -- 172.21.15.145:0/962867904 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7128032810 con 0x7f7138144c00 2024-05-09T04:34:22.630 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.630+0000 7f713d600640 1 -- 172.21.15.145:0/962867904 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7138144c00 msgr2=0x7f71380ce640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:22.630 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.630+0000 7f713d600640 1 --2- 172.21.15.145:0/962867904 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7138144c00 0x7f71380ce640 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f713806d440 tx=0x7f712802a1e0 comp rx=0 tx=0).stop 2024-05-09T04:34:22.630 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.630+0000 7f713d600640 1 -- 172.21.15.145:0/962867904 shutdown_connections 2024-05-09T04:34:22.630 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.630+0000 7f713d600640 1 --2- 172.21.15.145:0/962867904 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f713806f5d0 0x7f713806fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:22.630 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.630+0000 7f713d600640 1 --2- 172.21.15.145:0/962867904 >> v2:172.21.15.192:3300/0 conn(0x7f7138145440 0x7f71381458d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:22.631 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.630+0000 7f713d600640 1 --2- 172.21.15.145:0/962867904 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7138144c00 0x7f71380ce640 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:22.631 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.630+0000 7f713d600640 1 -- 172.21.15.145:0/962867904 >> 172.21.15.145:0/962867904 conn(0x7f713806cab0 msgr2=0x7f71380d0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:22.631 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.630+0000 7f713d600640 1 -- 172.21.15.145:0/962867904 shutdown_connections 2024-05-09T04:34:22.631 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.630+0000 7f713d600640 1 -- 172.21.15.145:0/962867904 wait complete. 2024-05-09T04:34:22.632 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.632+0000 7f713d600640 1 Processor -- start 2024-05-09T04:34:22.632 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.632+0000 7f713d600640 1 -- start start 2024-05-09T04:34:22.635 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.634+0000 7f713d600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f713806f5d0 0x7f71380cc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:22.635 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.634+0000 7f713d600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7138144c00 0x7f71380b2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:22.636 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.634+0000 7f713d600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7138145440 0x7f713815ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:22.636 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.634+0000 7f713d600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f713815fc30 con 0x7f7138145440 2024-05-09T04:34:22.636 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.634+0000 7f713d600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7138160b20 con 0x7f713806f5d0 2024-05-09T04:34:22.636 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.634+0000 7f713d600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f7138144fe0 con 0x7f7138144c00 2024-05-09T04:34:22.636 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.634+0000 7f7137400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7138145440 0x7f713815ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:22.636 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.634+0000 7f7137400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7138145440 0x7f713815ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:40468/0 (socket says 172.21.15.145:40468) 2024-05-09T04:34:22.636 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.634+0000 7f7137400640 1 -- 172.21.15.145:0/691416031 learned_addr learned my addr 172.21.15.145:0/691416031 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:22.637 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.634+0000 7f7136a00640 1 --2- 172.21.15.145:0/691416031 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f713806f5d0 0x7f71380cc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:22.637 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.634+0000 7f7137400640 1 -- 172.21.15.145:0/691416031 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7138144c00 msgr2=0x7f71380b2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:22.637 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.634+0000 7f7137400640 1 --2- 172.21.15.145:0/691416031 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7138144c00 0x7f71380b2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:22.637 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.634+0000 7f7137400640 1 -- 172.21.15.145:0/691416031 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f713806f5d0 msgr2=0x7f71380cc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:22.637 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.634+0000 7f7137400640 1 --2- 172.21.15.145:0/691416031 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f713806f5d0 0x7f71380cc9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:22.637 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.634+0000 7f7137400640 1 -- 172.21.15.145:0/691416031 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71380aa470 con 0x7f7138145440 2024-05-09T04:34:22.637 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.635+0000 7f7137400640 1 --2- 172.21.15.145:0/691416031 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7138145440 0x7f713815ebb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7130007d20 tx=0x7f713000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:22.637 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.635+0000 7f712ea00640 1 -- 172.21.15.145:0/691416031 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7130014420 con 0x7f7138145440 2024-05-09T04:34:22.637 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.635+0000 7f712ea00640 1 -- 172.21.15.145:0/691416031 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7130019070 con 0x7f7138145440 2024-05-09T04:34:22.637 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.635+0000 7f712ea00640 1 -- 172.21.15.145:0/691416031 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7130014820 con 0x7f7138145440 2024-05-09T04:34:22.638 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.635+0000 7f713d600640 1 -- 172.21.15.145:0/691416031 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71380cd210 con 0x7f7138145440 2024-05-09T04:34:22.638 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.635+0000 7f713d600640 1 -- 172.21.15.145:0/691416031 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71380aa730 con 0x7f7138145440 2024-05-09T04:34:22.638 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.637+0000 7f713d600640 1 -- 172.21.15.145:0/691416031 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f713806a310 con 0x7f7138145440 2024-05-09T04:34:22.638 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.638+0000 7f712ea00640 1 -- 172.21.15.145:0/691416031 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f71300149c0 con 0x7f7138145440 2024-05-09T04:34:22.639 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.639+0000 7f712ea00640 1 --2- 172.21.15.145:0/691416031 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f71180762d0 0x7f7118078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:22.639 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.639+0000 7f712ea00640 1 -- 172.21.15.145:0/691416031 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7130097e90 con 0x7f7138145440 2024-05-09T04:34:22.639 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.639+0000 7f7136a00640 1 --2- 172.21.15.145:0/691416031 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f71180762d0 0x7f7118078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:22.640 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.640+0000 7f7136a00640 1 --2- 172.21.15.145:0/691416031 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f71180762d0 0x7f7118078790 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7128034000 tx=0x7f712802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:22.643 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.644+0000 7f712ea00640 1 -- 172.21.15.145:0/691416031 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f7130061750 con 0x7f7138145440 2024-05-09T04:34:22.844 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.845+0000 7f713d600640 1 -- 172.21.15.145:0/691416031 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f71380b51c0 con 0x7f7138145440 2024-05-09T04:34:22.845 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.845+0000 7f712ea00640 1 -- 172.21.15.145:0/691416031 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f7130061050 con 0x7f7138145440 2024-05-09T04:34:22.845 INFO:teuthology.orchestra.run.smithi145.stdout:pg_num: 64 2024-05-09T04:34:22.848 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.848+0000 7f713d600640 1 -- 172.21.15.145:0/691416031 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f71180762d0 msgr2=0x7f7118078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:22.848 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.848+0000 7f713d600640 1 --2- 172.21.15.145:0/691416031 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f71180762d0 0x7f7118078790 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7128034000 tx=0x7f712802e000 comp rx=0 tx=0).stop 2024-05-09T04:34:22.848 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.848+0000 7f713d600640 1 -- 172.21.15.145:0/691416031 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7138145440 msgr2=0x7f713815ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:22.848 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.848+0000 7f713d600640 1 --2- 172.21.15.145:0/691416031 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7138145440 0x7f713815ebb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7130007d20 tx=0x7f713000e4b0 comp rx=0 tx=0).stop 2024-05-09T04:34:22.848 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.848+0000 7f713d600640 1 -- 172.21.15.145:0/691416031 shutdown_connections 2024-05-09T04:34:22.848 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.848+0000 7f713d600640 1 --2- 172.21.15.145:0/691416031 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f71180762d0 0x7f7118078790 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:22.848 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.848+0000 7f713d600640 1 --2- 172.21.15.145:0/691416031 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7138145440 0x7f713815ebb0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:22.848 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.848+0000 7f713d600640 1 --2- 172.21.15.145:0/691416031 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7138144c00 0x7f71380b2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:22.848 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.848+0000 7f713d600640 1 --2- 172.21.15.145:0/691416031 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f713806f5d0 0x7f71380cc9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:22.848 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.848+0000 7f713d600640 1 -- 172.21.15.145:0/691416031 >> 172.21.15.145:0/691416031 conn(0x7f713806cab0 msgr2=0x7f71380d0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:22.848 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.848+0000 7f713d600640 1 -- 172.21.15.145:0/691416031 shutdown_connections 2024-05-09T04:34:22.849 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:22.848+0000 7f713d600640 1 -- 172.21.15.145:0/691416031 wait complete. 2024-05-09T04:34:22.866 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_backtrace.TestBacktrace.test_backtrace' 2024-05-09T04:34:23.042 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.042+0000 7f975fa00640 1 Processor -- start 2024-05-09T04:34:23.042 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.042+0000 7f975fa00640 1 -- start start 2024-05-09T04:34:23.043 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.043+0000 7f975fa00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f975806e860 0x7f975806ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:23.044 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.043+0000 7f975fa00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f975806f180 con 0x7f975815d780 2024-05-09T04:34:23.044 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.044+0000 7f975fa00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97581615c0 con 0x7f97580b4910 2024-05-09T04:34:23.044 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.044+0000 7f975fa00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9758161360 con 0x7f975806e860 2024-05-09T04:34:23.044 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.044+0000 7f9757e00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f975815d780 0x7f9758160c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:44848/0 (socket says 172.21.15.145:44848) 2024-05-09T04:34:23.044 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.044+0000 7f9757e00640 1 -- 172.21.15.145:0/265729477 learned_addr learned my addr 172.21.15.145:0/265729477 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:34:23.044 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.044+0000 7f975d000640 1 --2- 172.21.15.145:0/265729477 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f975806e860 0x7f975806ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:23.046 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.044+0000 7f9757400640 1 -- 172.21.15.145:0/265729477 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2341191800 0 0) 0x7f975806f180 con 0x7f975815d780 2024-05-09T04:34:23.046 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.044+0000 7f9757400640 1 -- 172.21.15.145:0/265729477 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9740003520 con 0x7f975815d780 2024-05-09T04:34:23.046 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.045+0000 7f9757400640 1 -- 172.21.15.145:0/265729477 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1535838867 0 0) 0x7f9740003520 con 0x7f975815d780 2024-05-09T04:34:23.046 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.045+0000 7f9757400640 1 -- 172.21.15.145:0/265729477 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f975806e860 msgr2=0x7f975806ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:23.046 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.045+0000 7f9757400640 1 --2- 172.21.15.145:0/265729477 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f975806e860 0x7f975806ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:23.046 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.045+0000 7f9757400640 1 -- 172.21.15.145:0/265729477 >> v1:172.21.15.192:6789/0 conn(0x7f97580b4910 legacy=0x7f97580b4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:23.046 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.045+0000 7f9757400640 1 -- 172.21.15.145:0/265729477 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97581618a0 con 0x7f975815d780 2024-05-09T04:34:23.046 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.045+0000 7f9757400640 1 -- 172.21.15.145:0/265729477 <== mon.0 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3474413261 0 0) 0x7f97440016f0 con 0x7f975815d780 2024-05-09T04:34:23.046 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.045+0000 7f9757400640 1 -- 172.21.15.145:0/265729477 >> v1:172.21.15.145:6789/0 conn(0x7f975815d780 legacy=0x7f9758160c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:23.046 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.045+0000 7f9757400640 1 --2- 172.21.15.145:0/265729477 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f975806e860 0x7f9740003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:23.047 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.045+0000 7f9757400640 1 --2- 172.21.15.145:0/265729477 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9740004000 0x7f9740004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:23.047 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.045+0000 7f9757400640 1 --2- 172.21.15.145:0/265729477 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9740004d40 0x7f97400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:23.047 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.045+0000 7f9757400640 1 -- 172.21.15.145:0/265729477 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9758161360 con 0x7f9740004000 2024-05-09T04:34:23.047 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.045+0000 7f9757400640 1 -- 172.21.15.145:0/265729477 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f974000bb50 con 0x7f9740004d40 2024-05-09T04:34:23.047 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.045+0000 7f9757400640 1 -- 172.21.15.145:0/265729477 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f974000bd50 con 0x7f975806e860 2024-05-09T04:34:23.047 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.045+0000 7f9757e00640 1 --2- 172.21.15.145:0/265729477 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9740004000 0x7f9740004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:23.047 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.045+0000 7f975d000640 1 --2- 172.21.15.145:0/265729477 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f975806e860 0x7f9740003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:23.047 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.046+0000 7f975da00640 1 --2- 172.21.15.145:0/265729477 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9740004d40 0x7f97400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:23.047 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.046+0000 7f9757e00640 1 -- 172.21.15.145:0/265729477 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f975806e860 msgr2=0x7f9740003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:23.047 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.046+0000 7f9757e00640 1 --2- 172.21.15.145:0/265729477 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f975806e860 0x7f9740003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:23.047 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.046+0000 7f9757e00640 1 -- 172.21.15.145:0/265729477 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9740004d40 msgr2=0x7f97400051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:23.047 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.046+0000 7f9757e00640 1 --2- 172.21.15.145:0/265729477 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9740004d40 0x7f97400051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:23.047 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.046+0000 7f9757e00640 1 -- 172.21.15.145:0/265729477 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9740003520 con 0x7f9740004000 2024-05-09T04:34:23.049 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.047+0000 7f9757e00640 1 --2- 172.21.15.145:0/265729477 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9740004000 0x7f9740004800 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9744001050 tx=0x7f9740005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7df0e713ade3d136 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:23.049 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.047+0000 7f9757400640 1 -- 172.21.15.145:0/265729477 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9744007d10 con 0x7f9740004000 2024-05-09T04:34:23.049 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.047+0000 7f9757400640 1 -- 172.21.15.145:0/265729477 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9744005ad0 con 0x7f9740004000 2024-05-09T04:34:23.049 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.047+0000 7f9757400640 1 -- 172.21.15.145:0/265729477 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9744007070 con 0x7f9740004000 2024-05-09T04:34:23.049 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.047+0000 7f975fa00640 1 -- 172.21.15.145:0/265729477 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9740004000 msgr2=0x7f9740004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:23.050 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.047+0000 7f975fa00640 1 --2- 172.21.15.145:0/265729477 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9740004000 0x7f9740004800 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9744001050 tx=0x7f9740005830 comp rx=0 tx=0).stop 2024-05-09T04:34:23.050 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.047+0000 7f975da00640 1 -- 172.21.15.145:0/265729477 reap_dead start 2024-05-09T04:34:23.050 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.047+0000 7f975fa00640 1 -- 172.21.15.145:0/265729477 shutdown_connections 2024-05-09T04:34:23.050 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.047+0000 7f975fa00640 1 -- 172.21.15.145:0/265729477 >> 172.21.15.145:0/265729477 conn(0x7f975806c870 msgr2=0x7f975806cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:23.050 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.047+0000 7f975fa00640 1 -- 172.21.15.145:0/265729477 shutdown_connections 2024-05-09T04:34:23.050 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.048+0000 7f975fa00640 1 -- 172.21.15.145:0/265729477 wait complete. 2024-05-09T04:34:23.050 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.049+0000 7f975fa00640 1 Processor -- start 2024-05-09T04:34:23.050 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.049+0000 7f975fa00640 1 -- start start 2024-05-09T04:34:23.052 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.050+0000 7f975fa00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9740004000 0x7f9758137bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:23.052 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.050+0000 7f975fa00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f975813cbb0 0x7f975813d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:23.052 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.050+0000 7f975fa00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f975813d8c0 0x7f975813dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:23.052 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.050+0000 7f975fa00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9758146c00 con 0x7f975813d8c0 2024-05-09T04:34:23.052 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.050+0000 7f975fa00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f97580b43f0 con 0x7f975813cbb0 2024-05-09T04:34:23.052 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.050+0000 7f975fa00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9758146d80 con 0x7f9740004000 2024-05-09T04:34:23.052 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.050+0000 7f975da00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f975813d8c0 0x7f975813dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:23.052 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.051+0000 7f975da00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f975813d8c0 0x7f975813dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:40490/0 (socket says 172.21.15.145:40490) 2024-05-09T04:34:23.053 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.051+0000 7f975da00640 1 -- 172.21.15.145:0/3380336898 learned_addr learned my addr 172.21.15.145:0/3380336898 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:23.053 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.051+0000 7f975d000640 1 --2- 172.21.15.145:0/3380336898 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9740004000 0x7f9758137bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:23.053 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.051+0000 7f9757e00640 1 --2- 172.21.15.145:0/3380336898 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f975813cbb0 0x7f975813d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:23.053 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.051+0000 7f975da00640 1 -- 172.21.15.145:0/3380336898 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9740004000 msgr2=0x7f9758137bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:23.054 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.051+0000 7f975da00640 1 --2- 172.21.15.145:0/3380336898 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9740004000 0x7f9758137bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:23.054 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.051+0000 7f975da00640 1 -- 172.21.15.145:0/3380336898 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f975813cbb0 msgr2=0x7f975813d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:23.054 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.051+0000 7f975da00640 1 --2- 172.21.15.145:0/3380336898 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f975813cbb0 0x7f975813d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:23.054 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.051+0000 7f975da00640 1 -- 172.21.15.145:0/3380336898 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9758138110 con 0x7f975813d8c0 2024-05-09T04:34:23.054 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.051+0000 7f975da00640 1 --2- 172.21.15.145:0/3380336898 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f975813d8c0 0x7f975813dd40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f97580ce280 tx=0x7f975000db50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:23.054 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.051+0000 7f9754c00640 1 -- 172.21.15.145:0/3380336898 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9750018070 con 0x7f975813d8c0 2024-05-09T04:34:23.054 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.052+0000 7f9754c00640 1 -- 172.21.15.145:0/3380336898 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9750003b70 con 0x7f975813d8c0 2024-05-09T04:34:23.054 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.052+0000 7f9754c00640 1 -- 172.21.15.145:0/3380336898 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9750005070 con 0x7f975813d8c0 2024-05-09T04:34:23.054 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.052+0000 7f975fa00640 1 -- 172.21.15.145:0/3380336898 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97581e6cc0 con 0x7f975813d8c0 2024-05-09T04:34:23.055 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.052+0000 7f975fa00640 1 -- 172.21.15.145:0/3380336898 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97581e7280 con 0x7f975813d8c0 2024-05-09T04:34:23.055 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.053+0000 7f975fa00640 1 -- 172.21.15.145:0/3380336898 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f975806a310 con 0x7f975813d8c0 2024-05-09T04:34:23.055 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.054+0000 7f9754c00640 1 -- 172.21.15.145:0/3380336898 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f975001b070 con 0x7f975813d8c0 2024-05-09T04:34:23.056 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.055+0000 7f9754c00640 1 --2- 172.21.15.145:0/3380336898 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f9738076300 0x7f97380787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:23.056 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.055+0000 7f9754c00640 1 -- 172.21.15.145:0/3380336898 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9750096d50 con 0x7f975813d8c0 2024-05-09T04:34:23.056 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.055+0000 7f975d000640 1 --2- 172.21.15.145:0/3380336898 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f9738076300 0x7f97380787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:23.056 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.056+0000 7f975d000640 1 --2- 172.21.15.145:0/3380336898 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f9738076300 0x7f97380787c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f974c009a10 tx=0x7f974c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:23.059 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.059+0000 7f9754c00640 1 -- 172.21.15.145:0/3380336898 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f9758138110 con 0x7f975813d8c0 2024-05-09T04:34:23.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.324+0000 7f975fa00640 1 -- 172.21.15.145:0/3380336898 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_backtrace.TestBacktrace.test_backtrace"]} v 0) -- 0x7f97581463b0 con 0x7f975813d8c0 2024-05-09T04:34:23.350 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.350+0000 7f9754c00640 1 -- 172.21.15.145:0/3380336898 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_backtrace.TestBacktrace.test_backtrace"]}]=0 v49) ==== 138+0+0 (secure 0 0 0) 0x7f97500606a0 con 0x7f975813d8c0 2024-05-09T04:34:23.353 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.353+0000 7f975fa00640 1 -- 172.21.15.145:0/3380336898 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f9738076300 msgr2=0x7f97380787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:23.353 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.353+0000 7f975fa00640 1 --2- 172.21.15.145:0/3380336898 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f9738076300 0x7f97380787c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f974c009a10 tx=0x7f974c008040 comp rx=0 tx=0).stop 2024-05-09T04:34:23.353 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.353+0000 7f975fa00640 1 -- 172.21.15.145:0/3380336898 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f975813d8c0 msgr2=0x7f975813dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:23.353 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.353+0000 7f975fa00640 1 --2- 172.21.15.145:0/3380336898 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f975813d8c0 0x7f975813dd40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f97580ce280 tx=0x7f975000db50 comp rx=0 tx=0).stop 2024-05-09T04:34:23.353 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.353+0000 7f975fa00640 1 -- 172.21.15.145:0/3380336898 shutdown_connections 2024-05-09T04:34:23.353 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.353+0000 7f975fa00640 1 --2- 172.21.15.145:0/3380336898 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f9738076300 0x7f97380787c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:23.353 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.353+0000 7f975fa00640 1 --2- 172.21.15.145:0/3380336898 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f975813d8c0 0x7f975813dd40 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:23.354 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.353+0000 7f975fa00640 1 --2- 172.21.15.145:0/3380336898 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f975813cbb0 0x7f975813d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:23.354 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.353+0000 7f975fa00640 1 --2- 172.21.15.145:0/3380336898 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9740004000 0x7f9758137bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:23.354 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.353+0000 7f975fa00640 1 -- 172.21.15.145:0/3380336898 >> 172.21.15.145:0/3380336898 conn(0x7f975806c870 msgr2=0x7f9758142200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:23.354 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.353+0000 7f975fa00640 1 -- 172.21.15.145:0/3380336898 shutdown_connections 2024-05-09T04:34:23.354 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.353+0000 7f975fa00640 1 -- 172.21.15.145:0/3380336898 wait complete. 2024-05-09T04:34:23.372 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-09T04:34:23.573 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.573+0000 7fa835000640 1 Processor -- start 2024-05-09T04:34:23.573 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.573+0000 7fa835000640 1 -- start start 2024-05-09T04:34:23.574 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.574+0000 7fa835000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fa8300d1940 0x7fa8300d5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:23.574 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.575+0000 7fa835000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa8300d2c40 0x7fa8300d30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:23.575 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.575+0000 7fa835000640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa830160390 con 0x7fa8300b3db0 2024-05-09T04:34:23.575 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.575+0000 7fa835000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fa8300d0340 con 0x7fa8300d1940 2024-05-09T04:34:23.575 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.575+0000 7fa835000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fa830160630 con 0x7fa8300d2c40 2024-05-09T04:34:23.575 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.575+0000 7fa82f400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fa8300d1940 0x7fa8300d5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:23.575 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.575+0000 7fa82f400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fa8300d1940 0x7fa8300d5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:40492/0 (socket says 172.21.15.145:40492) 2024-05-09T04:34:23.575 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.575+0000 7fa82f400640 1 -- 172.21.15.145:0/3326275347 learned_addr learned my addr 172.21.15.145:0/3326275347 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:23.575 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.575+0000 7fa82f400640 1 -- 172.21.15.145:0/3326275347 >> v2:172.21.15.192:3300/0 conn(0x7fa8300d2c40 msgr2=0x7fa8300d30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:23.575 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.575+0000 7fa82f400640 1 --2- 172.21.15.145:0/3326275347 >> v2:172.21.15.192:3300/0 conn(0x7fa8300d2c40 0x7fa8300d30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:23.576 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.575+0000 7fa82f400640 1 -- 172.21.15.145:0/3326275347 >> v1:172.21.15.192:6789/0 conn(0x7fa8300b3db0 legacy=0x7fa8300d1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:23.576 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.575+0000 7fa82f400640 1 -- 172.21.15.145:0/3326275347 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8300cffd0 con 0x7fa8300d1940 2024-05-09T04:34:23.576 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.576+0000 7fa82f400640 1 --2- 172.21.15.145:0/3326275347 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8300d1940 0x7fa8300d5b90 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa8300b7c00 tx=0x7fa82002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f875a7ee06f525ca server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:23.576 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.577+0000 7fa82e000640 1 -- 172.21.15.145:0/3326275347 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa820036070 con 0x7fa8300d1940 2024-05-09T04:34:23.576 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.577+0000 7fa82e000640 1 -- 172.21.15.145:0/3326275347 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa820031600 con 0x7fa8300d1940 2024-05-09T04:34:23.576 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.577+0000 7fa82e000640 1 -- 172.21.15.145:0/3326275347 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8200319d0 con 0x7fa8300d1940 2024-05-09T04:34:23.577 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.577+0000 7fa835000640 1 -- 172.21.15.145:0/3326275347 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8300d1940 msgr2=0x7fa8300d5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:23.577 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.577+0000 7fa835000640 1 --2- 172.21.15.145:0/3326275347 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8300d1940 0x7fa8300d5b90 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa8300b7c00 tx=0x7fa82002a610 comp rx=0 tx=0).stop 2024-05-09T04:34:23.577 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.577+0000 7fa835000640 1 -- 172.21.15.145:0/3326275347 shutdown_connections 2024-05-09T04:34:23.577 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.577+0000 7fa835000640 1 --2- 172.21.15.145:0/3326275347 >> v2:172.21.15.192:3300/0 conn(0x7fa8300d2c40 0x7fa8300d30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:23.577 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.577+0000 7fa835000640 1 --2- 172.21.15.145:0/3326275347 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8300d1940 0x7fa8300d5b90 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:23.578 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.577+0000 7fa835000640 1 -- 172.21.15.145:0/3326275347 >> 172.21.15.145:0/3326275347 conn(0x7fa8300d5070 msgr2=0x7fa8300d5480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:23.578 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.577+0000 7fa835000640 1 -- 172.21.15.145:0/3326275347 shutdown_connections 2024-05-09T04:34:23.578 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.577+0000 7fa835000640 1 -- 172.21.15.145:0/3326275347 wait complete. 2024-05-09T04:34:23.579 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.579+0000 7fa835000640 1 Processor -- start 2024-05-09T04:34:23.579 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.579+0000 7fa835000640 1 -- start start 2024-05-09T04:34:23.581 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.580+0000 7fa835000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa8300b3db0 0x7fa8301f5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:23.581 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.580+0000 7fa835000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8300d1940 0x7fa8301522b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:23.582 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.580+0000 7fa835000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8300d2c40 0x7fa8300b8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:23.582 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.580+0000 7fa835000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa83015f620 con 0x7fa8300d1940 2024-05-09T04:34:23.582 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.580+0000 7fa835000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa8300cfc80 con 0x7fa8300d2c40 2024-05-09T04:34:23.582 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.580+0000 7fa835000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa8300d07d0 con 0x7fa8300b3db0 2024-05-09T04:34:23.582 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.580+0000 7fa82ea00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8300d1940 0x7fa8301522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:23.582 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.581+0000 7fa82ea00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8300d1940 0x7fa8301522b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:40496/0 (socket says 172.21.15.145:40496) 2024-05-09T04:34:23.582 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.581+0000 7fa82ea00640 1 -- 172.21.15.145:0/2029911321 learned_addr learned my addr 172.21.15.145:0/2029911321 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:23.582 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.581+0000 7fa82f400640 1 --2- 172.21.15.145:0/2029911321 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa8300b3db0 0x7fa8301f5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:23.582 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.581+0000 7fa82fe00640 1 --2- 172.21.15.145:0/2029911321 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8300d2c40 0x7fa8300b8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:23.582 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.581+0000 7fa82ea00640 1 -- 172.21.15.145:0/2029911321 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa8300b3db0 msgr2=0x7fa8301f5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:23.582 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.581+0000 7fa82ea00640 1 --2- 172.21.15.145:0/2029911321 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa8300b3db0 0x7fa8301f5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:23.582 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.581+0000 7fa82ea00640 1 -- 172.21.15.145:0/2029911321 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8300d2c40 msgr2=0x7fa8300b8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:23.582 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.581+0000 7fa82ea00640 1 --2- 172.21.15.145:0/2029911321 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8300d2c40 0x7fa8300b8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:23.582 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.581+0000 7fa82ea00640 1 -- 172.21.15.145:0/2029911321 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8300b9000 con 0x7fa8300d1940 2024-05-09T04:34:23.583 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.581+0000 7fa82f400640 1 --2- 172.21.15.145:0/2029911321 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa8300b3db0 0x7fa8301f5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:34:23.583 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.581+0000 7fa82ea00640 1 --2- 172.21.15.145:0/2029911321 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8300d1940 0x7fa8301522b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa81800c410 tx=0x7fa81800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:23.583 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.581+0000 7fa827400640 1 -- 172.21.15.145:0/2029911321 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa818017070 con 0x7fa8300d1940 2024-05-09T04:34:23.583 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.582+0000 7fa827400640 1 -- 172.21.15.145:0/2029911321 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8180035f0 con 0x7fa8300d1940 2024-05-09T04:34:23.583 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.582+0000 7fa835000640 1 -- 172.21.15.145:0/2029911321 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8300b92e0 con 0x7fa8300d1940 2024-05-09T04:34:23.583 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.582+0000 7fa827400640 1 -- 172.21.15.145:0/2029911321 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa818003a20 con 0x7fa8300d1940 2024-05-09T04:34:23.583 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.582+0000 7fa835000640 1 -- 172.21.15.145:0/2029911321 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8300c8f20 con 0x7fa8300d1940 2024-05-09T04:34:23.583 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.583+0000 7fa835000640 1 -- 172.21.15.145:0/2029911321 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa83006a310 con 0x7fa8300d1940 2024-05-09T04:34:23.584 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.584+0000 7fa827400640 1 -- 172.21.15.145:0/2029911321 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa81801a070 con 0x7fa8300d1940 2024-05-09T04:34:23.585 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.585+0000 7fa827400640 1 --2- 172.21.15.145:0/2029911321 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fa80c0762d0 0x7fa80c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:23.585 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.586+0000 7fa827400640 1 -- 172.21.15.145:0/2029911321 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa818096a70 con 0x7fa8300d1940 2024-05-09T04:34:23.585 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.586+0000 7fa82f400640 1 --2- 172.21.15.145:0/2029911321 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fa80c0762d0 0x7fa80c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:23.586 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.586+0000 7fa82f400640 1 --2- 172.21.15.145:0/2029911321 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fa80c0762d0 0x7fa80c078790 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa820033000 tx=0x7fa820009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:23.589 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.590+0000 7fa827400640 1 -- 172.21.15.145:0/2029911321 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa8180603c0 con 0x7fa8300d1940 2024-05-09T04:34:23.780 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.781+0000 7fa835000640 1 -- 172.21.15.145:0/2029911321 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fa83008e800 con 0x7fa8300d1940 2024-05-09T04:34:23.785 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.785+0000 7fa827400640 1 -- 172.21.15.145:0/2029911321 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7fa81805fcc0 con 0x7fa8300d1940 2024-05-09T04:34:23.788 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.788+0000 7fa835000640 1 -- 172.21.15.145:0/2029911321 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fa80c0762d0 msgr2=0x7fa80c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:23.788 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.788+0000 7fa835000640 1 --2- 172.21.15.145:0/2029911321 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fa80c0762d0 0x7fa80c078790 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa820033000 tx=0x7fa820009000 comp rx=0 tx=0).stop 2024-05-09T04:34:23.788 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.788+0000 7fa835000640 1 -- 172.21.15.145:0/2029911321 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8300d1940 msgr2=0x7fa8301522b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:23.788 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.788+0000 7fa835000640 1 --2- 172.21.15.145:0/2029911321 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8300d1940 0x7fa8301522b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa81800c410 tx=0x7fa81800c8d0 comp rx=0 tx=0).stop 2024-05-09T04:34:23.788 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.788+0000 7fa835000640 1 -- 172.21.15.145:0/2029911321 shutdown_connections 2024-05-09T04:34:23.788 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.788+0000 7fa835000640 1 --2- 172.21.15.145:0/2029911321 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fa80c0762d0 0x7fa80c078790 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:23.788 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.788+0000 7fa835000640 1 --2- 172.21.15.145:0/2029911321 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8300d2c40 0x7fa8300b8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:23.788 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.788+0000 7fa835000640 1 --2- 172.21.15.145:0/2029911321 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa8300d1940 0x7fa8301522b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:23.788 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.788+0000 7fa835000640 1 --2- 172.21.15.145:0/2029911321 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa8300b3db0 0x7fa8301f5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:23.788 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.788+0000 7fa835000640 1 -- 172.21.15.145:0/2029911321 >> 172.21.15.145:0/2029911321 conn(0x7fa8300d5070 msgr2=0x7fa83006e0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:23.788 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.788+0000 7fa835000640 1 -- 172.21.15.145:0/2029911321 shutdown_connections 2024-05-09T04:34:23.789 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:23.788+0000 7fa835000640 1 -- 172.21.15.145:0/2029911321 wait complete. 2024-05-09T04:34:23.806 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-09T04:34:23.807 DEBUG:teuthology.orchestra.run.smithi145:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-09T04:34:23.906 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-09T04:34:24.014 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-09T04:34:24.014 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-05-09T04:34:24.014 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T04:34:24.072 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-05-09T04:34:24.072 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T04:34:24.127 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T04:34:24.322 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.322+0000 7f4014e00640 1 Processor -- start 2024-05-09T04:34:24.323 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.323+0000 7f4014e00640 1 -- start start 2024-05-09T04:34:24.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.324+0000 7f4014e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f40100d06f0 0x7f40100ae450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:24.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.324+0000 7f4014e00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40100b4ef0 con 0x7f401006cfb0 2024-05-09T04:34:24.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.324+0000 7f4014e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40100b51d0 con 0x7f401015d190 2024-05-09T04:34:24.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.324+0000 7f4014e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f40100b54b0 con 0x7f40100d06f0 2024-05-09T04:34:24.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.324+0000 7f4007e00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f401006cfb0 0x7f40100add40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:44850/0 (socket says 172.21.15.145:44850) 2024-05-09T04:34:24.325 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.325+0000 7f4007e00640 1 -- 172.21.15.145:0/2358201208 learned_addr learned my addr 172.21.15.145:0/2358201208 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:34:24.326 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.325+0000 7f400fe00640 1 --2- 172.21.15.145:0/2358201208 >> v2:172.21.15.192:3300/0 conn(0x7f40100d06f0 0x7f40100ae450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:24.326 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.325+0000 7f400ea00640 1 -- 172.21.15.145:0/2358201208 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2029030997 0 0) 0x7f40100b4ef0 con 0x7f401006cfb0 2024-05-09T04:34:24.326 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.325+0000 7f400ea00640 1 -- 172.21.15.145:0/2358201208 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ff0003520 con 0x7f401006cfb0 2024-05-09T04:34:24.326 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.325+0000 7f400ea00640 1 -- 172.21.15.145:0/2358201208 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3170466510 0 0) 0x7f3ff0003520 con 0x7f401006cfb0 2024-05-09T04:34:24.327 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.325+0000 7f400ea00640 1 -- 172.21.15.145:0/2358201208 >> v2:172.21.15.192:3300/0 conn(0x7f40100d06f0 msgr2=0x7f40100ae450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:24.327 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.325+0000 7f400ea00640 1 --2- 172.21.15.145:0/2358201208 >> v2:172.21.15.192:3300/0 conn(0x7f40100d06f0 0x7f40100ae450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:24.327 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.325+0000 7f400ea00640 1 -- 172.21.15.145:0/2358201208 >> v1:172.21.15.192:6789/0 conn(0x7f401015d190 legacy=0x7f401006c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:24.327 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.325+0000 7f400ea00640 1 -- 172.21.15.145:0/2358201208 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40100b5790 con 0x7f401006cfb0 2024-05-09T04:34:24.327 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.325+0000 7f400ea00640 1 -- 172.21.15.145:0/2358201208 <== mon.0 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3474413261 0 0) 0x7f3ff8001710 con 0x7f401006cfb0 2024-05-09T04:34:24.327 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.325+0000 7f400ea00640 1 -- 172.21.15.145:0/2358201208 >> v1:172.21.15.145:6789/0 conn(0x7f401006cfb0 legacy=0x7f40100add40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:24.327 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.325+0000 7f400ea00640 1 --2- 172.21.15.145:0/2358201208 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ff0003a50 0x7f3ff0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:24.327 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.326+0000 7f400ea00640 1 --2- 172.21.15.145:0/2358201208 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3ff0004800 0x7f3ff0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:24.327 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.326+0000 7f400ea00640 1 --2- 172.21.15.145:0/2358201208 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ff00051c0 0x7f3ff000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:24.327 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.326+0000 7f400ea00640 1 -- 172.21.15.145:0/2358201208 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f40100b54b0 con 0x7f3ff00051c0 2024-05-09T04:34:24.327 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.326+0000 7f400ea00640 1 -- 172.21.15.145:0/2358201208 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff000c160 con 0x7f3ff0003a50 2024-05-09T04:34:24.327 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.326+0000 7f400ea00640 1 -- 172.21.15.145:0/2358201208 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f3ff000c360 con 0x7f3ff0004800 2024-05-09T04:34:24.327 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.326+0000 7f400fe00640 1 --2- 172.21.15.145:0/2358201208 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ff00051c0 0x7f3ff000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:24.327 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.326+0000 7f400f400640 1 --2- 172.21.15.145:0/2358201208 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ff0003a50 0x7f3ff0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:24.327 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.326+0000 7f4007e00640 1 --2- 172.21.15.145:0/2358201208 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3ff0004800 0x7f3ff0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:24.328 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.326+0000 7f400fe00640 1 -- 172.21.15.145:0/2358201208 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3ff0004800 msgr2=0x7f3ff0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:24.328 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.326+0000 7f400fe00640 1 --2- 172.21.15.145:0/2358201208 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3ff0004800 0x7f3ff0004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:24.328 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.326+0000 7f400fe00640 1 -- 172.21.15.145:0/2358201208 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ff0003a50 msgr2=0x7f3ff0003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:24.328 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.326+0000 7f400fe00640 1 --2- 172.21.15.145:0/2358201208 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ff0003a50 0x7f3ff0003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:24.328 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.326+0000 7f400fe00640 1 -- 172.21.15.145:0/2358201208 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ff0003520 con 0x7f3ff00051c0 2024-05-09T04:34:24.328 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.326+0000 7f400f400640 1 --2- 172.21.15.145:0/2358201208 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ff0003a50 0x7f3ff0003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:34:24.329 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.327+0000 7f400fe00640 1 -- 172.21.15.145:0/2358201208 reap_dead start 2024-05-09T04:34:24.329 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.327+0000 7f400fe00640 1 --2- 172.21.15.145:0/2358201208 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ff00051c0 0x7f3ff000b9c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4008009fc0 tx=0x7f3ff0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2c0527075cb20bf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:24.330 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.328+0000 7f400ea00640 1 -- 172.21.15.145:0/2358201208 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4008035020 con 0x7f3ff00051c0 2024-05-09T04:34:24.330 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.328+0000 7f400ea00640 1 -- 172.21.15.145:0/2358201208 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4008003bd0 con 0x7f3ff00051c0 2024-05-09T04:34:24.330 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.328+0000 7f400ea00640 1 -- 172.21.15.145:0/2358201208 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4008009c10 con 0x7f3ff00051c0 2024-05-09T04:34:24.330 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.328+0000 7f4014e00640 1 -- 172.21.15.145:0/2358201208 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ff00051c0 msgr2=0x7f3ff000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:24.330 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.328+0000 7f4014e00640 1 --2- 172.21.15.145:0/2358201208 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ff00051c0 0x7f3ff000b9c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4008009fc0 tx=0x7f3ff0005830 comp rx=0 tx=0).stop 2024-05-09T04:34:24.330 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.328+0000 7f4014e00640 1 -- 172.21.15.145:0/2358201208 shutdown_connections 2024-05-09T04:34:24.330 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.328+0000 7f4014e00640 1 --2- 172.21.15.145:0/2358201208 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ff00051c0 0x7f3ff000b9c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:24.330 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.328+0000 7f4014e00640 1 -- 172.21.15.145:0/2358201208 >> 172.21.15.145:0/2358201208 conn(0x7f40100b2d80 msgr2=0x7f40100b3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:24.330 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.328+0000 7f4014e00640 1 -- 172.21.15.145:0/2358201208 shutdown_connections 2024-05-09T04:34:24.330 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.328+0000 7f4014e00640 1 -- 172.21.15.145:0/2358201208 wait complete. 2024-05-09T04:34:24.330 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.330+0000 7f4014e00640 1 Processor -- start 2024-05-09T04:34:24.332 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.330+0000 7f4014e00640 1 -- start start 2024-05-09T04:34:24.332 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.331+0000 7f4014e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ff00051c0 0x7f4010160250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:24.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.331+0000 7f4014e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f401006fb60 0x7f4010153020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:24.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.331+0000 7f4014e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4010159b20 0x7f4010159fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:24.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.331+0000 7f4014e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f40100cf210 con 0x7f3ff00051c0 2024-05-09T04:34:24.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.331+0000 7f4014e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f40100b4cd0 con 0x7f4010159b20 2024-05-09T04:34:24.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.331+0000 7f4014e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f40100cf390 con 0x7f401006fb60 2024-05-09T04:34:24.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.331+0000 7f400f400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ff00051c0 0x7f4010160250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:24.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.331+0000 7f400f400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ff00051c0 0x7f4010160250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:40506/0 (socket says 172.21.15.145:40506) 2024-05-09T04:34:24.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.331+0000 7f400f400640 1 -- 172.21.15.145:0/33293192 learned_addr learned my addr 172.21.15.145:0/33293192 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:24.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.332+0000 7f4007e00640 1 --2- 172.21.15.145:0/33293192 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f401006fb60 0x7f4010153020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:24.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.332+0000 7f400fe00640 1 --2- 172.21.15.145:0/33293192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4010159b20 0x7f4010159fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:24.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.332+0000 7f400f400640 1 -- 172.21.15.145:0/33293192 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f401006fb60 msgr2=0x7f4010153020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:24.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.332+0000 7f400f400640 1 --2- 172.21.15.145:0/33293192 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f401006fb60 0x7f4010153020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:24.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.332+0000 7f400f400640 1 -- 172.21.15.145:0/33293192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4010159b20 msgr2=0x7f4010159fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:24.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.332+0000 7f400f400640 1 --2- 172.21.15.145:0/33293192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4010159b20 0x7f4010159fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:24.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.332+0000 7f400f400640 1 -- 172.21.15.145:0/33293192 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f401015a4e0 con 0x7f3ff00051c0 2024-05-09T04:34:24.334 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.332+0000 7f400f400640 1 --2- 172.21.15.145:0/33293192 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ff00051c0 0x7f4010160250 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f400000d7a0 tx=0x7f400000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:24.334 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.332+0000 7f4007400640 1 -- 172.21.15.145:0/33293192 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4000005270 con 0x7f3ff00051c0 2024-05-09T04:34:24.334 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.332+0000 7f4007400640 1 -- 172.21.15.145:0/33293192 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4000017070 con 0x7f3ff00051c0 2024-05-09T04:34:24.334 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.332+0000 7f4007400640 1 -- 172.21.15.145:0/33293192 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400000b650 con 0x7f3ff00051c0 2024-05-09T04:34:24.334 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.332+0000 7f4014e00640 1 -- 172.21.15.145:0/33293192 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4010154d90 con 0x7f3ff00051c0 2024-05-09T04:34:24.334 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.333+0000 7f4014e00640 1 -- 172.21.15.145:0/33293192 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4010153720 con 0x7f3ff00051c0 2024-05-09T04:34:24.334 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.334+0000 7f4014e00640 1 -- 172.21.15.145:0/33293192 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40100d12e0 con 0x7f3ff00051c0 2024-05-09T04:34:24.334 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.335+0000 7f4007400640 1 -- 172.21.15.145:0/33293192 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f400000b7f0 con 0x7f3ff00051c0 2024-05-09T04:34:24.335 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.336+0000 7f4007400640 1 --2- 172.21.15.145:0/33293192 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f3fec0762d0 0x7f3fec078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:24.336 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.336+0000 7f4007400640 1 -- 172.21.15.145:0/33293192 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f40000974c0 con 0x7f3ff00051c0 2024-05-09T04:34:24.336 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.336+0000 7f4007e00640 1 --2- 172.21.15.145:0/33293192 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f3fec0762d0 0x7f3fec078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:24.336 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.337+0000 7f4007e00640 1 --2- 172.21.15.145:0/33293192 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f3fec0762d0 0x7f3fec078790 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f401015a890 tx=0x7f3ff8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:24.340 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.340+0000 7f4007400640 1 -- 172.21.15.145:0/33293192 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f40100d12e0 con 0x7f3ff00051c0 2024-05-09T04:34:24.560 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.560+0000 7f4014e00640 1 -- 172.21.15.145:0/33293192 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4010076780 con 0x7f3ff00051c0 2024-05-09T04:34:24.560 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.561+0000 7f4007400640 1 -- 172.21.15.145:0/33293192 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7f4010154d90 con 0x7f3ff00051c0 2024-05-09T04:34:24.560 INFO:teuthology.orchestra.run.smithi145.stderr:listed 0 entries 2024-05-09T04:34:24.564 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.564+0000 7f4014e00640 1 -- 172.21.15.145:0/33293192 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f3fec0762d0 msgr2=0x7f3fec078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:24.564 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.564+0000 7f4014e00640 1 --2- 172.21.15.145:0/33293192 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f3fec0762d0 0x7f3fec078790 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f401015a890 tx=0x7f3ff8009040 comp rx=0 tx=0).stop 2024-05-09T04:34:24.564 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.564+0000 7f4014e00640 1 -- 172.21.15.145:0/33293192 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ff00051c0 msgr2=0x7f4010160250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:24.564 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.564+0000 7f4014e00640 1 --2- 172.21.15.145:0/33293192 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ff00051c0 0x7f4010160250 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f400000d7a0 tx=0x7f400000dc60 comp rx=0 tx=0).stop 2024-05-09T04:34:24.564 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.564+0000 7f4014e00640 1 -- 172.21.15.145:0/33293192 shutdown_connections 2024-05-09T04:34:24.564 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.564+0000 7f4014e00640 1 --2- 172.21.15.145:0/33293192 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f3fec0762d0 0x7f3fec078790 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:24.564 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.564+0000 7f4014e00640 1 --2- 172.21.15.145:0/33293192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4010159b20 0x7f4010159fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:24.564 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.564+0000 7f4014e00640 1 --2- 172.21.15.145:0/33293192 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f401006fb60 0x7f4010153020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:24.564 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.564+0000 7f4014e00640 1 --2- 172.21.15.145:0/33293192 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3ff00051c0 0x7f4010160250 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:24.564 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.564+0000 7f4014e00640 1 -- 172.21.15.145:0/33293192 >> 172.21.15.145:0/33293192 conn(0x7f40100b2d80 msgr2=0x7f4010073430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:24.564 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.564+0000 7f4014e00640 1 -- 172.21.15.145:0/33293192 shutdown_connections 2024-05-09T04:34:24.564 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.564+0000 7f4014e00640 1 -- 172.21.15.145:0/33293192 wait complete. 2024-05-09T04:34:24.582 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-05-09T04:34:24.582 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T04:34:24.632 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T04:34:24.832 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.832+0000 7fe8ed400640 1 Processor -- start 2024-05-09T04:34:24.832 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.833+0000 7fe8ed400640 1 -- start start 2024-05-09T04:34:24.834 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.834+0000 7fe8ed400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe8e815d190 0x7fe8e806c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:24.834 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.834+0000 7fe8ed400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8e80d06f0 0x7fe8e806cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:24.834 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.834+0000 7fe8ed400640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8e80b4ef0 con 0x7fe8e80cf210 2024-05-09T04:34:24.834 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.834+0000 7fe8ed400640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fe8e80b51d0 con 0x7fe8e815d190 2024-05-09T04:34:24.834 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.834+0000 7fe8ed400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe8e80b54b0 con 0x7fe8e80d06f0 2024-05-09T04:34:24.834 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.835+0000 7fe8eca00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7fe8e80cf210 0x7fe8e8072140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:44854/0 (socket says 172.21.15.145:44854) 2024-05-09T04:34:24.834 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.835+0000 7fe8eca00640 1 -- 172.21.15.145:0/9391569 learned_addr learned my addr 172.21.15.145:0/9391569 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:34:24.836 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.835+0000 7fe8df400640 1 --2- 172.21.15.145:0/9391569 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8e80d06f0 0x7fe8e806cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:24.836 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.835+0000 7fe8e7e00640 1 --2- 172.21.15.145:0/9391569 >> v2:172.21.15.192:3300/0 conn(0x7fe8e815d190 0x7fe8e806c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:24.837 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.835+0000 7fe8e7400640 1 -- 172.21.15.145:0/9391569 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1135524717 0 0) 0x7fe8e80b4ef0 con 0x7fe8e80cf210 2024-05-09T04:34:24.837 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.835+0000 7fe8e7400640 1 -- 172.21.15.145:0/9391569 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe8cc003520 con 0x7fe8e80cf210 2024-05-09T04:34:24.837 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.835+0000 7fe8e7400640 1 -- 172.21.15.145:0/9391569 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 163582248 0 0) 0x7fe8cc003520 con 0x7fe8e80cf210 2024-05-09T04:34:24.837 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.835+0000 7fe8e7400640 1 -- 172.21.15.145:0/9391569 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8e80d06f0 msgr2=0x7fe8e806cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:24.837 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.835+0000 7fe8e7400640 1 --2- 172.21.15.145:0/9391569 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8e80d06f0 0x7fe8e806cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:24.837 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.835+0000 7fe8e7400640 1 -- 172.21.15.145:0/9391569 >> v2:172.21.15.192:3300/0 conn(0x7fe8e815d190 msgr2=0x7fe8e806c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:24.837 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.835+0000 7fe8e7400640 1 --2- 172.21.15.145:0/9391569 >> v2:172.21.15.192:3300/0 conn(0x7fe8e815d190 0x7fe8e806c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:24.837 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.835+0000 7fe8e7400640 1 -- 172.21.15.145:0/9391569 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8e80b5790 con 0x7fe8e80cf210 2024-05-09T04:34:24.837 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.835+0000 7fe8e7400640 1 -- 172.21.15.145:0/9391569 <== mon.0 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3474413261 0 0) 0x7fe8e0001a70 con 0x7fe8e80cf210 2024-05-09T04:34:24.837 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.835+0000 7fe8e7400640 1 -- 172.21.15.145:0/9391569 >> v1:172.21.15.145:6789/0 conn(0x7fe8e80cf210 legacy=0x7fe8e8072140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:24.837 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.836+0000 7fe8e7400640 1 --2- 172.21.15.145:0/9391569 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8e80d06f0 0x7fe8cc003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:24.837 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.836+0000 7fe8e7400640 1 --2- 172.21.15.145:0/9391569 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8cc003fd0 0x7fe8cc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:24.837 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.836+0000 7fe8e7400640 1 --2- 172.21.15.145:0/9391569 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe8cc004d40 0x7fe8cc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:24.837 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.836+0000 7fe8e7400640 1 -- 172.21.15.145:0/9391569 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e80b51d0 con 0x7fe8cc004d40 2024-05-09T04:34:24.838 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.836+0000 7fe8e7400640 1 -- 172.21.15.145:0/9391569 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e80b54b0 con 0x7fe8cc003fd0 2024-05-09T04:34:24.838 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.836+0000 7fe8e7400640 1 -- 172.21.15.145:0/9391569 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe8cc00bac0 con 0x7fe8e80d06f0 2024-05-09T04:34:24.838 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.836+0000 7fe8e7e00640 1 --2- 172.21.15.145:0/9391569 >> v2:172.21.15.192:3300/0 conn(0x7fe8e815d190 0x7fe8e806c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:34:24.838 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.836+0000 7fe8eca00640 1 --2- 172.21.15.145:0/9391569 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe8cc004d40 0x7fe8cc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:24.838 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.836+0000 7fe8df400640 1 --2- 172.21.15.145:0/9391569 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8cc003fd0 0x7fe8cc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:24.838 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.836+0000 7fe8eca00640 1 -- 172.21.15.145:0/9391569 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8e80d06f0 msgr2=0x7fe8cc003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:24.838 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.836+0000 7fe8eca00640 1 --2- 172.21.15.145:0/9391569 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8e80d06f0 0x7fe8cc003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:24.838 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.836+0000 7fe8eca00640 1 -- 172.21.15.145:0/9391569 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8cc003fd0 msgr2=0x7fe8cc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:24.838 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.836+0000 7fe8eca00640 1 --2- 172.21.15.145:0/9391569 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8cc003fd0 0x7fe8cc004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:24.838 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.836+0000 7fe8eca00640 1 -- 172.21.15.145:0/9391569 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8cc003520 con 0x7fe8cc004d40 2024-05-09T04:34:24.840 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.837+0000 7fe8eca00640 1 --2- 172.21.15.145:0/9391569 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe8cc004d40 0x7fe8cc0051c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe8e000a8d0 tx=0x7fe8cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a72e877d22eb299 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:24.840 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.838+0000 7fe8e7400640 1 -- 172.21.15.145:0/9391569 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e000a4c0 con 0x7fe8cc004d40 2024-05-09T04:34:24.840 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.838+0000 7fe8e7400640 1 -- 172.21.15.145:0/9391569 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8e002f070 con 0x7fe8cc004d40 2024-05-09T04:34:24.840 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.838+0000 7fe8e7400640 1 -- 172.21.15.145:0/9391569 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e002bc70 con 0x7fe8cc004d40 2024-05-09T04:34:24.840 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.838+0000 7fe8ed400640 1 -- 172.21.15.145:0/9391569 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe8cc004d40 msgr2=0x7fe8cc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:24.840 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.838+0000 7fe8ed400640 1 --2- 172.21.15.145:0/9391569 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe8cc004d40 0x7fe8cc0051c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe8e000a8d0 tx=0x7fe8cc005830 comp rx=0 tx=0).stop 2024-05-09T04:34:24.840 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.838+0000 7fe8eca00640 1 -- 172.21.15.145:0/9391569 reap_dead start 2024-05-09T04:34:24.840 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.838+0000 7fe8ed400640 1 -- 172.21.15.145:0/9391569 shutdown_connections 2024-05-09T04:34:24.840 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.838+0000 7fe8ed400640 1 -- 172.21.15.145:0/9391569 >> 172.21.15.145:0/9391569 conn(0x7fe8e80b2d80 msgr2=0x7fe8e80b3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:24.840 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.838+0000 7fe8ed400640 1 -- 172.21.15.145:0/9391569 shutdown_connections 2024-05-09T04:34:24.840 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.838+0000 7fe8ed400640 1 -- 172.21.15.145:0/9391569 wait complete. 2024-05-09T04:34:24.841 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.840+0000 7fe8ed400640 1 Processor -- start 2024-05-09T04:34:24.841 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.840+0000 7fe8ed400640 1 -- start start 2024-05-09T04:34:24.842 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.841+0000 7fe8ed400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe8cc004d40 0x7fe8e8151db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:24.843 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.841+0000 7fe8ed400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8e806f200 0x7fe8e806f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:24.843 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.841+0000 7fe8ed400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8e81549d0 0x7fe8e8154e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:24.843 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.841+0000 7fe8ed400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e80aed50 con 0x7fe8cc004d40 2024-05-09T04:34:24.843 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.841+0000 7fe8ed400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e80b4cd0 con 0x7fe8e81549d0 2024-05-09T04:34:24.843 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.841+0000 7fe8ed400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe8e80aeed0 con 0x7fe8e806f200 2024-05-09T04:34:24.843 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.842+0000 7fe8e7e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe8cc004d40 0x7fe8e8151db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:24.843 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.842+0000 7fe8e7e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe8cc004d40 0x7fe8e8151db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:40526/0 (socket says 172.21.15.145:40526) 2024-05-09T04:34:24.843 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.842+0000 7fe8e7e00640 1 -- 172.21.15.145:0/1545574030 learned_addr learned my addr 172.21.15.145:0/1545574030 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:24.843 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.842+0000 7fe8df400640 1 --2- 172.21.15.145:0/1545574030 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8e806f200 0x7fe8e806f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:24.843 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.842+0000 7fe8eca00640 1 --2- 172.21.15.145:0/1545574030 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8e81549d0 0x7fe8e8154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:24.843 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.842+0000 7fe8e7e00640 1 -- 172.21.15.145:0/1545574030 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8e806f200 msgr2=0x7fe8e806f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:24.843 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.842+0000 7fe8e7e00640 1 --2- 172.21.15.145:0/1545574030 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8e806f200 0x7fe8e806f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:24.843 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.842+0000 7fe8e7e00640 1 -- 172.21.15.145:0/1545574030 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8e81549d0 msgr2=0x7fe8e8154e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:24.843 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.842+0000 7fe8e7e00640 1 --2- 172.21.15.145:0/1545574030 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8e81549d0 0x7fe8e8154e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:24.844 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.842+0000 7fe8e7e00640 1 -- 172.21.15.145:0/1545574030 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8e806dec0 con 0x7fe8cc004d40 2024-05-09T04:34:24.844 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.842+0000 7fe8df400640 1 --2- 172.21.15.145:0/1545574030 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8e806f200 0x7fe8e806f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:34:24.844 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.843+0000 7fe8e7e00640 1 --2- 172.21.15.145:0/1545574030 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe8cc004d40 0x7fe8e8151db0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe8d800e7e0 tx=0x7fe8d800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:24.844 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.843+0000 7fe8e4c00640 1 -- 172.21.15.145:0/1545574030 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d800f040 con 0x7fe8cc004d40 2024-05-09T04:34:24.844 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.843+0000 7fe8e4c00640 1 -- 172.21.15.145:0/1545574030 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8d8004030 con 0x7fe8cc004d40 2024-05-09T04:34:24.844 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.843+0000 7fe8e4c00640 1 -- 172.21.15.145:0/1545574030 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d8004420 con 0x7fe8cc004d40 2024-05-09T04:34:24.844 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.843+0000 7fe8ed400640 1 -- 172.21.15.145:0/1545574030 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8e806e160 con 0x7fe8cc004d40 2024-05-09T04:34:24.844 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.843+0000 7fe8ed400640 1 -- 172.21.15.145:0/1545574030 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8e806e720 con 0x7fe8cc004d40 2024-05-09T04:34:24.844 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.844+0000 7fe8ed400640 1 -- 172.21.15.145:0/1545574030 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8e8076780 con 0x7fe8cc004d40 2024-05-09T04:34:24.845 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.845+0000 7fe8e4c00640 1 -- 172.21.15.145:0/1545574030 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe8d800c6d0 con 0x7fe8cc004d40 2024-05-09T04:34:24.846 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.847+0000 7fe8e4c00640 1 --2- 172.21.15.145:0/1545574030 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fe8c40762d0 0x7fe8c4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:24.847 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.847+0000 7fe8e4c00640 1 -- 172.21.15.145:0/1545574030 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe8d8097430 con 0x7fe8cc004d40 2024-05-09T04:34:24.847 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.847+0000 7fe8df400640 1 --2- 172.21.15.145:0/1545574030 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fe8c40762d0 0x7fe8c4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:24.847 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.848+0000 7fe8df400640 1 --2- 172.21.15.145:0/1545574030 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fe8c40762d0 0x7fe8c4078790 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe8e80b25f0 tx=0x7fe8d0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:24.851 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:24.852+0000 7fe8e4c00640 1 -- 172.21.15.145:0/1545574030 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe8d8017020 con 0x7fe8cc004d40 2024-05-09T04:34:25.072 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.072+0000 7fe8ed400640 1 -- 172.21.15.145:0/1545574030 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe8e80ce4b0 con 0x7fe8cc004d40 2024-05-09T04:34:25.073 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.073+0000 7fe8e4c00640 1 -- 172.21.15.145:0/1545574030 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7fe8d8060cf0 con 0x7fe8cc004d40 2024-05-09T04:34:25.073 INFO:teuthology.orchestra.run.smithi145.stderr:listed 0 entries 2024-05-09T04:34:25.076 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.076+0000 7fe8ed400640 1 -- 172.21.15.145:0/1545574030 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fe8c40762d0 msgr2=0x7fe8c4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:25.076 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.076+0000 7fe8ed400640 1 --2- 172.21.15.145:0/1545574030 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fe8c40762d0 0x7fe8c4078790 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe8e80b25f0 tx=0x7fe8d0008040 comp rx=0 tx=0).stop 2024-05-09T04:34:25.076 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.076+0000 7fe8ed400640 1 -- 172.21.15.145:0/1545574030 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe8cc004d40 msgr2=0x7fe8e8151db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:25.076 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.076+0000 7fe8ed400640 1 --2- 172.21.15.145:0/1545574030 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe8cc004d40 0x7fe8e8151db0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe8d800e7e0 tx=0x7fe8d800eca0 comp rx=0 tx=0).stop 2024-05-09T04:34:25.076 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.076+0000 7fe8ed400640 1 -- 172.21.15.145:0/1545574030 shutdown_connections 2024-05-09T04:34:25.076 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.076+0000 7fe8ed400640 1 --2- 172.21.15.145:0/1545574030 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fe8c40762d0 0x7fe8c4078790 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:25.076 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.076+0000 7fe8ed400640 1 --2- 172.21.15.145:0/1545574030 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8e81549d0 0x7fe8e8154e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:25.076 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.076+0000 7fe8ed400640 1 --2- 172.21.15.145:0/1545574030 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8e806f200 0x7fe8e806f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:25.076 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.076+0000 7fe8ed400640 1 --2- 172.21.15.145:0/1545574030 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe8cc004d40 0x7fe8e8151db0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:25.076 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.076+0000 7fe8ed400640 1 -- 172.21.15.145:0/1545574030 >> 172.21.15.145:0/1545574030 conn(0x7fe8e80b2d80 msgr2=0x7fe8e80d14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:25.076 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.076+0000 7fe8ed400640 1 -- 172.21.15.145:0/1545574030 shutdown_connections 2024-05-09T04:34:25.077 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.076+0000 7fe8ed400640 1 -- 172.21.15.145:0/1545574030 wait complete. 2024-05-09T04:34:25.094 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi145.front.sepia.ceph.com... 2024-05-09T04:34:25.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:25.094 DEBUG:teuthology.orchestra.run.smithi145:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-09T04:34:25.173 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T04:34:25.173 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi145.front.sepia.ceph.com 2024-05-09T04:34:25.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:25.173 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-09T04:34:25.231 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-09T04:34:25.231 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T04:34:25.290 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-09T04:34:25.290 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T04:34:25.343 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T04:34:25.545 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.545+0000 7f44d6a00640 1 Processor -- start 2024-05-09T04:34:25.545 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.545+0000 7f44d6a00640 1 -- start start 2024-05-09T04:34:25.547 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.547+0000 7f44d6a00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f44d015f4e0 0x7f44d016a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:25.547 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.547+0000 7f44d6a00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44d00cfda0 con 0x7f44d00d3cd0 2024-05-09T04:34:25.547 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.547+0000 7f44d6a00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44d00d0080 con 0x7f44d015f100 2024-05-09T04:34:25.547 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.547+0000 7f44d6a00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f44d00b8e70 con 0x7f44d015f4e0 2024-05-09T04:34:25.547 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.547+0000 7f44d4c00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f44d00d3cd0 0x7f44d00d1950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:44856/0 (socket says 172.21.15.145:44856) 2024-05-09T04:34:25.547 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.547+0000 7f44d4c00640 1 -- 172.21.15.145:0/2175762723 learned_addr learned my addr 172.21.15.145:0/2175762723 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:34:25.549 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.548+0000 7f44cfe00640 1 -- 172.21.15.145:0/2175762723 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1794974697 0 0) 0x7f44d00cfda0 con 0x7f44d00d3cd0 2024-05-09T04:34:25.549 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.548+0000 7f44cfe00640 1 -- 172.21.15.145:0/2175762723 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f44b4003520 con 0x7f44d00d3cd0 2024-05-09T04:34:25.549 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.548+0000 7f44d6000640 1 --2- 172.21.15.145:0/2175762723 >> v2:172.21.15.192:3300/0 conn(0x7f44d015f4e0 0x7f44d016a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:25.549 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.548+0000 7f44cfe00640 1 -- 172.21.15.145:0/2175762723 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3501984238 0 0) 0x7f44b4003520 con 0x7f44d00d3cd0 2024-05-09T04:34:25.550 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.548+0000 7f44cfe00640 1 -- 172.21.15.145:0/2175762723 >> v2:172.21.15.192:3300/0 conn(0x7f44d015f4e0 msgr2=0x7f44d016a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:25.550 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.548+0000 7f44cfe00640 1 --2- 172.21.15.145:0/2175762723 >> v2:172.21.15.192:3300/0 conn(0x7f44d015f4e0 0x7f44d016a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:25.550 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.548+0000 7f44cfe00640 1 -- 172.21.15.145:0/2175762723 >> v1:172.21.15.192:6789/0 conn(0x7f44d015f100 legacy=0x7f44d00d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:25.550 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.548+0000 7f44cfe00640 1 -- 172.21.15.145:0/2175762723 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44d00d0360 con 0x7f44d00d3cd0 2024-05-09T04:34:25.550 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.548+0000 7f44cfe00640 1 -- 172.21.15.145:0/2175762723 <== mon.0 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3474413261 0 0) 0x7f44bc001a70 con 0x7f44d00d3cd0 2024-05-09T04:34:25.550 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.548+0000 7f44cfe00640 1 -- 172.21.15.145:0/2175762723 >> v1:172.21.15.145:6789/0 conn(0x7f44d00d3cd0 legacy=0x7f44d00d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:25.550 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.548+0000 7f44cfe00640 1 --2- 172.21.15.145:0/2175762723 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f44b4003a30 0x7f44b4003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:25.550 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.548+0000 7f44cfe00640 1 --2- 172.21.15.145:0/2175762723 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44b4004800 0x7f44b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:25.550 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.548+0000 7f44cfe00640 1 --2- 172.21.15.145:0/2175762723 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f44b4005230 0x7f44b400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:25.550 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.548+0000 7f44cfe00640 1 -- 172.21.15.145:0/2175762723 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f44d00b8e70 con 0x7f44b4003a30 2024-05-09T04:34:25.550 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.548+0000 7f44cfe00640 1 -- 172.21.15.145:0/2175762723 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f44b400c190 con 0x7f44b4004800 2024-05-09T04:34:25.550 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.548+0000 7f44cfe00640 1 -- 172.21.15.145:0/2175762723 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f44b400c390 con 0x7f44b4005230 2024-05-09T04:34:25.550 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.549+0000 7f44d5600640 1 --2- 172.21.15.145:0/2175762723 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f44b4003a30 0x7f44b4003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:25.551 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.549+0000 7f44d4c00640 1 --2- 172.21.15.145:0/2175762723 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44b4004800 0x7f44b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:25.551 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.549+0000 7f44d5600640 1 -- 172.21.15.145:0/2175762723 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f44b4005230 msgr2=0x7f44b400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:25.551 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.549+0000 7f44d5600640 1 --2- 172.21.15.145:0/2175762723 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f44b4005230 0x7f44b400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:25.551 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.549+0000 7f44d5600640 1 -- 172.21.15.145:0/2175762723 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44b4004800 msgr2=0x7f44b4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:25.551 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.549+0000 7f44d5600640 1 --2- 172.21.15.145:0/2175762723 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44b4004800 0x7f44b4004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:25.551 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.549+0000 7f44d5600640 1 -- 172.21.15.145:0/2175762723 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44b4003520 con 0x7f44b4003a30 2024-05-09T04:34:25.551 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.549+0000 7f44d6000640 1 -- 172.21.15.145:0/2175762723 reap_dead start 2024-05-09T04:34:25.552 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.550+0000 7f44d5600640 1 --2- 172.21.15.145:0/2175762723 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f44b4003a30 0x7f44b4003e30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f44c0003db0 tx=0x7f44b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18cc012e0f933c62 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:25.552 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.550+0000 7f44cfe00640 1 -- 172.21.15.145:0/2175762723 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c0037070 con 0x7f44b4003a30 2024-05-09T04:34:25.552 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.550+0000 7f44cfe00640 1 -- 172.21.15.145:0/2175762723 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44c002a930 con 0x7f44b4003a30 2024-05-09T04:34:25.552 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.550+0000 7f44cfe00640 1 -- 172.21.15.145:0/2175762723 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c0003980 con 0x7f44b4003a30 2024-05-09T04:34:25.552 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.551+0000 7f44d6a00640 1 -- 172.21.15.145:0/2175762723 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f44b4003a30 msgr2=0x7f44b4003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:25.552 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.551+0000 7f44d6a00640 1 --2- 172.21.15.145:0/2175762723 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f44b4003a30 0x7f44b4003e30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f44c0003db0 tx=0x7f44b4005830 comp rx=0 tx=0).stop 2024-05-09T04:34:25.552 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.551+0000 7f44d6a00640 1 -- 172.21.15.145:0/2175762723 shutdown_connections 2024-05-09T04:34:25.553 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.551+0000 7f44d6a00640 1 --2- 172.21.15.145:0/2175762723 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f44b4003a30 0x7f44b4003e30 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:25.553 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.551+0000 7f44d6a00640 1 -- 172.21.15.145:0/2175762723 >> 172.21.15.145:0/2175762723 conn(0x7f44d009e2e0 msgr2=0x7f44d00d5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:25.553 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.551+0000 7f44d6a00640 1 -- 172.21.15.145:0/2175762723 shutdown_connections 2024-05-09T04:34:25.553 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.551+0000 7f44d6a00640 1 -- 172.21.15.145:0/2175762723 wait complete. 2024-05-09T04:34:25.554 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.553+0000 7f44d6a00640 1 Processor -- start 2024-05-09T04:34:25.554 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.553+0000 7f44d6a00640 1 -- start start 2024-05-09T04:34:25.554 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.554+0000 7f44d6a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f44b4003a30 0x7f44d0156220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:25.554 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.554+0000 7f44d6a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f44d0160940 0x7f44d0160dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:25.554 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.554+0000 7f44d6a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44d0161300 0x7f44d00c9c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:25.554 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.554+0000 7f44d6a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f44d00d0e40 con 0x7f44d0160940 2024-05-09T04:34:25.554 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.554+0000 7f44d6a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f44d00d0920 con 0x7f44d0161300 2024-05-09T04:34:25.554 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.554+0000 7f44d6a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f44d0156760 con 0x7f44b4003a30 2024-05-09T04:34:25.554 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.554+0000 7f44d4c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f44d0160940 0x7f44d0160dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:25.555 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.554+0000 7f44d4c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f44d0160940 0x7f44d0160dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:40542/0 (socket says 172.21.15.145:40542) 2024-05-09T04:34:25.555 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.554+0000 7f44d4c00640 1 -- 172.21.15.145:0/4183315276 learned_addr learned my addr 172.21.15.145:0/4183315276 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:25.556 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.555+0000 7f44d6000640 1 --2- 172.21.15.145:0/4183315276 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44d0161300 0x7f44d00c9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:25.556 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.555+0000 7f44d5600640 1 --2- 172.21.15.145:0/4183315276 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f44b4003a30 0x7f44d0156220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:25.556 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.555+0000 7f44d4c00640 1 -- 172.21.15.145:0/4183315276 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f44b4003a30 msgr2=0x7f44d0156220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:25.556 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.555+0000 7f44d4c00640 1 --2- 172.21.15.145:0/4183315276 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f44b4003a30 0x7f44d0156220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:25.556 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.555+0000 7f44d4c00640 1 -- 172.21.15.145:0/4183315276 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44d0161300 msgr2=0x7f44d00c9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:25.556 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.555+0000 7f44d4c00640 1 --2- 172.21.15.145:0/4183315276 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44d0161300 0x7f44d00c9c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:25.556 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.555+0000 7f44d4c00640 1 -- 172.21.15.145:0/4183315276 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44d0161780 con 0x7f44d0160940 2024-05-09T04:34:25.556 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.555+0000 7f44d4c00640 1 --2- 172.21.15.145:0/4183315276 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f44d0160940 0x7f44d0160dc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f44bc00b100 tx=0x7f44bc00b5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:25.556 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.555+0000 7f44cd600640 1 -- 172.21.15.145:0/4183315276 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44bc004d60 con 0x7f44d0160940 2024-05-09T04:34:25.556 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.555+0000 7f44cd600640 1 -- 172.21.15.145:0/4183315276 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44bc007110 con 0x7f44d0160940 2024-05-09T04:34:25.557 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.555+0000 7f44d6a00640 1 -- 172.21.15.145:0/4183315276 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44d00c8b80 con 0x7f44d0160940 2024-05-09T04:34:25.557 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.555+0000 7f44cd600640 1 -- 172.21.15.145:0/4183315276 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44bc0074c0 con 0x7f44d0160940 2024-05-09T04:34:25.557 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.556+0000 7f44d6a00640 1 -- 172.21.15.145:0/4183315276 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44d00c9140 con 0x7f44d0160940 2024-05-09T04:34:25.557 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.557+0000 7f44d6a00640 1 -- 172.21.15.145:0/4183315276 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44d0076780 con 0x7f44d0160940 2024-05-09T04:34:25.557 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.558+0000 7f44cd600640 1 -- 172.21.15.145:0/4183315276 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f44bc007660 con 0x7f44d0160940 2024-05-09T04:34:25.559 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.559+0000 7f44cd600640 1 --2- 172.21.15.145:0/4183315276 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f44b0076300 0x7f44b00787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:25.559 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.559+0000 7f44cd600640 1 -- 172.21.15.145:0/4183315276 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f44bc097a80 con 0x7f44d0160940 2024-05-09T04:34:25.559 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.559+0000 7f44d5600640 1 --2- 172.21.15.145:0/4183315276 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f44b0076300 0x7f44b00787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:25.559 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.560+0000 7f44d5600640 1 --2- 172.21.15.145:0/4183315276 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f44b0076300 0x7f44b00787c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f44c0034000 tx=0x7f44c002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:25.562 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.563+0000 7f44cd600640 1 -- 172.21.15.145:0/4183315276 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f44bc099e20 con 0x7f44d0160940 2024-05-09T04:34:25.783 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.783+0000 7f44d6a00640 1 -- 172.21.15.145:0/4183315276 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f44d00d0fc0 con 0x7f44d0160940 2024-05-09T04:34:25.783 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.784+0000 7f44cd600640 1 -- 172.21.15.145:0/4183315276 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7f44bc061340 con 0x7f44d0160940 2024-05-09T04:34:25.783 INFO:teuthology.orchestra.run.smithi145.stderr:listed 0 entries 2024-05-09T04:34:25.787 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.786+0000 7f44d6a00640 1 -- 172.21.15.145:0/4183315276 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f44b0076300 msgr2=0x7f44b00787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:25.787 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.786+0000 7f44d6a00640 1 --2- 172.21.15.145:0/4183315276 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f44b0076300 0x7f44b00787c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f44c0034000 tx=0x7f44c002e000 comp rx=0 tx=0).stop 2024-05-09T04:34:25.787 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.786+0000 7f44d6a00640 1 -- 172.21.15.145:0/4183315276 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f44d0160940 msgr2=0x7f44d0160dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:25.787 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.786+0000 7f44d6a00640 1 --2- 172.21.15.145:0/4183315276 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f44d0160940 0x7f44d0160dc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f44bc00b100 tx=0x7f44bc00b5c0 comp rx=0 tx=0).stop 2024-05-09T04:34:25.787 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.787+0000 7f44d6a00640 1 -- 172.21.15.145:0/4183315276 shutdown_connections 2024-05-09T04:34:25.787 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.787+0000 7f44d6a00640 1 --2- 172.21.15.145:0/4183315276 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f44b0076300 0x7f44b00787c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:25.787 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.787+0000 7f44d6a00640 1 --2- 172.21.15.145:0/4183315276 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44d0161300 0x7f44d00c9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:25.787 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.787+0000 7f44d6a00640 1 --2- 172.21.15.145:0/4183315276 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f44d0160940 0x7f44d0160dc0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:25.787 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.787+0000 7f44d6a00640 1 --2- 172.21.15.145:0/4183315276 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f44b4003a30 0x7f44d0156220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:25.787 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.787+0000 7f44d6a00640 1 -- 172.21.15.145:0/4183315276 >> 172.21.15.145:0/4183315276 conn(0x7f44d009e2e0 msgr2=0x7f44d00d11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:25.787 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.787+0000 7f44d6a00640 1 -- 172.21.15.145:0/4183315276 shutdown_connections 2024-05-09T04:34:25.787 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:25.787+0000 7f44d6a00640 1 -- 172.21.15.145:0/4183315276 wait complete. 2024-05-09T04:34:25.806 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-09T04:34:25.806 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T04:34:25.862 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T04:34:26.065 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.065+0000 7fafe7600640 1 Processor -- start 2024-05-09T04:34:26.066 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.066+0000 7fafe7600640 1 -- start start 2024-05-09T04:34:26.067 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.067+0000 7fafe7600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafe00d22e0 0x7fafe00cfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:26.067 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.067+0000 7fafe7600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fafe00d2db0 0x7fafe00d0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:26.068 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.067+0000 7fafe7600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafe00b4b70 con 0x7fafe015f5f0 2024-05-09T04:34:26.068 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.067+0000 7fafe7600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fafe015f9d0 con 0x7fafe00d2db0 2024-05-09T04:34:26.068 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.067+0000 7fafe7600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fafe00b4e50 con 0x7fafe00d22e0 2024-05-09T04:34:26.068 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.068+0000 7fafe4c00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7fafe015f5f0 0x7fafe00d1bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:44868/0 (socket says 172.21.15.145:44868) 2024-05-09T04:34:26.068 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.068+0000 7fafe4c00640 1 -- 172.21.15.145:0/3032227721 learned_addr learned my addr 172.21.15.145:0/3032227721 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:34:26.069 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.068+0000 7fafdf400640 1 -- 172.21.15.145:0/3032227721 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 113796328 0 0) 0x7fafe00b4b70 con 0x7fafe015f5f0 2024-05-09T04:34:26.070 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.068+0000 7fafdf400640 1 -- 172.21.15.145:0/3032227721 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fafc4003520 con 0x7fafe015f5f0 2024-05-09T04:34:26.070 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.068+0000 7fafe5600640 1 --2- 172.21.15.145:0/3032227721 >> v2:172.21.15.192:3300/0 conn(0x7fafe00d2db0 0x7fafe00d0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:26.070 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.068+0000 7fafdfe00640 1 --2- 172.21.15.145:0/3032227721 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafe00d22e0 0x7fafe00cfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:26.070 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.069+0000 7fafdf400640 1 -- 172.21.15.145:0/3032227721 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1354762888 0 0) 0x7fafc4003520 con 0x7fafe015f5f0 2024-05-09T04:34:26.070 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.069+0000 7fafdf400640 1 -- 172.21.15.145:0/3032227721 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafe00d22e0 msgr2=0x7fafe00cfcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:26.070 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.069+0000 7fafdf400640 1 --2- 172.21.15.145:0/3032227721 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafe00d22e0 0x7fafe00cfcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:26.070 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.069+0000 7fafdf400640 1 -- 172.21.15.145:0/3032227721 >> v2:172.21.15.192:3300/0 conn(0x7fafe00d2db0 msgr2=0x7fafe00d0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:26.070 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.069+0000 7fafdf400640 1 --2- 172.21.15.145:0/3032227721 >> v2:172.21.15.192:3300/0 conn(0x7fafe00d2db0 0x7fafe00d0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:26.070 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.069+0000 7fafdf400640 1 -- 172.21.15.145:0/3032227721 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafe00b5130 con 0x7fafe015f5f0 2024-05-09T04:34:26.070 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.069+0000 7fafdf400640 1 -- 172.21.15.145:0/3032227721 <== mon.0 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3474413261 0 0) 0x7fafd80016f0 con 0x7fafe015f5f0 2024-05-09T04:34:26.070 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.069+0000 7fafdf400640 1 -- 172.21.15.145:0/3032227721 >> v1:172.21.15.145:6789/0 conn(0x7fafe015f5f0 legacy=0x7fafe00d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:26.070 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.069+0000 7fafdf400640 1 --2- 172.21.15.145:0/3032227721 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafc4003a50 0x7fafc4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:26.070 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.069+0000 7fafdf400640 1 --2- 172.21.15.145:0/3032227721 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafe00d22e0 0x7fafc4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:26.070 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.069+0000 7fafdf400640 1 --2- 172.21.15.145:0/3032227721 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fafc4004d40 0x7fafc40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:26.071 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.069+0000 7fafdf400640 1 -- 172.21.15.145:0/3032227721 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fafe015f9d0 con 0x7fafc4004d40 2024-05-09T04:34:26.071 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.069+0000 7fafdf400640 1 -- 172.21.15.145:0/3032227721 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fafe00b4e50 con 0x7fafc4003a50 2024-05-09T04:34:26.071 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.069+0000 7fafdf400640 1 -- 172.21.15.145:0/3032227721 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fafc400bba0 con 0x7fafe00d22e0 2024-05-09T04:34:26.071 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.069+0000 7fafe5600640 1 --2- 172.21.15.145:0/3032227721 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fafc4004d40 0x7fafc40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:26.071 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.069+0000 7fafe4c00640 1 --2- 172.21.15.145:0/3032227721 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafc4003a50 0x7fafc4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:26.071 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.069+0000 7fafe5600640 1 -- 172.21.15.145:0/3032227721 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafe00d22e0 msgr2=0x7fafc4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:26.071 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.069+0000 7fafe5600640 1 --2- 172.21.15.145:0/3032227721 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafe00d22e0 0x7fafc4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:26.071 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.069+0000 7fafe5600640 1 -- 172.21.15.145:0/3032227721 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafc4003a50 msgr2=0x7fafc4003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:26.071 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.069+0000 7fafe5600640 1 --2- 172.21.15.145:0/3032227721 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafc4003a50 0x7fafc4003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:26.071 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.069+0000 7fafe5600640 1 -- 172.21.15.145:0/3032227721 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafc4003520 con 0x7fafc4004d40 2024-05-09T04:34:26.071 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.070+0000 7fafe4c00640 1 --2- 172.21.15.145:0/3032227721 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafc4003a50 0x7fafc4003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:34:26.072 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.071+0000 7fafe5600640 1 --2- 172.21.15.145:0/3032227721 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fafc4004d40 0x7fafc40051c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fafd000ba30 tx=0x7fafc4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8d05cd88bc17170 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:26.072 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.071+0000 7fafdf400640 1 -- 172.21.15.145:0/3032227721 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd0030070 con 0x7fafc4004d40 2024-05-09T04:34:26.073 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.071+0000 7fafdf400640 1 -- 172.21.15.145:0/3032227721 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafd002ae10 con 0x7fafc4004d40 2024-05-09T04:34:26.073 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.071+0000 7fafdf400640 1 -- 172.21.15.145:0/3032227721 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd000b600 con 0x7fafc4004d40 2024-05-09T04:34:26.073 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.071+0000 7fafe7600640 1 -- 172.21.15.145:0/3032227721 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fafc4004d40 msgr2=0x7fafc40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:26.073 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.071+0000 7fafe7600640 1 --2- 172.21.15.145:0/3032227721 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fafc4004d40 0x7fafc40051c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fafd000ba30 tx=0x7fafc4005830 comp rx=0 tx=0).stop 2024-05-09T04:34:26.073 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.071+0000 7fafe5600640 1 -- 172.21.15.145:0/3032227721 reap_dead start 2024-05-09T04:34:26.073 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.072+0000 7fafe7600640 1 -- 172.21.15.145:0/3032227721 shutdown_connections 2024-05-09T04:34:26.073 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.072+0000 7fafe7600640 1 -- 172.21.15.145:0/3032227721 >> 172.21.15.145:0/3032227721 conn(0x7fafe009e520 msgr2=0x7fafe00d51e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:26.073 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.072+0000 7fafe7600640 1 -- 172.21.15.145:0/3032227721 shutdown_connections 2024-05-09T04:34:26.073 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.072+0000 7fafe7600640 1 -- 172.21.15.145:0/3032227721 wait complete. 2024-05-09T04:34:26.074 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.073+0000 7fafe7600640 1 Processor -- start 2024-05-09T04:34:26.074 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.074+0000 7fafe7600640 1 -- start start 2024-05-09T04:34:26.074 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.074+0000 7fafe7600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fafc4004d40 0x7fafe00ba780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:26.074 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.074+0000 7fafe7600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafe015e210 0x7fafe0161b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:26.075 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.074+0000 7fafe7600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafe00ce350 0x7fafe01620a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:26.075 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.074+0000 7fafe7600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fafe00d3c90 con 0x7fafc4004d40 2024-05-09T04:34:26.075 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.074+0000 7fafe7600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fafe00d4040 con 0x7fafe015e210 2024-05-09T04:34:26.075 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.074+0000 7fafe7600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fafe00ce8d0 con 0x7fafe00ce350 2024-05-09T04:34:26.075 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.075+0000 7fafe4c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fafc4004d40 0x7fafe00ba780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:26.075 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.075+0000 7fafe4c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fafc4004d40 0x7fafe00ba780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:40560/0 (socket says 172.21.15.145:40560) 2024-05-09T04:34:26.075 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.075+0000 7fafe4c00640 1 -- 172.21.15.145:0/3877666267 learned_addr learned my addr 172.21.15.145:0/3877666267 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:26.076 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.075+0000 7fafdfe00640 1 --2- 172.21.15.145:0/3877666267 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafe015e210 0x7fafe0161b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:26.076 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.075+0000 7fafe4c00640 1 -- 172.21.15.145:0/3877666267 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafe00ce350 msgr2=0x7fafe01620a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:26.076 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.075+0000 7fafe4c00640 1 --2- 172.21.15.145:0/3877666267 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafe00ce350 0x7fafe01620a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:26.076 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.075+0000 7fafe4c00640 1 -- 172.21.15.145:0/3877666267 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafe015e210 msgr2=0x7fafe0161b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:26.076 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.075+0000 7fafe4c00640 1 --2- 172.21.15.145:0/3877666267 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafe015e210 0x7fafe0161b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:26.076 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.075+0000 7fafe4c00640 1 -- 172.21.15.145:0/3877666267 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafe00cebb0 con 0x7fafc4004d40 2024-05-09T04:34:26.076 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.075+0000 7fafe4c00640 1 --2- 172.21.15.145:0/3877666267 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fafc4004d40 0x7fafe00ba780 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fafd800d890 tx=0x7fafd800dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:26.076 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.076+0000 7fafdcc00640 1 -- 172.21.15.145:0/3877666267 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd80074e0 con 0x7fafc4004d40 2024-05-09T04:34:26.077 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.076+0000 7fafdcc00640 1 -- 172.21.15.145:0/3877666267 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafd8007680 con 0x7fafc4004d40 2024-05-09T04:34:26.077 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.076+0000 7fafe7600640 1 -- 172.21.15.145:0/3877666267 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafe00cee90 con 0x7fafc4004d40 2024-05-09T04:34:26.077 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.076+0000 7fafdcc00640 1 -- 172.21.15.145:0/3877666267 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd80049e0 con 0x7fafc4004d40 2024-05-09T04:34:26.077 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.076+0000 7fafe7600640 1 -- 172.21.15.145:0/3877666267 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafe00bacc0 con 0x7fafc4004d40 2024-05-09T04:34:26.077 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.077+0000 7fafe7600640 1 -- 172.21.15.145:0/3877666267 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafe006a310 con 0x7fafc4004d40 2024-05-09T04:34:26.078 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.078+0000 7fafdcc00640 1 -- 172.21.15.145:0/3877666267 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fafd801b020 con 0x7fafc4004d40 2024-05-09T04:34:26.079 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.079+0000 7fafdcc00640 1 --2- 172.21.15.145:0/3877666267 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fafbc0762d0 0x7fafbc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:26.079 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.079+0000 7fafdcc00640 1 -- 172.21.15.145:0/3877666267 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fafd80975f0 con 0x7fafc4004d40 2024-05-09T04:34:26.079 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.080+0000 7fafdfe00640 1 --2- 172.21.15.145:0/3877666267 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fafbc0762d0 0x7fafbc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:26.080 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.080+0000 7fafdfe00640 1 --2- 172.21.15.145:0/3877666267 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fafbc0762d0 0x7fafbc078790 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fafe006ed00 tx=0x7fafc800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:26.084 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.084+0000 7fafdcc00640 1 -- 172.21.15.145:0/3877666267 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fafd8060eb0 con 0x7fafc4004d40 2024-05-09T04:34:26.304 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.304+0000 7fafe7600640 1 -- 172.21.15.145:0/3877666267 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fafe006ded0 con 0x7fafc4004d40 2024-05-09T04:34:26.305 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.305+0000 7fafdcc00640 1 -- 172.21.15.145:0/3877666267 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7fafd80607b0 con 0x7fafc4004d40 2024-05-09T04:34:26.305 INFO:teuthology.orchestra.run.smithi145.stderr:listed 0 entries 2024-05-09T04:34:26.307 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.308+0000 7fafe7600640 1 -- 172.21.15.145:0/3877666267 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fafbc0762d0 msgr2=0x7fafbc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:26.308 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.308+0000 7fafe7600640 1 --2- 172.21.15.145:0/3877666267 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fafbc0762d0 0x7fafbc078790 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fafe006ed00 tx=0x7fafc800b040 comp rx=0 tx=0).stop 2024-05-09T04:34:26.308 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.308+0000 7fafe7600640 1 -- 172.21.15.145:0/3877666267 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fafc4004d40 msgr2=0x7fafe00ba780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:26.308 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.308+0000 7fafe7600640 1 --2- 172.21.15.145:0/3877666267 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fafc4004d40 0x7fafe00ba780 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fafd800d890 tx=0x7fafd800dd50 comp rx=0 tx=0).stop 2024-05-09T04:34:26.308 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.308+0000 7fafe7600640 1 -- 172.21.15.145:0/3877666267 shutdown_connections 2024-05-09T04:34:26.308 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.308+0000 7fafe7600640 1 --2- 172.21.15.145:0/3877666267 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fafbc0762d0 0x7fafbc078790 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:26.309 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.308+0000 7fafe7600640 1 --2- 172.21.15.145:0/3877666267 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafe00ce350 0x7fafe01620a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:26.309 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.308+0000 7fafe7600640 1 --2- 172.21.15.145:0/3877666267 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafe015e210 0x7fafe0161b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:26.309 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.308+0000 7fafe7600640 1 --2- 172.21.15.145:0/3877666267 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fafc4004d40 0x7fafe00ba780 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:26.309 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.308+0000 7fafe7600640 1 -- 172.21.15.145:0/3877666267 >> 172.21.15.145:0/3877666267 conn(0x7fafe009e520 msgr2=0x7fafe00b5830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:26.309 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.308+0000 7fafe7600640 1 -- 172.21.15.145:0/3877666267 shutdown_connections 2024-05-09T04:34:26.309 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.308+0000 7fafe7600640 1 -- 172.21.15.145:0/3877666267 wait complete. 2024-05-09T04:34:26.328 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi192.front.sepia.ceph.com... 2024-05-09T04:34:26.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:26.328 DEBUG:teuthology.orchestra.run.smithi192:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-05-09T04:34:26.416 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T04:34:26.417 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi192.front.sepia.ceph.com 2024-05-09T04:34:26.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:26.417 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-09T04:34:26.472 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-05-09T04:34:26.472 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T04:34:26.523 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-05-09T04:34:26.523 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T04:34:26.617 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T04:34:26.803 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.803+0000 7ff6fb200640 1 Processor -- start 2024-05-09T04:34:26.803 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.803+0000 7ff6fb200640 1 -- start start 2024-05-09T04:34:26.805 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.805+0000 7ff6fb200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff6fc15d190 0x7ff6fc06c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:26.805 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.805+0000 7ff6fb200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff6fc0aed50 0x7ff6fc06cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:26.805 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.805+0000 7ff6fb200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6fc0b4ef0 con 0x7ff6fc0d06f0 2024-05-09T04:34:26.805 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.805+0000 7ff6fb200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7ff6fc0b51d0 con 0x7ff6fc0aed50 2024-05-09T04:34:26.805 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.805+0000 7ff6fb200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff6fc0b54b0 con 0x7ff6fc15d190 2024-05-09T04:34:26.805 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.805+0000 7ff6fa800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff6fc0aed50 0x7ff6fc06cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:26.805 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.805+0000 7ff6fa800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff6fc0aed50 0x7ff6fc06cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:40572/0 (socket says 172.21.15.145:40572) 2024-05-09T04:34:26.805 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.805+0000 7ff6fa800640 1 -- 172.21.15.145:0/980325737 learned_addr learned my addr 172.21.15.145:0/980325737 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:26.806 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.806+0000 7ff6f9e00640 1 --2- 172.21.15.145:0/980325737 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff6fc15d190 0x7ff6fc06c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:26.806 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.806+0000 7ff6fa800640 1 -- 172.21.15.145:0/980325737 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff6fc15d190 msgr2=0x7ff6fc06c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:26.806 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.806+0000 7ff6fa800640 1 --2- 172.21.15.145:0/980325737 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff6fc15d190 0x7ff6fc06c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:26.806 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.806+0000 7ff6fa800640 1 -- 172.21.15.145:0/980325737 >> v1:172.21.15.192:6789/0 conn(0x7ff6fc0d06f0 legacy=0x7ff6fc072140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:26.806 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.806+0000 7ff6fa800640 1 -- 172.21.15.145:0/980325737 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6fc0b5790 con 0x7ff6fc0aed50 2024-05-09T04:34:26.807 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.808+0000 7ff6fa800640 1 --2- 172.21.15.145:0/980325737 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff6fc0aed50 0x7ff6fc06cde0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff6f4009870 tx=0x7ff6f402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b23d74d0f88f056 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:26.807 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.808+0000 7ff6f9400640 1 -- 172.21.15.145:0/980325737 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f4037070 con 0x7ff6fc0aed50 2024-05-09T04:34:26.808 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.808+0000 7ff6f9400640 1 -- 172.21.15.145:0/980325737 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6f4032440 con 0x7ff6fc0aed50 2024-05-09T04:34:26.808 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.808+0000 7ff6f9400640 1 -- 172.21.15.145:0/980325737 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f4032810 con 0x7ff6fc0aed50 2024-05-09T04:34:26.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.809+0000 7ff6fb200640 1 -- 172.21.15.145:0/980325737 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff6fc0aed50 msgr2=0x7ff6fc06cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:26.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.809+0000 7ff6fb200640 1 --2- 172.21.15.145:0/980325737 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff6fc0aed50 0x7ff6fc06cde0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff6f4009870 tx=0x7ff6f402a1c0 comp rx=0 tx=0).stop 2024-05-09T04:34:26.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.809+0000 7ff6fb200640 1 -- 172.21.15.145:0/980325737 shutdown_connections 2024-05-09T04:34:26.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.809+0000 7ff6fb200640 1 --2- 172.21.15.145:0/980325737 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff6fc0aed50 0x7ff6fc06cde0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:26.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.809+0000 7ff6fb200640 1 --2- 172.21.15.145:0/980325737 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff6fc15d190 0x7ff6fc06c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:26.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.809+0000 7ff6fb200640 1 -- 172.21.15.145:0/980325737 >> 172.21.15.145:0/980325737 conn(0x7ff6fc0b2d80 msgr2=0x7ff6fc0b3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:26.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.809+0000 7ff6fb200640 1 -- 172.21.15.145:0/980325737 shutdown_connections 2024-05-09T04:34:26.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.809+0000 7ff6fb200640 1 -- 172.21.15.145:0/980325737 wait complete. 2024-05-09T04:34:26.810 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.811+0000 7ff6fb200640 1 Processor -- start 2024-05-09T04:34:26.810 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.811+0000 7ff6fb200640 1 -- start start 2024-05-09T04:34:26.812 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.812+0000 7ff6fb200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff6fc0aed50 0x7ff6fc153680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:26.812 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.812+0000 7ff6fb200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff6fc0d06f0 0x7ff6fc0d0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:26.812 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.812+0000 7ff6fb200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff6fc15d190 0x7ff6fc0d10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:26.812 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.812+0000 7ff6fb200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff6fc0cf210 con 0x7ff6fc0d06f0 2024-05-09T04:34:26.812 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.812+0000 7ff6fb200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff6fc0b4cd0 con 0x7ff6fc15d190 2024-05-09T04:34:26.812 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.812+0000 7ff6fb200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff6fc0cf390 con 0x7ff6fc0aed50 2024-05-09T04:34:26.813 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.812+0000 7ff6f1400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff6fc0d06f0 0x7ff6fc0d0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:26.813 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.812+0000 7ff6f1400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff6fc0d06f0 0x7ff6fc0d0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:40582/0 (socket says 172.21.15.145:40582) 2024-05-09T04:34:26.813 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.812+0000 7ff6f1400640 1 -- 172.21.15.145:0/3449638461 learned_addr learned my addr 172.21.15.145:0/3449638461 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:26.813 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.812+0000 7ff6fa800640 1 --2- 172.21.15.145:0/3449638461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff6fc15d190 0x7ff6fc0d10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:26.813 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.812+0000 7ff6f9e00640 1 --2- 172.21.15.145:0/3449638461 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff6fc0aed50 0x7ff6fc153680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:26.813 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.812+0000 7ff6f1400640 1 -- 172.21.15.145:0/3449638461 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff6fc0aed50 msgr2=0x7ff6fc153680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:26.813 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.812+0000 7ff6f1400640 1 --2- 172.21.15.145:0/3449638461 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff6fc0aed50 0x7ff6fc153680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:26.813 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.812+0000 7ff6f1400640 1 -- 172.21.15.145:0/3449638461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff6fc15d190 msgr2=0x7ff6fc0d10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:26.813 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.812+0000 7ff6f1400640 1 --2- 172.21.15.145:0/3449638461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff6fc15d190 0x7ff6fc0d10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:26.813 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.812+0000 7ff6f1400640 1 -- 172.21.15.145:0/3449638461 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6fc15e660 con 0x7ff6fc0d06f0 2024-05-09T04:34:26.813 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.813+0000 7ff6f1400640 1 --2- 172.21.15.145:0/3449638461 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff6fc0d06f0 0x7ff6fc0d0bb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff6e400c430 tx=0x7ff6e400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:26.814 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.813+0000 7ff6f2a00640 1 -- 172.21.15.145:0/3449638461 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e4011790 con 0x7ff6fc0d06f0 2024-05-09T04:34:26.814 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.813+0000 7ff6f2a00640 1 -- 172.21.15.145:0/3449638461 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6e4017070 con 0x7ff6fc0d06f0 2024-05-09T04:34:26.814 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.813+0000 7ff6f2a00640 1 -- 172.21.15.145:0/3449638461 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e4011bb0 con 0x7ff6fc0d06f0 2024-05-09T04:34:26.814 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.813+0000 7ff6fb200640 1 -- 172.21.15.145:0/3449638461 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6fc15e900 con 0x7ff6fc0d06f0 2024-05-09T04:34:26.814 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.813+0000 7ff6fb200640 1 -- 172.21.15.145:0/3449638461 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6fc15eec0 con 0x7ff6fc0d06f0 2024-05-09T04:34:26.815 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.814+0000 7ff6fb200640 1 -- 172.21.15.145:0/3449638461 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6fc076780 con 0x7ff6fc0d06f0 2024-05-09T04:34:26.815 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.816+0000 7ff6f2a00640 1 -- 172.21.15.145:0/3449638461 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff6e401c460 con 0x7ff6fc0d06f0 2024-05-09T04:34:26.816 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.816+0000 7ff6f2a00640 1 --2- 172.21.15.145:0/3449638461 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff6d8076300 0x7ff6d80787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:26.816 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.817+0000 7ff6f2a00640 1 -- 172.21.15.145:0/3449638461 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff6e4097490 con 0x7ff6fc0d06f0 2024-05-09T04:34:26.817 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.817+0000 7ff6f9e00640 1 --2- 172.21.15.145:0/3449638461 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff6d8076300 0x7ff6d80787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:26.817 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.818+0000 7ff6f9e00640 1 --2- 172.21.15.145:0/3449638461 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff6d8076300 0x7ff6d80787c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc0b0780 tx=0x7ff6ec00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:26.821 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:26.822+0000 7ff6f2a00640 1 -- 172.21.15.145:0/3449638461 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff6e4060de0 con 0x7ff6fc0d06f0 2024-05-09T04:34:27.073 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.073+0000 7ff6fb200640 1 -- 172.21.15.145:0/3449638461 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff6fc09ded0 con 0x7ff6fc0d06f0 2024-05-09T04:34:27.073 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.074+0000 7ff6f2a00640 1 -- 172.21.15.145:0/3449638461 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7ff6e40606e0 con 0x7ff6fc0d06f0 2024-05-09T04:34:27.073 INFO:teuthology.orchestra.run.smithi145.stderr:listed 0 entries 2024-05-09T04:34:27.077 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.076+0000 7ff6fb200640 1 -- 172.21.15.145:0/3449638461 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff6d8076300 msgr2=0x7ff6d80787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:27.077 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.076+0000 7ff6fb200640 1 --2- 172.21.15.145:0/3449638461 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff6d8076300 0x7ff6d80787c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc0b0780 tx=0x7ff6ec00b040 comp rx=0 tx=0).stop 2024-05-09T04:34:27.077 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.077+0000 7ff6fb200640 1 -- 172.21.15.145:0/3449638461 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff6fc0d06f0 msgr2=0x7ff6fc0d0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:27.077 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.077+0000 7ff6fb200640 1 --2- 172.21.15.145:0/3449638461 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff6fc0d06f0 0x7ff6fc0d0bb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff6e400c430 tx=0x7ff6e400c8f0 comp rx=0 tx=0).stop 2024-05-09T04:34:27.077 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.077+0000 7ff6fb200640 1 -- 172.21.15.145:0/3449638461 shutdown_connections 2024-05-09T04:34:27.077 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.077+0000 7ff6fb200640 1 --2- 172.21.15.145:0/3449638461 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff6d8076300 0x7ff6d80787c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:27.077 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.077+0000 7ff6fb200640 1 --2- 172.21.15.145:0/3449638461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff6fc15d190 0x7ff6fc0d10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:27.077 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.077+0000 7ff6fb200640 1 --2- 172.21.15.145:0/3449638461 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff6fc0d06f0 0x7ff6fc0d0bb0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:27.077 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.077+0000 7ff6fb200640 1 --2- 172.21.15.145:0/3449638461 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff6fc0aed50 0x7ff6fc153680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:27.077 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.077+0000 7ff6fb200640 1 -- 172.21.15.145:0/3449638461 >> 172.21.15.145:0/3449638461 conn(0x7ff6fc0b2d80 msgr2=0x7ff6fc0b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:27.077 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.077+0000 7ff6fb200640 1 -- 172.21.15.145:0/3449638461 shutdown_connections 2024-05-09T04:34:27.077 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.077+0000 7ff6fb200640 1 -- 172.21.15.145:0/3449638461 wait complete. 2024-05-09T04:34:27.095 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-05-09T04:34:27.095 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T04:34:27.140 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T04:34:27.373 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.373+0000 7f3bdd000640 1 Processor -- start 2024-05-09T04:34:27.373 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.373+0000 7f3bdd000640 1 -- start start 2024-05-09T04:34:27.374 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.375+0000 7f3bdd000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f3bd80d2db0 0x7f3bd80d0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:27.375 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.375+0000 7f3bdd000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bd80b4b70 con 0x7f3bd80d22e0 2024-05-09T04:34:27.375 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.375+0000 7f3bdd000640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bd80b4e50 con 0x7f3bd815f5f0 2024-05-09T04:34:27.375 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.375+0000 7f3bdd000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f3bd815f9d0 con 0x7f3bd80d2db0 2024-05-09T04:34:27.375 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.375+0000 7f3bd5400640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f3bd80d22e0 0x7f3bd80cfcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:44884/0 (socket says 172.21.15.145:44884) 2024-05-09T04:34:27.375 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.375+0000 7f3bd5400640 1 -- 172.21.15.145:0/3325144590 learned_addr learned my addr 172.21.15.145:0/3325144590 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:34:27.377 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.375+0000 7f3bd4a00640 1 -- 172.21.15.145:0/3325144590 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4099177399 0 0) 0x7f3bd80b4b70 con 0x7f3bd80d22e0 2024-05-09T04:34:27.377 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.376+0000 7f3bd4a00640 1 -- 172.21.15.145:0/3325144590 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3bbc003520 con 0x7f3bd80d22e0 2024-05-09T04:34:27.377 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.376+0000 7f3bd6800640 1 --2- 172.21.15.145:0/3325144590 >> v2:172.21.15.192:3300/0 conn(0x7f3bd80d2db0 0x7f3bd80d0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:27.377 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.376+0000 7f3bd4a00640 1 -- 172.21.15.145:0/3325144590 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3212632385 0 0) 0x7f3bbc003520 con 0x7f3bd80d22e0 2024-05-09T04:34:27.377 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.376+0000 7f3bd4a00640 1 -- 172.21.15.145:0/3325144590 >> v2:172.21.15.192:3300/0 conn(0x7f3bd80d2db0 msgr2=0x7f3bd80d0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:27.377 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.376+0000 7f3bd4a00640 1 --2- 172.21.15.145:0/3325144590 >> v2:172.21.15.192:3300/0 conn(0x7f3bd80d2db0 0x7f3bd80d0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:27.377 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.376+0000 7f3bd4a00640 1 -- 172.21.15.145:0/3325144590 >> v1:172.21.15.192:6789/0 conn(0x7f3bd815f5f0 legacy=0x7f3bd80d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:27.377 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.376+0000 7f3bd4a00640 1 -- 172.21.15.145:0/3325144590 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bd80b5130 con 0x7f3bd80d22e0 2024-05-09T04:34:27.378 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.376+0000 7f3bd4a00640 1 -- 172.21.15.145:0/3325144590 <== mon.0 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3474413261 0 0) 0x7f3bc40016f0 con 0x7f3bd80d22e0 2024-05-09T04:34:27.378 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.376+0000 7f3bd4a00640 1 -- 172.21.15.145:0/3325144590 >> v1:172.21.15.145:6789/0 conn(0x7f3bd80d22e0 legacy=0x7f3bd80cfcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:27.378 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.376+0000 7f3bd4a00640 1 --2- 172.21.15.145:0/3325144590 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3bbc003a50 0x7f3bbc003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:27.378 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.376+0000 7f3bd4a00640 1 --2- 172.21.15.145:0/3325144590 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3bbc004800 0x7f3bbc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:27.378 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.376+0000 7f3bd4a00640 1 --2- 172.21.15.145:0/3325144590 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3bbc0052e0 0x7f3bbc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:27.378 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.376+0000 7f3bd4a00640 1 -- 172.21.15.145:0/3325144590 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd815f9d0 con 0x7f3bbc004800 2024-05-09T04:34:27.378 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.376+0000 7f3bd4a00640 1 -- 172.21.15.145:0/3325144590 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3bbc00c1b0 con 0x7f3bbc0052e0 2024-05-09T04:34:27.378 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.376+0000 7f3bd4a00640 1 -- 172.21.15.145:0/3325144590 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f3bbc00c3b0 con 0x7f3bbc003a50 2024-05-09T04:34:27.378 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.376+0000 7f3bd5400640 1 --2- 172.21.15.145:0/3325144590 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3bbc004800 0x7f3bbc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:27.378 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.377+0000 7f3bd6800640 1 --2- 172.21.15.145:0/3325144590 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3bbc0052e0 0x7f3bbc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:27.378 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.377+0000 7f3bd5400640 1 -- 172.21.15.145:0/3325144590 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3bbc003a50 msgr2=0x7f3bbc003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:27.378 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.377+0000 7f3bd5400640 1 --2- 172.21.15.145:0/3325144590 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3bbc003a50 0x7f3bbc003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:27.378 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.377+0000 7f3bd5400640 1 -- 172.21.15.145:0/3325144590 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3bbc0052e0 msgr2=0x7f3bbc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:27.378 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.377+0000 7f3bd5400640 1 --2- 172.21.15.145:0/3325144590 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3bbc0052e0 0x7f3bbc00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:27.379 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.377+0000 7f3bd5400640 1 -- 172.21.15.145:0/3325144590 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bbc003520 con 0x7f3bbc004800 2024-05-09T04:34:27.379 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.377+0000 7f3bd6800640 1 -- 172.21.15.145:0/3325144590 reap_dead start 2024-05-09T04:34:27.379 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.378+0000 7f3bd5400640 1 --2- 172.21.15.145:0/3325144590 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3bbc004800 0x7f3bbc004c80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3bc4007fd0 tx=0x7f3bbc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b63971a811df38f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:27.380 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.380+0000 7f3bd4a00640 1 -- 172.21.15.145:0/3325144590 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc4007d50 con 0x7f3bbc004800 2024-05-09T04:34:27.380 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.380+0000 7f3bd4a00640 1 -- 172.21.15.145:0/3325144590 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bc4005af0 con 0x7f3bbc004800 2024-05-09T04:34:27.380 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.380+0000 7f3bd4a00640 1 -- 172.21.15.145:0/3325144590 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc4007070 con 0x7f3bbc004800 2024-05-09T04:34:27.381 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.381+0000 7f3bdd000640 1 -- 172.21.15.145:0/3325144590 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3bbc004800 msgr2=0x7f3bbc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:27.381 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.381+0000 7f3bdd000640 1 --2- 172.21.15.145:0/3325144590 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3bbc004800 0x7f3bbc004c80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3bc4007fd0 tx=0x7f3bbc005830 comp rx=0 tx=0).stop 2024-05-09T04:34:27.381 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.381+0000 7f3bdd000640 1 -- 172.21.15.145:0/3325144590 shutdown_connections 2024-05-09T04:34:27.381 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.381+0000 7f3bdd000640 1 --2- 172.21.15.145:0/3325144590 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3bbc004800 0x7f3bbc004c80 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:27.381 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.381+0000 7f3bdd000640 1 -- 172.21.15.145:0/3325144590 >> 172.21.15.145:0/3325144590 conn(0x7f3bd809e520 msgr2=0x7f3bd80d51e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:27.381 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.381+0000 7f3bdd000640 1 -- 172.21.15.145:0/3325144590 shutdown_connections 2024-05-09T04:34:27.381 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.381+0000 7f3bdd000640 1 -- 172.21.15.145:0/3325144590 wait complete. 2024-05-09T04:34:27.382 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.383+0000 7f3bdd000640 1 Processor -- start 2024-05-09T04:34:27.382 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.383+0000 7f3bdd000640 1 -- start start 2024-05-09T04:34:27.385 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.384+0000 7f3bdd000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3bbc004800 0x7f3bd815cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:27.385 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.384+0000 7f3bdd000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3bd815b140 0x7f3bd81505a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:27.385 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.384+0000 7f3bdd000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3bd815d480 0x7f3bd815d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:27.385 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.384+0000 7f3bdd000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd80d3c90 con 0x7f3bd815b140 2024-05-09T04:34:27.385 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.384+0000 7f3bdd000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3bd80d4040 con 0x7f3bd815d480 2024-05-09T04:34:27.385 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.384+0000 7f3bdd000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f3bd815d280 con 0x7f3bbc004800 2024-05-09T04:34:27.385 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.384+0000 7f3bd5400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3bd815b140 0x7f3bd81505a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:27.385 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.384+0000 7f3bd5400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3bd815b140 0x7f3bd81505a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:40604/0 (socket says 172.21.15.145:40604) 2024-05-09T04:34:27.386 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.384+0000 7f3bd5400640 1 -- 172.21.15.145:0/4209847745 learned_addr learned my addr 172.21.15.145:0/4209847745 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:27.386 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.385+0000 7f3bd6800640 1 --2- 172.21.15.145:0/4209847745 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3bd815d480 0x7f3bd815d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:27.386 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.385+0000 7f3bd5e00640 1 --2- 172.21.15.145:0/4209847745 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3bbc004800 0x7f3bd815cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:27.386 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.385+0000 7f3bd5400640 1 -- 172.21.15.145:0/4209847745 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3bbc004800 msgr2=0x7f3bd815cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:27.386 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.385+0000 7f3bd5400640 1 --2- 172.21.15.145:0/4209847745 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3bbc004800 0x7f3bd815cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:27.386 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.385+0000 7f3bd5400640 1 -- 172.21.15.145:0/4209847745 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3bd815d480 msgr2=0x7f3bd815d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:27.386 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.385+0000 7f3bd5400640 1 --2- 172.21.15.145:0/4209847745 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3bd815d480 0x7f3bd815d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:27.386 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.385+0000 7f3bd5400640 1 -- 172.21.15.145:0/4209847745 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bd81f9390 con 0x7f3bd815b140 2024-05-09T04:34:27.386 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.385+0000 7f3bd5e00640 1 --2- 172.21.15.145:0/4209847745 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3bbc004800 0x7f3bd815cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:34:27.386 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.385+0000 7f3bd5400640 1 --2- 172.21.15.145:0/4209847745 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3bd815b140 0x7f3bd81505a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3bc4009000 tx=0x7f3bc4001a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:27.386 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.385+0000 7f3bce000640 1 -- 172.21.15.145:0/4209847745 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc4037040 con 0x7f3bd815b140 2024-05-09T04:34:27.386 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.385+0000 7f3bce000640 1 -- 172.21.15.145:0/4209847745 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bc4041070 con 0x7f3bd815b140 2024-05-09T04:34:27.386 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.385+0000 7f3bdd000640 1 -- 172.21.15.145:0/4209847745 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bd81f85d0 con 0x7f3bd815b140 2024-05-09T04:34:27.386 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.385+0000 7f3bce000640 1 -- 172.21.15.145:0/4209847745 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc40016f0 con 0x7f3bd815b140 2024-05-09T04:34:27.387 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.386+0000 7f3bdd000640 1 -- 172.21.15.145:0/4209847745 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bd81f8b90 con 0x7f3bd815b140 2024-05-09T04:34:27.387 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.387+0000 7f3bdd000640 1 -- 172.21.15.145:0/4209847745 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bd806a310 con 0x7f3bd815b140 2024-05-09T04:34:27.388 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.389+0000 7f3bce000640 1 -- 172.21.15.145:0/4209847745 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3bc4004a20 con 0x7f3bd815b140 2024-05-09T04:34:27.389 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.390+0000 7f3bce000640 1 --2- 172.21.15.145:0/4209847745 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f3bb8076300 0x7f3bb80787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:27.390 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.390+0000 7f3bce000640 1 -- 172.21.15.145:0/4209847745 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3bc40b73f0 con 0x7f3bd815b140 2024-05-09T04:34:27.390 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.390+0000 7f3bd5e00640 1 --2- 172.21.15.145:0/4209847745 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f3bb8076300 0x7f3bb80787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:27.390 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.391+0000 7f3bd5e00640 1 --2- 172.21.15.145:0/4209847745 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f3bb8076300 0x7f3bb80787c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3bd806be20 tx=0x7f3bc8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:27.394 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.395+0000 7f3bce000640 1 -- 172.21.15.145:0/4209847745 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3bc4080cb0 con 0x7f3bd815b140 2024-05-09T04:34:27.615 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.615+0000 7f3bdd000640 1 -- 172.21.15.145:0/4209847745 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3bd80cb5b0 con 0x7f3bd815b140 2024-05-09T04:34:27.615 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.616+0000 7f3bce000640 1 -- 172.21.15.145:0/4209847745 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7f3bc40805b0 con 0x7f3bd815b140 2024-05-09T04:34:27.615 INFO:teuthology.orchestra.run.smithi145.stderr:listed 0 entries 2024-05-09T04:34:27.617 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.618+0000 7f3bdd000640 1 -- 172.21.15.145:0/4209847745 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f3bb8076300 msgr2=0x7f3bb80787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:27.618 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.618+0000 7f3bdd000640 1 --2- 172.21.15.145:0/4209847745 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f3bb8076300 0x7f3bb80787c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3bd806be20 tx=0x7f3bc8009040 comp rx=0 tx=0).stop 2024-05-09T04:34:27.618 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.618+0000 7f3bdd000640 1 -- 172.21.15.145:0/4209847745 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3bd815b140 msgr2=0x7f3bd81505a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:27.618 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.618+0000 7f3bdd000640 1 --2- 172.21.15.145:0/4209847745 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3bd815b140 0x7f3bd81505a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3bc4009000 tx=0x7f3bc4001a30 comp rx=0 tx=0).stop 2024-05-09T04:34:27.618 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.618+0000 7f3bdd000640 1 -- 172.21.15.145:0/4209847745 shutdown_connections 2024-05-09T04:34:27.619 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.618+0000 7f3bdd000640 1 --2- 172.21.15.145:0/4209847745 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f3bb8076300 0x7f3bb80787c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:27.619 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.618+0000 7f3bdd000640 1 --2- 172.21.15.145:0/4209847745 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3bd815d480 0x7f3bd815d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:27.619 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.618+0000 7f3bdd000640 1 --2- 172.21.15.145:0/4209847745 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3bd815b140 0x7f3bd81505a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:27.619 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.618+0000 7f3bdd000640 1 --2- 172.21.15.145:0/4209847745 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3bbc004800 0x7f3bd815cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:27.619 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.618+0000 7f3bdd000640 1 -- 172.21.15.145:0/4209847745 >> 172.21.15.145:0/4209847745 conn(0x7f3bd809e520 msgr2=0x7f3bd806d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:27.619 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.619+0000 7f3bdd000640 1 -- 172.21.15.145:0/4209847745 shutdown_connections 2024-05-09T04:34:27.619 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:27.619+0000 7f3bdd000640 1 -- 172.21.15.145:0/4209847745 wait complete. 2024-05-09T04:34:27.636 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi145.front.sepia.ceph.com... 2024-05-09T04:34:27.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:27.636 DEBUG:teuthology.orchestra.run.smithi145:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-05-09T04:34:27.707 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T04:34:27.707 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi145.front.sepia.ceph.com 2024-05-09T04:34:27.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:27.707 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-05-09T04:34:27.751 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-09T04:34:27.782 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T04:34:27.841 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-09T04:34:27.841 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T04:34:27.898 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T04:34:28.082 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.082+0000 7f08f1600640 1 Processor -- start 2024-05-09T04:34:28.083 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.083+0000 7f08f1600640 1 -- start start 2024-05-09T04:34:28.084 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.084+0000 7f08f1600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f08ec0ce060 0x7f08ec06e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:28.084 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.084+0000 7f08f1600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f08ec0ce440 0x7f08ec15d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:28.084 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.084+0000 7f08f1600640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08ec06ebd0 con 0x7f08ec1453c0 2024-05-09T04:34:28.084 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.084+0000 7f08f1600640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f08ec073460 con 0x7f08ec0ce440 2024-05-09T04:34:28.084 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.084+0000 7f08f1600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f08ec06f750 con 0x7f08ec0ce060 2024-05-09T04:34:28.085 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.084+0000 7f08e2000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f08ec0ce440 0x7f08ec15d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:28.085 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.085+0000 7f08e2000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f08ec0ce440 0x7f08ec15d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:40612/0 (socket says 172.21.15.145:40612) 2024-05-09T04:34:28.085 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.085+0000 7f08e2000640 1 -- 172.21.15.145:0/312872489 learned_addr learned my addr 172.21.15.145:0/312872489 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:28.085 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.085+0000 7f08e2000640 1 -- 172.21.15.145:0/312872489 >> v2:172.21.15.192:3300/0 conn(0x7f08ec0ce060 msgr2=0x7f08ec06e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:28.085 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.085+0000 7f08e2000640 1 --2- 172.21.15.145:0/312872489 >> v2:172.21.15.192:3300/0 conn(0x7f08ec0ce060 0x7f08ec06e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:28.085 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.085+0000 7f08e2000640 1 -- 172.21.15.145:0/312872489 >> v1:172.21.15.192:6789/0 conn(0x7f08ec1453c0 legacy=0x7f08ec15fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:28.085 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.085+0000 7f08e2000640 1 -- 172.21.15.145:0/312872489 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08ec06eeb0 con 0x7f08ec0ce440 2024-05-09T04:34:28.086 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.086+0000 7f08e2000640 1 --2- 172.21.15.145:0/312872489 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f08ec0ce440 0x7f08ec15d1f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f08ec0ceab0 tx=0x7f08d402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d50a3ffe53b92615 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:28.086 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.086+0000 7f08ea000640 1 -- 172.21.15.145:0/312872489 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d4037070 con 0x7f08ec0ce440 2024-05-09T04:34:28.086 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.086+0000 7f08ea000640 1 -- 172.21.15.145:0/312872489 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08d4032440 con 0x7f08ec0ce440 2024-05-09T04:34:28.086 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.086+0000 7f08ea000640 1 -- 172.21.15.145:0/312872489 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d4032810 con 0x7f08ec0ce440 2024-05-09T04:34:28.087 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.087+0000 7f08f1600640 1 -- 172.21.15.145:0/312872489 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f08ec0ce440 msgr2=0x7f08ec15d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:28.087 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.087+0000 7f08f1600640 1 --2- 172.21.15.145:0/312872489 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f08ec0ce440 0x7f08ec15d1f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f08ec0ceab0 tx=0x7f08d402a210 comp rx=0 tx=0).stop 2024-05-09T04:34:28.088 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.087+0000 7f08f1600640 1 -- 172.21.15.145:0/312872489 shutdown_connections 2024-05-09T04:34:28.088 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.087+0000 7f08f1600640 1 --2- 172.21.15.145:0/312872489 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f08ec0ce440 0x7f08ec15d1f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:28.088 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.087+0000 7f08f1600640 1 --2- 172.21.15.145:0/312872489 >> v2:172.21.15.192:3300/0 conn(0x7f08ec0ce060 0x7f08ec06e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:28.088 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.087+0000 7f08f1600640 1 -- 172.21.15.145:0/312872489 >> 172.21.15.145:0/312872489 conn(0x7f08ec06c8d0 msgr2=0x7f08ec06cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:28.088 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.087+0000 7f08f1600640 1 -- 172.21.15.145:0/312872489 shutdown_connections 2024-05-09T04:34:28.088 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.087+0000 7f08f1600640 1 -- 172.21.15.145:0/312872489 wait complete. 2024-05-09T04:34:28.088 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.089+0000 7f08f1600640 1 Processor -- start 2024-05-09T04:34:28.088 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.089+0000 7f08f1600640 1 -- start start 2024-05-09T04:34:28.090 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.090+0000 7f08f1600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08ec0ce060 0x7f08ec139750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:28.090 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.090+0000 7f08f1600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08ec0ce440 0x7f08ec15f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:28.090 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.090+0000 7f08f1600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f08ec1453c0 0x7f08ec160600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:28.090 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.090+0000 7f08f1600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f08ec15d8c0 con 0x7f08ec1453c0 2024-05-09T04:34:28.090 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.090+0000 7f08f1600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f08ec06f470 con 0x7f08ec0ce060 2024-05-09T04:34:28.090 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.090+0000 7f08f1600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f08ec15da40 con 0x7f08ec0ce440 2024-05-09T04:34:28.090 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.090+0000 7f08eb400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f08ec1453c0 0x7f08ec160600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:28.090 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.090+0000 7f08eb400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f08ec1453c0 0x7f08ec160600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:40620/0 (socket says 172.21.15.145:40620) 2024-05-09T04:34:28.090 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.090+0000 7f08eb400640 1 -- 172.21.15.145:0/2745198108 learned_addr learned my addr 172.21.15.145:0/2745198108 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:28.091 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.091+0000 7f08eaa00640 1 --2- 172.21.15.145:0/2745198108 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08ec0ce060 0x7f08ec139750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:28.092 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.091+0000 7f08e2000640 1 --2- 172.21.15.145:0/2745198108 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08ec0ce440 0x7f08ec15f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:28.092 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.091+0000 7f08eb400640 1 -- 172.21.15.145:0/2745198108 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08ec0ce440 msgr2=0x7f08ec15f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:28.092 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.091+0000 7f08eb400640 1 --2- 172.21.15.145:0/2745198108 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08ec0ce440 0x7f08ec15f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:28.092 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.091+0000 7f08eb400640 1 -- 172.21.15.145:0/2745198108 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08ec0ce060 msgr2=0x7f08ec139750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:28.092 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.091+0000 7f08eb400640 1 --2- 172.21.15.145:0/2745198108 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08ec0ce060 0x7f08ec139750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:28.092 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.091+0000 7f08eb400640 1 -- 172.21.15.145:0/2745198108 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08ec15fe60 con 0x7f08ec1453c0 2024-05-09T04:34:28.092 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.091+0000 7f08eb400640 1 --2- 172.21.15.145:0/2745198108 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f08ec1453c0 0x7f08ec160600 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f08e400b990 tx=0x7f08e400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:28.092 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.091+0000 7f08e3400640 1 -- 172.21.15.145:0/2745198108 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e4019070 con 0x7f08ec1453c0 2024-05-09T04:34:28.092 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.091+0000 7f08e3400640 1 -- 172.21.15.145:0/2745198108 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08e400cb40 con 0x7f08ec1453c0 2024-05-09T04:34:28.092 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.091+0000 7f08e3400640 1 -- 172.21.15.145:0/2745198108 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e4014440 con 0x7f08ec1453c0 2024-05-09T04:34:28.092 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.091+0000 7f08f1600640 1 -- 172.21.15.145:0/2745198108 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08ec15e880 con 0x7f08ec1453c0 2024-05-09T04:34:28.092 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.092+0000 7f08f1600640 1 -- 172.21.15.145:0/2745198108 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08ec15ee40 con 0x7f08ec1453c0 2024-05-09T04:34:28.093 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.093+0000 7f08f1600640 1 -- 172.21.15.145:0/2745198108 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08ec06a310 con 0x7f08ec1453c0 2024-05-09T04:34:28.094 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.094+0000 7f08e3400640 1 -- 172.21.15.145:0/2745198108 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f08e401c070 con 0x7f08ec1453c0 2024-05-09T04:34:28.095 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.095+0000 7f08e3400640 1 --2- 172.21.15.145:0/2745198108 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f08c80762d0 0x7f08c8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:28.095 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.095+0000 7f08e3400640 1 -- 172.21.15.145:0/2745198108 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f08e4097370 con 0x7f08ec1453c0 2024-05-09T04:34:28.095 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.096+0000 7f08eaa00640 1 --2- 172.21.15.145:0/2745198108 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f08c80762d0 0x7f08c8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:28.096 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.096+0000 7f08eaa00640 1 --2- 172.21.15.145:0/2745198108 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f08c80762d0 0x7f08c8078790 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f08ec15c170 tx=0x7f08dc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:28.100 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.100+0000 7f08e3400640 1 -- 172.21.15.145:0/2745198108 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f08ec15fe60 con 0x7f08ec1453c0 2024-05-09T04:34:28.317 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.317+0000 7f08f1600640 1 -- 172.21.15.145:0/2745198108 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f08ec15c1a0 con 0x7f08ec1453c0 2024-05-09T04:34:28.318 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.318+0000 7f08e3400640 1 -- 172.21.15.145:0/2745198108 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7f08e4060c30 con 0x7f08ec1453c0 2024-05-09T04:34:28.318 INFO:teuthology.orchestra.run.smithi145.stderr:listed 0 entries 2024-05-09T04:34:28.321 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.321+0000 7f08f1600640 1 -- 172.21.15.145:0/2745198108 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f08c80762d0 msgr2=0x7f08c8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:28.321 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.321+0000 7f08f1600640 1 --2- 172.21.15.145:0/2745198108 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f08c80762d0 0x7f08c8078790 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f08ec15c170 tx=0x7f08dc008040 comp rx=0 tx=0).stop 2024-05-09T04:34:28.321 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.321+0000 7f08f1600640 1 -- 172.21.15.145:0/2745198108 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f08ec1453c0 msgr2=0x7f08ec160600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:28.322 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.321+0000 7f08f1600640 1 --2- 172.21.15.145:0/2745198108 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f08ec1453c0 0x7f08ec160600 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f08e400b990 tx=0x7f08e400be50 comp rx=0 tx=0).stop 2024-05-09T04:34:28.322 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.321+0000 7f08f1600640 1 -- 172.21.15.145:0/2745198108 shutdown_connections 2024-05-09T04:34:28.322 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.321+0000 7f08f1600640 1 --2- 172.21.15.145:0/2745198108 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f08c80762d0 0x7f08c8078790 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:28.322 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.321+0000 7f08f1600640 1 --2- 172.21.15.145:0/2745198108 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f08ec1453c0 0x7f08ec160600 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:28.322 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.321+0000 7f08f1600640 1 --2- 172.21.15.145:0/2745198108 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08ec0ce440 0x7f08ec15f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:28.322 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.321+0000 7f08f1600640 1 --2- 172.21.15.145:0/2745198108 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08ec0ce060 0x7f08ec139750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:28.322 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.321+0000 7f08f1600640 1 -- 172.21.15.145:0/2745198108 >> 172.21.15.145:0/2745198108 conn(0x7f08ec06c8d0 msgr2=0x7f08ec0cd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:28.322 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.321+0000 7f08f1600640 1 -- 172.21.15.145:0/2745198108 shutdown_connections 2024-05-09T04:34:28.322 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.321+0000 7f08f1600640 1 -- 172.21.15.145:0/2745198108 wait complete. 2024-05-09T04:34:28.338 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-09T04:34:28.338 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T04:34:28.397 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T04:34:28.599 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.598+0000 7f8c28c00640 1 Processor -- start 2024-05-09T04:34:28.599 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.599+0000 7f8c28c00640 1 -- start start 2024-05-09T04:34:28.601 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.600+0000 7f8c28c00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f8c240ce060 0x7f8c2406e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:28.601 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.600+0000 7f8c28c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c240ce440 0x7f8c2415d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:28.601 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.600+0000 7f8c28c00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c2406ebd0 con 0x7f8c241453c0 2024-05-09T04:34:28.601 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.600+0000 7f8c28c00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f8c24073460 con 0x7f8c240ce060 2024-05-09T04:34:28.601 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.600+0000 7f8c28c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8c2406f750 con 0x7f8c240ce440 2024-05-09T04:34:28.601 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.601+0000 7f8c22800640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f8c241453c0 0x7f8c2415fcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:44888/0 (socket says 172.21.15.145:44888) 2024-05-09T04:34:28.601 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.601+0000 7f8c22800640 1 -- 172.21.15.145:0/3866551027 learned_addr learned my addr 172.21.15.145:0/3866551027 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:34:28.601 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.601+0000 7f8c21e00640 1 --2- 172.21.15.145:0/3866551027 >> v2:172.21.15.192:3300/0 conn(0x7f8c240ce060 0x7f8c2406e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:28.601 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.601+0000 7f8c1be00640 1 --2- 172.21.15.145:0/3866551027 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c240ce440 0x7f8c2415d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:28.601 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.601+0000 7f8c21400640 1 -- 172.21.15.145:0/3866551027 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4195825747 0 0) 0x7f8c2406ebd0 con 0x7f8c241453c0 2024-05-09T04:34:28.601 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.601+0000 7f8c21400640 1 -- 172.21.15.145:0/3866551027 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8c08003520 con 0x7f8c241453c0 2024-05-09T04:34:28.603 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.601+0000 7f8c21400640 1 -- 172.21.15.145:0/3866551027 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4279781466 0 0) 0x7f8c08003520 con 0x7f8c241453c0 2024-05-09T04:34:28.603 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.601+0000 7f8c21400640 1 -- 172.21.15.145:0/3866551027 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c240ce440 msgr2=0x7f8c2415d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:28.603 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.601+0000 7f8c21400640 1 --2- 172.21.15.145:0/3866551027 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c240ce440 0x7f8c2415d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:28.604 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.601+0000 7f8c21400640 1 -- 172.21.15.145:0/3866551027 >> v2:172.21.15.192:3300/0 conn(0x7f8c240ce060 msgr2=0x7f8c2406e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:28.604 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.601+0000 7f8c21400640 1 --2- 172.21.15.145:0/3866551027 >> v2:172.21.15.192:3300/0 conn(0x7f8c240ce060 0x7f8c2406e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:28.604 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.602+0000 7f8c21400640 1 -- 172.21.15.145:0/3866551027 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c2406eeb0 con 0x7f8c241453c0 2024-05-09T04:34:28.604 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.602+0000 7f8c21400640 1 -- 172.21.15.145:0/3866551027 <== mon.0 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3474413261 0 0) 0x7f8c1c001a70 con 0x7f8c241453c0 2024-05-09T04:34:28.604 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.602+0000 7f8c21400640 1 -- 172.21.15.145:0/3866551027 >> v1:172.21.15.145:6789/0 conn(0x7f8c241453c0 legacy=0x7f8c2415fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:28.604 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.602+0000 7f8c21400640 1 --2- 172.21.15.145:0/3866551027 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c240ce440 0x7f8c08003a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:28.604 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.602+0000 7f8c21400640 1 --2- 172.21.15.145:0/3866551027 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c08004050 0x7f8c08004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:28.604 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.602+0000 7f8c21400640 1 --2- 172.21.15.145:0/3866551027 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8c08004d40 0x7f8c080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:28.604 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.602+0000 7f8c21400640 1 -- 172.21.15.145:0/3866551027 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8c2406f750 con 0x7f8c08004d40 2024-05-09T04:34:28.604 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.602+0000 7f8c21400640 1 -- 172.21.15.145:0/3866551027 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0800ba20 con 0x7f8c08004050 2024-05-09T04:34:28.604 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.602+0000 7f8c21400640 1 -- 172.21.15.145:0/3866551027 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8c0800bc20 con 0x7f8c240ce440 2024-05-09T04:34:28.604 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.602+0000 7f8c22800640 1 --2- 172.21.15.145:0/3866551027 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8c08004d40 0x7f8c080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:28.604 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.602+0000 7f8c21e00640 1 --2- 172.21.15.145:0/3866551027 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c240ce440 0x7f8c08003a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:28.604 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.602+0000 7f8c1be00640 1 --2- 172.21.15.145:0/3866551027 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c08004050 0x7f8c08004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:28.605 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.602+0000 7f8c22800640 1 -- 172.21.15.145:0/3866551027 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c240ce440 msgr2=0x7f8c08003a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:28.605 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.602+0000 7f8c22800640 1 --2- 172.21.15.145:0/3866551027 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c240ce440 0x7f8c08003a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:28.605 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.602+0000 7f8c22800640 1 -- 172.21.15.145:0/3866551027 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c08004050 msgr2=0x7f8c08004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:28.605 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.602+0000 7f8c22800640 1 --2- 172.21.15.145:0/3866551027 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c08004050 0x7f8c08004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:28.605 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.602+0000 7f8c22800640 1 -- 172.21.15.145:0/3866551027 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c08003520 con 0x7f8c08004d40 2024-05-09T04:34:28.605 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.603+0000 7f8c22800640 1 --2- 172.21.15.145:0/3866551027 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8c08004d40 0x7f8c080051c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c00a800 tx=0x7f8c08005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9f508e40463f96e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:28.605 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.604+0000 7f8c21400640 1 -- 172.21.15.145:0/3866551027 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1c00a450 con 0x7f8c08004d40 2024-05-09T04:34:28.605 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.604+0000 7f8c21400640 1 -- 172.21.15.145:0/3866551027 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c1c02f070 con 0x7f8c08004d40 2024-05-09T04:34:28.605 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.604+0000 7f8c21400640 1 -- 172.21.15.145:0/3866551027 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1c02bbc0 con 0x7f8c08004d40 2024-05-09T04:34:28.606 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.604+0000 7f8c28c00640 1 -- 172.21.15.145:0/3866551027 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8c08004d40 msgr2=0x7f8c080051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:28.606 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.604+0000 7f8c28c00640 1 --2- 172.21.15.145:0/3866551027 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8c08004d40 0x7f8c080051c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c00a800 tx=0x7f8c08005830 comp rx=0 tx=0).stop 2024-05-09T04:34:28.606 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.604+0000 7f8c22800640 1 -- 172.21.15.145:0/3866551027 reap_dead start 2024-05-09T04:34:28.606 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.604+0000 7f8c28c00640 1 -- 172.21.15.145:0/3866551027 shutdown_connections 2024-05-09T04:34:28.607 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.604+0000 7f8c28c00640 1 -- 172.21.15.145:0/3866551027 >> 172.21.15.145:0/3866551027 conn(0x7f8c2406c8d0 msgr2=0x7f8c2406cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:28.607 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.604+0000 7f8c28c00640 1 -- 172.21.15.145:0/3866551027 shutdown_connections 2024-05-09T04:34:28.607 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.605+0000 7f8c28c00640 1 -- 172.21.15.145:0/3866551027 wait complete. 2024-05-09T04:34:28.607 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.606+0000 7f8c28c00640 1 Processor -- start 2024-05-09T04:34:28.607 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.606+0000 7f8c28c00640 1 -- start start 2024-05-09T04:34:28.608 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.607+0000 7f8c28c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8c24159310 0x7f8c24160840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:28.609 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.607+0000 7f8c28c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c2415feb0 0x7f8c240afa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:28.609 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.607+0000 7f8c28c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c240aff40 0x7f8c2415ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:28.609 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.607+0000 7f8c28c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8c2415d8c0 con 0x7f8c24159310 2024-05-09T04:34:28.609 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.607+0000 7f8c28c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8c2406f470 con 0x7f8c240aff40 2024-05-09T04:34:28.609 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.607+0000 7f8c28c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8c2415da40 con 0x7f8c2415feb0 2024-05-09T04:34:28.609 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.608+0000 7f8c21e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8c24159310 0x7f8c24160840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:28.609 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.608+0000 7f8c21e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8c24159310 0x7f8c24160840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:40626/0 (socket says 172.21.15.145:40626) 2024-05-09T04:34:28.609 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.608+0000 7f8c21e00640 1 -- 172.21.15.145:0/2096947266 learned_addr learned my addr 172.21.15.145:0/2096947266 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:28.609 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.608+0000 7f8c1be00640 1 --2- 172.21.15.145:0/2096947266 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c2415feb0 0x7f8c240afa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:28.609 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.608+0000 7f8c21e00640 1 -- 172.21.15.145:0/2096947266 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c2415feb0 msgr2=0x7f8c240afa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:28.609 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.608+0000 7f8c21e00640 1 --2- 172.21.15.145:0/2096947266 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c2415feb0 0x7f8c240afa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:28.609 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.608+0000 7f8c21e00640 1 -- 172.21.15.145:0/2096947266 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c240aff40 msgr2=0x7f8c2415ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:28.609 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.608+0000 7f8c21e00640 1 --2- 172.21.15.145:0/2096947266 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c240aff40 0x7f8c2415ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:28.610 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.608+0000 7f8c21e00640 1 -- 172.21.15.145:0/2096947266 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c240c6010 con 0x7f8c24159310 2024-05-09T04:34:28.610 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.608+0000 7f8c21e00640 1 --2- 172.21.15.145:0/2096947266 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8c24159310 0x7f8c24160840 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8c14011700 tx=0x7f8c14011bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:28.610 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.609+0000 7f8c1a000640 1 -- 172.21.15.145:0/2096947266 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1400f940 con 0x7f8c24159310 2024-05-09T04:34:28.610 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.609+0000 7f8c1a000640 1 -- 172.21.15.145:0/2096947266 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c1400fae0 con 0x7f8c24159310 2024-05-09T04:34:28.610 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.609+0000 7f8c1a000640 1 -- 172.21.15.145:0/2096947266 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1400fe90 con 0x7f8c24159310 2024-05-09T04:34:28.610 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.609+0000 7f8c28c00640 1 -- 172.21.15.145:0/2096947266 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c240c62f0 con 0x7f8c24159310 2024-05-09T04:34:28.610 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.609+0000 7f8c28c00640 1 -- 172.21.15.145:0/2096947266 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c240c68b0 con 0x7f8c24159310 2024-05-09T04:34:28.610 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.610+0000 7f8c28c00640 1 -- 172.21.15.145:0/2096947266 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c24160290 con 0x7f8c24159310 2024-05-09T04:34:28.611 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.611+0000 7f8c1a000640 1 -- 172.21.15.145:0/2096947266 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8c14015460 con 0x7f8c24159310 2024-05-09T04:34:28.613 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.612+0000 7f8c1a000640 1 --2- 172.21.15.145:0/2096947266 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f8c00076470 0x7f8c00078930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:28.613 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.612+0000 7f8c1a000640 1 -- 172.21.15.145:0/2096947266 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8c14097fe0 con 0x7f8c24159310 2024-05-09T04:34:28.613 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.612+0000 7f8c1be00640 1 --2- 172.21.15.145:0/2096947266 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f8c00076470 0x7f8c00078930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:28.614 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.613+0000 7f8c1be00640 1 --2- 172.21.15.145:0/2096947266 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f8c00076470 0x7f8c00078930 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8c24144480 tx=0x7f8c0c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:28.616 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.617+0000 7f8c1a000640 1 -- 172.21.15.145:0/2096947266 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8c14019030 con 0x7f8c24159310 2024-05-09T04:34:28.832 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.832+0000 7f8c28c00640 1 -- 172.21.15.145:0/2096947266 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8c2406a310 con 0x7f8c24159310 2024-05-09T04:34:28.833 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.833+0000 7f8c1a000640 1 -- 172.21.15.145:0/2096947266 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7f8c14061930 con 0x7f8c24159310 2024-05-09T04:34:28.833 INFO:teuthology.orchestra.run.smithi145.stderr:listed 0 entries 2024-05-09T04:34:28.836 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.835+0000 7f8c28c00640 1 -- 172.21.15.145:0/2096947266 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f8c00076470 msgr2=0x7f8c00078930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:28.836 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.835+0000 7f8c28c00640 1 --2- 172.21.15.145:0/2096947266 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f8c00076470 0x7f8c00078930 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8c24144480 tx=0x7f8c0c008040 comp rx=0 tx=0).stop 2024-05-09T04:34:28.836 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.835+0000 7f8c28c00640 1 -- 172.21.15.145:0/2096947266 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8c24159310 msgr2=0x7f8c24160840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:28.836 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.835+0000 7f8c28c00640 1 --2- 172.21.15.145:0/2096947266 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8c24159310 0x7f8c24160840 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8c14011700 tx=0x7f8c14011bc0 comp rx=0 tx=0).stop 2024-05-09T04:34:28.836 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.836+0000 7f8c28c00640 1 -- 172.21.15.145:0/2096947266 shutdown_connections 2024-05-09T04:34:28.836 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.836+0000 7f8c28c00640 1 --2- 172.21.15.145:0/2096947266 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f8c00076470 0x7f8c00078930 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:28.836 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.836+0000 7f8c28c00640 1 --2- 172.21.15.145:0/2096947266 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c240aff40 0x7f8c2415ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:28.836 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.836+0000 7f8c28c00640 1 --2- 172.21.15.145:0/2096947266 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c2415feb0 0x7f8c240afa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:28.836 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.836+0000 7f8c28c00640 1 --2- 172.21.15.145:0/2096947266 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8c24159310 0x7f8c24160840 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:28.836 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.836+0000 7f8c28c00640 1 -- 172.21.15.145:0/2096947266 >> 172.21.15.145:0/2096947266 conn(0x7f8c2406c8d0 msgr2=0x7f8c2415c770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:28.836 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.836+0000 7f8c28c00640 1 -- 172.21.15.145:0/2096947266 shutdown_connections 2024-05-09T04:34:28.836 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:28.836+0000 7f8c28c00640 1 -- 172.21.15.145:0/2096947266 wait complete. 2024-05-09T04:34:28.853 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi192.front.sepia.ceph.com... 2024-05-09T04:34:28.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:28.853 DEBUG:teuthology.orchestra.run.smithi192:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-05-09T04:34:28.945 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T04:34:28.945 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi192.front.sepia.ceph.com 2024-05-09T04:34:28.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:28.946 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-05-09T04:34:28.997 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-09T04:34:28.997 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T04:34:29.201 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.201+0000 7f6876800640 1 Processor -- start 2024-05-09T04:34:29.201 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.202+0000 7f6876800640 1 -- start start 2024-05-09T04:34:29.203 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.203+0000 7f6876800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f68700d3180 0x7f68700d5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:29.203 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.203+0000 7f6876800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68700d22f0 0x7f68700d3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:29.203 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.203+0000 7f6876800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68700d3e70 con 0x7f68700b4760 2024-05-09T04:34:29.203 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.203+0000 7f6876800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f68700d4150 con 0x7f68700d3180 2024-05-09T04:34:29.203 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.203+0000 7f6876800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f68700b6200 con 0x7f68700d22f0 2024-05-09T04:34:29.205 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.204+0000 7f6875e00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f68700b4760 0x7f68700b4bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:44892/0 (socket says 172.21.15.145:44892) 2024-05-09T04:34:29.205 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.204+0000 7f6875e00640 1 -- 172.21.15.145:0/1711829004 learned_addr learned my addr 172.21.15.145:0/1711829004 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:34:29.205 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.204+0000 7f6875400640 1 --2- 172.21.15.145:0/1711829004 >> v2:172.21.15.192:3300/0 conn(0x7f68700d3180 0x7f68700d5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:29.206 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.204+0000 7f6874a00640 1 --2- 172.21.15.145:0/1711829004 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68700d22f0 0x7f68700d3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:29.206 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.204+0000 7f686fe00640 1 -- 172.21.15.145:0/1711829004 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2955272314 0 0) 0x7f68700d3e70 con 0x7f68700b4760 2024-05-09T04:34:29.206 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.204+0000 7f686fe00640 1 -- 172.21.15.145:0/1711829004 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6854003520 con 0x7f68700b4760 2024-05-09T04:34:29.206 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.204+0000 7f686fe00640 1 -- 172.21.15.145:0/1711829004 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1802573424 0 0) 0x7f6854003520 con 0x7f68700b4760 2024-05-09T04:34:29.206 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.205+0000 7f686fe00640 1 -- 172.21.15.145:0/1711829004 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68700d22f0 msgr2=0x7f68700d3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:29.206 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.205+0000 7f686fe00640 1 --2- 172.21.15.145:0/1711829004 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68700d22f0 0x7f68700d3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:29.206 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.205+0000 7f686fe00640 1 -- 172.21.15.145:0/1711829004 >> v2:172.21.15.192:3300/0 conn(0x7f68700d3180 msgr2=0x7f68700d5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:29.206 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.205+0000 7f686fe00640 1 --2- 172.21.15.145:0/1711829004 >> v2:172.21.15.192:3300/0 conn(0x7f68700d3180 0x7f68700d5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:29.206 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.205+0000 7f686fe00640 1 -- 172.21.15.145:0/1711829004 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68700b6450 con 0x7f68700b4760 2024-05-09T04:34:29.206 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.205+0000 7f686fe00640 1 -- 172.21.15.145:0/1711829004 <== mon.0 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3474413261 0 0) 0x7f6868001a70 con 0x7f68700b4760 2024-05-09T04:34:29.206 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.205+0000 7f686fe00640 1 -- 172.21.15.145:0/1711829004 >> v1:172.21.15.145:6789/0 conn(0x7f68700b4760 legacy=0x7f68700b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:29.206 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.205+0000 7f686fe00640 1 --2- 172.21.15.145:0/1711829004 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6854003a00 0x7f6854003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:29.206 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.205+0000 7f686fe00640 1 --2- 172.21.15.145:0/1711829004 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68700d22f0 0x7f6854004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:29.206 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.205+0000 7f686fe00640 1 --2- 172.21.15.145:0/1711829004 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6854004d40 0x7f68540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:29.207 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.205+0000 7f686fe00640 1 -- 172.21.15.145:0/1711829004 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f68700b6200 con 0x7f6854004d40 2024-05-09T04:34:29.207 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.205+0000 7f686fe00640 1 -- 172.21.15.145:0/1711829004 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f685400b9c0 con 0x7f6854003a00 2024-05-09T04:34:29.207 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.205+0000 7f686fe00640 1 -- 172.21.15.145:0/1711829004 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f685400bbc0 con 0x7f68700d22f0 2024-05-09T04:34:29.207 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.205+0000 7f6875e00640 1 --2- 172.21.15.145:0/1711829004 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6854004d40 0x7f68540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:29.207 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.205+0000 7f6875400640 1 --2- 172.21.15.145:0/1711829004 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6854003a00 0x7f6854003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:29.207 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.205+0000 7f6874a00640 1 --2- 172.21.15.145:0/1711829004 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68700d22f0 0x7f6854004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:29.207 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.206+0000 7f6875e00640 1 -- 172.21.15.145:0/1711829004 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68700d22f0 msgr2=0x7f6854004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:29.207 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.206+0000 7f6875e00640 1 --2- 172.21.15.145:0/1711829004 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68700d22f0 0x7f6854004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:29.207 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.206+0000 7f6875e00640 1 -- 172.21.15.145:0/1711829004 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6854003a00 msgr2=0x7f6854003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:29.207 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.206+0000 7f6875e00640 1 --2- 172.21.15.145:0/1711829004 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6854003a00 0x7f6854003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:29.207 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.206+0000 7f6875e00640 1 -- 172.21.15.145:0/1711829004 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6854003520 con 0x7f6854004d40 2024-05-09T04:34:29.208 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.207+0000 7f6875e00640 1 --2- 172.21.15.145:0/1711829004 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6854004d40 0x7f68540051c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f686800a920 tx=0x7f6854005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b924f9367039b85b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:29.208 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.207+0000 7f686fe00640 1 -- 172.21.15.145:0/1711829004 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686800a4c0 con 0x7f6854004d40 2024-05-09T04:34:29.209 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.207+0000 7f686fe00640 1 -- 172.21.15.145:0/1711829004 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f686802f070 con 0x7f6854004d40 2024-05-09T04:34:29.209 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.207+0000 7f686fe00640 1 -- 172.21.15.145:0/1711829004 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686802bc70 con 0x7f6854004d40 2024-05-09T04:34:29.209 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.208+0000 7f6876800640 1 -- 172.21.15.145:0/1711829004 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6854004d40 msgr2=0x7f68540051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:29.209 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.208+0000 7f6876800640 1 --2- 172.21.15.145:0/1711829004 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6854004d40 0x7f68540051c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f686800a920 tx=0x7f6854005830 comp rx=0 tx=0).stop 2024-05-09T04:34:29.209 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.208+0000 7f6875e00640 1 -- 172.21.15.145:0/1711829004 reap_dead start 2024-05-09T04:34:29.209 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.208+0000 7f6876800640 1 -- 172.21.15.145:0/1711829004 shutdown_connections 2024-05-09T04:34:29.209 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.208+0000 7f6876800640 1 -- 172.21.15.145:0/1711829004 >> 172.21.15.145:0/1711829004 conn(0x7f68700d4fb0 msgr2=0x7f68700d53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:29.209 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.208+0000 7f6876800640 1 -- 172.21.15.145:0/1711829004 shutdown_connections 2024-05-09T04:34:29.209 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.208+0000 7f6876800640 1 -- 172.21.15.145:0/1711829004 wait complete. 2024-05-09T04:34:29.209 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.210+0000 7f6876800640 1 Processor -- start 2024-05-09T04:34:29.210 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.210+0000 7f6876800640 1 -- start start 2024-05-09T04:34:29.211 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.211+0000 7f6876800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6870158700 0x7f687015cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:29.211 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.211+0000 7f6876800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f687015a4d0 0x7f687015ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:29.211 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.211+0000 7f6876800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f687015b2d0 0x7f6870158f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:29.211 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.211+0000 7f6876800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f68700b52e0 con 0x7f687015b2d0 2024-05-09T04:34:29.211 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.211+0000 7f6876800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f68700d2750 con 0x7f687015a4d0 2024-05-09T04:34:29.211 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.211+0000 7f6876800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f68700b3fa0 con 0x7f6870158700 2024-05-09T04:34:29.211 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.211+0000 7f6875e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f687015b2d0 0x7f6870158f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:29.211 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.211+0000 7f6875e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f687015b2d0 0x7f6870158f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:40636/0 (socket says 172.21.15.145:40636) 2024-05-09T04:34:29.211 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.211+0000 7f6875e00640 1 -- 172.21.15.145:0/3958686095 learned_addr learned my addr 172.21.15.145:0/3958686095 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:29.212 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.212+0000 7f6875400640 1 --2- 172.21.15.145:0/3958686095 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6870158700 0x7f687015cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:29.213 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.212+0000 7f6874a00640 1 --2- 172.21.15.145:0/3958686095 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f687015a4d0 0x7f687015ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:29.213 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.212+0000 7f6875e00640 1 -- 172.21.15.145:0/3958686095 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6870158700 msgr2=0x7f687015cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:29.213 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.212+0000 7f6875e00640 1 --2- 172.21.15.145:0/3958686095 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6870158700 0x7f687015cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:29.213 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.212+0000 7f6875e00640 1 -- 172.21.15.145:0/3958686095 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f687015a4d0 msgr2=0x7f687015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:29.213 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.212+0000 7f6875e00640 1 --2- 172.21.15.145:0/3958686095 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f687015a4d0 0x7f687015ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:29.213 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.212+0000 7f6875e00640 1 -- 172.21.15.145:0/3958686095 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f687015d610 con 0x7f687015b2d0 2024-05-09T04:34:29.213 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.212+0000 7f6875e00640 1 --2- 172.21.15.145:0/3958686095 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f687015b2d0 0x7f6870158f80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f686802b4c0 tx=0x7f686802d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:29.213 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.212+0000 7f686d600640 1 -- 172.21.15.145:0/3958686095 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686800a4c0 con 0x7f687015b2d0 2024-05-09T04:34:29.213 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.212+0000 7f686d600640 1 -- 172.21.15.145:0/3958686095 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f686802be50 con 0x7f687015b2d0 2024-05-09T04:34:29.213 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.212+0000 7f686d600640 1 -- 172.21.15.145:0/3958686095 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68680346b0 con 0x7f687015b2d0 2024-05-09T04:34:29.213 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.212+0000 7f6876800640 1 -- 172.21.15.145:0/3958686095 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f687015d8f0 con 0x7f687015b2d0 2024-05-09T04:34:29.213 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.212+0000 7f6876800640 1 -- 172.21.15.145:0/3958686095 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f687015deb0 con 0x7f687015b2d0 2024-05-09T04:34:29.214 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.213+0000 7f6876800640 1 -- 172.21.15.145:0/3958686095 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68700c87e0 con 0x7f687015b2d0 2024-05-09T04:34:29.214 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.215+0000 7f686d600640 1 -- 172.21.15.145:0/3958686095 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6868034850 con 0x7f687015b2d0 2024-05-09T04:34:29.215 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.215+0000 7f686d600640 1 --2- 172.21.15.145:0/3958686095 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f68500762d0 0x7f6850078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:29.215 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.216+0000 7f686d600640 1 -- 172.21.15.145:0/3958686095 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f68680b7880 con 0x7f687015b2d0 2024-05-09T04:34:29.215 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.216+0000 7f6875400640 1 --2- 172.21.15.145:0/3958686095 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f68500762d0 0x7f6850078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:29.216 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.216+0000 7f6875400640 1 --2- 172.21.15.145:0/3958686095 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f68500762d0 0x7f6850078790 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6860009950 tx=0x7f6860008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:29.220 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.220+0000 7f686d600640 1 -- 172.21.15.145:0/3958686095 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f68680810b0 con 0x7f687015b2d0 2024-05-09T04:34:29.463 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.463+0000 7f6876800640 1 -- 172.21.15.145:0/3958686095 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f687006a310 con 0x7f687015b2d0 2024-05-09T04:34:29.464 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.465+0000 7f686d600640 1 -- 172.21.15.145:0/3958686095 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5187 (secure 0 0 0) 0x7f6868089130 con 0x7f687015b2d0 2024-05-09T04:34:29.465 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:34:29.465 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/1284540932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":1284540932},{"type":"v1","addr":"172.21.15.145:6835","nonce":1284540932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/1308656248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1308656248},{"type":"v1","addr":"172.21.15.145:6837","nonce":1308656248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3323136057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3323136057},{"type":"v1","addr":"172.21.15.192:6833","nonce":3323136057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T04:33:51.628921+0000","modified":"2024-05-09T04:33:54.699907+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.192:6835/1113351558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1113351558},{"type":"v1","addr":"172.21.15.192:6835","nonce":1113351558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4270,"qdb_cluster":[4270]},"id":1}]} 2024-05-09T04:34:29.465 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 8 2024-05-09T04:34:29.467 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.467+0000 7f6876800640 1 -- 172.21.15.145:0/3958686095 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f68500762d0 msgr2=0x7f6850078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:29.468 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.467+0000 7f6876800640 1 --2- 172.21.15.145:0/3958686095 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f68500762d0 0x7f6850078790 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6860009950 tx=0x7f6860008040 comp rx=0 tx=0).stop 2024-05-09T04:34:29.468 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.467+0000 7f6876800640 1 -- 172.21.15.145:0/3958686095 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f687015b2d0 msgr2=0x7f6870158f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:29.468 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.467+0000 7f6876800640 1 --2- 172.21.15.145:0/3958686095 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f687015b2d0 0x7f6870158f80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f686802b4c0 tx=0x7f686802d070 comp rx=0 tx=0).stop 2024-05-09T04:34:29.468 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.467+0000 7f6876800640 1 -- 172.21.15.145:0/3958686095 shutdown_connections 2024-05-09T04:34:29.468 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.467+0000 7f6876800640 1 --2- 172.21.15.145:0/3958686095 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f68500762d0 0x7f6850078790 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:29.468 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.467+0000 7f6876800640 1 --2- 172.21.15.145:0/3958686095 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f687015b2d0 0x7f6870158f80 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:29.468 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.467+0000 7f6876800640 1 --2- 172.21.15.145:0/3958686095 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f687015a4d0 0x7f687015ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:29.468 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.467+0000 7f6876800640 1 --2- 172.21.15.145:0/3958686095 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6870158700 0x7f687015cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:29.468 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.467+0000 7f6876800640 1 -- 172.21.15.145:0/3958686095 >> 172.21.15.145:0/3958686095 conn(0x7f68700d4fb0 msgr2=0x7f687006de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:29.468 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.467+0000 7f6876800640 1 -- 172.21.15.145:0/3958686095 shutdown_connections 2024-05-09T04:34:29.468 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.467+0000 7f6876800640 1 -- 172.21.15.145:0/3958686095 wait complete. 2024-05-09T04:34:29.486 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T04:34:29.688 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.687+0000 7f81e1c00640 1 Processor -- start 2024-05-09T04:34:29.689 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.687+0000 7f81e1c00640 1 -- start start 2024-05-09T04:34:29.689 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.689+0000 7f81e1c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f81dc0d5ad0 0x7f81dc0d22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:29.689 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.689+0000 7f81e1c00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f81dc0b4760 0x7f81dc0b4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:29.689 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.689+0000 7f81e1c00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81dc0d3e70 con 0x7f81dc0d3180 2024-05-09T04:34:29.689 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.689+0000 7f81e1c00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f81dc0d4150 con 0x7f81dc0b4760 2024-05-09T04:34:29.689 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.689+0000 7f81e1c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f81dc0d2830 con 0x7f81dc0d5ad0 2024-05-09T04:34:29.690 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.690+0000 7f81db400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f81dc0d5ad0 0x7f81dc0d22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:29.690 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.690+0000 7f81db400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f81dc0d5ad0 0x7f81dc0d22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.145:33860/0 (socket says 172.21.15.145:33860) 2024-05-09T04:34:29.690 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.690+0000 7f81db400640 1 -- 172.21.15.145:0/2454671698 learned_addr learned my addr 172.21.15.145:0/2454671698 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:29.690 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.690+0000 7f81e1200640 1 --2- 172.21.15.145:0/2454671698 >> v2:172.21.15.192:3300/0 conn(0x7f81dc0b4760 0x7f81dc0b4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:29.691 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.691+0000 7f81db400640 1 -- 172.21.15.145:0/2454671698 >> v2:172.21.15.192:3300/0 conn(0x7f81dc0b4760 msgr2=0x7f81dc0b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:29.691 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.691+0000 7f81db400640 1 --2- 172.21.15.145:0/2454671698 >> v2:172.21.15.192:3300/0 conn(0x7f81dc0b4760 0x7f81dc0b4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:29.691 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.691+0000 7f81db400640 1 -- 172.21.15.145:0/2454671698 >> v1:172.21.15.192:6789/0 conn(0x7f81dc0d3180 legacy=0x7f81dc0d3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:29.691 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.691+0000 7f81db400640 1 -- 172.21.15.145:0/2454671698 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81dc0b6450 con 0x7f81dc0d5ad0 2024-05-09T04:34:29.692 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.693+0000 7f81db400640 1 --2- 172.21.15.145:0/2454671698 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f81dc0d5ad0 0x7f81dc0d22f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f81c8009870 tx=0x7f81c802a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8396a72a2156eff server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:29.693 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.693+0000 7f81daa00640 1 -- 172.21.15.145:0/2454671698 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c8037070 con 0x7f81dc0d5ad0 2024-05-09T04:34:29.693 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.693+0000 7f81daa00640 1 -- 172.21.15.145:0/2454671698 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81c8032440 con 0x7f81dc0d5ad0 2024-05-09T04:34:29.693 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.693+0000 7f81daa00640 1 -- 172.21.15.145:0/2454671698 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c8032810 con 0x7f81dc0d5ad0 2024-05-09T04:34:29.694 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.694+0000 7f81e1c00640 1 -- 172.21.15.145:0/2454671698 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f81dc0d5ad0 msgr2=0x7f81dc0d22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:29.694 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.694+0000 7f81e1c00640 1 --2- 172.21.15.145:0/2454671698 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f81dc0d5ad0 0x7f81dc0d22f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f81c8009870 tx=0x7f81c802a1c0 comp rx=0 tx=0).stop 2024-05-09T04:34:29.694 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.694+0000 7f81e1c00640 1 -- 172.21.15.145:0/2454671698 shutdown_connections 2024-05-09T04:34:29.694 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.694+0000 7f81e1c00640 1 --2- 172.21.15.145:0/2454671698 >> v2:172.21.15.192:3300/0 conn(0x7f81dc0b4760 0x7f81dc0b4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:29.694 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.694+0000 7f81e1c00640 1 --2- 172.21.15.145:0/2454671698 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f81dc0d5ad0 0x7f81dc0d22f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:29.694 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.694+0000 7f81e1c00640 1 -- 172.21.15.145:0/2454671698 >> 172.21.15.145:0/2454671698 conn(0x7f81dc0d4fb0 msgr2=0x7f81dc0d53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:29.694 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.694+0000 7f81e1c00640 1 -- 172.21.15.145:0/2454671698 shutdown_connections 2024-05-09T04:34:29.695 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.694+0000 7f81e1c00640 1 -- 172.21.15.145:0/2454671698 wait complete. 2024-05-09T04:34:29.695 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.696+0000 7f81e1c00640 1 Processor -- start 2024-05-09T04:34:29.695 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.696+0000 7f81e1c00640 1 -- start start 2024-05-09T04:34:29.697 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.697+0000 7f81e1c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81dc0b4760 0x7f81dc1583f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:29.697 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.697+0000 7f81e1c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f81dc0d3180 0x7f81dc15f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:29.697 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.697+0000 7f81e1c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f81dc0d5ad0 0x7f81dc15d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:29.698 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.697+0000 7f81e1c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f81dc0d1e60 con 0x7f81dc0d5ad0 2024-05-09T04:34:29.698 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.697+0000 7f81e1c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f81dc0b6200 con 0x7f81dc0b4760 2024-05-09T04:34:29.698 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.697+0000 7f81e1c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f81dc0b3fa0 con 0x7f81dc0d3180 2024-05-09T04:34:29.698 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.697+0000 7f81e1200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f81dc0d5ad0 0x7f81dc15d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:29.698 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.697+0000 7f81e1200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f81dc0d5ad0 0x7f81dc15d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:40646/0 (socket says 172.21.15.145:40646) 2024-05-09T04:34:29.698 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.697+0000 7f81e1200640 1 -- 172.21.15.145:0/2112057338 learned_addr learned my addr 172.21.15.145:0/2112057338 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:29.698 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.697+0000 7f81db400640 1 --2- 172.21.15.145:0/2112057338 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f81dc0d3180 0x7f81dc15f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:29.698 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.697+0000 7f81dbe00640 1 --2- 172.21.15.145:0/2112057338 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81dc0b4760 0x7f81dc1583f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:29.698 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.698+0000 7f81e1200640 1 -- 172.21.15.145:0/2112057338 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f81dc0d3180 msgr2=0x7f81dc15f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:29.699 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.698+0000 7f81e1200640 1 --2- 172.21.15.145:0/2112057338 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f81dc0d3180 0x7f81dc15f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:29.699 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.698+0000 7f81e1200640 1 -- 172.21.15.145:0/2112057338 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81dc0b4760 msgr2=0x7f81dc1583f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:29.699 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.698+0000 7f81e1200640 1 --2- 172.21.15.145:0/2112057338 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81dc0b4760 0x7f81dc1583f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:29.699 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.698+0000 7f81e1200640 1 -- 172.21.15.145:0/2112057338 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81dc0c9980 con 0x7f81dc0d5ad0 2024-05-09T04:34:29.699 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.698+0000 7f81e1200640 1 --2- 172.21.15.145:0/2112057338 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f81dc0d5ad0 0x7f81dc15d490 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f81d4009dd0 tx=0x7f81d400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:29.700 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.698+0000 7f81d3e00640 1 -- 172.21.15.145:0/2112057338 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d4018020 con 0x7f81dc0d5ad0 2024-05-09T04:34:29.700 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.698+0000 7f81d3e00640 1 -- 172.21.15.145:0/2112057338 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81d4014480 con 0x7f81dc0d5ad0 2024-05-09T04:34:29.700 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.698+0000 7f81e1c00640 1 -- 172.21.15.145:0/2112057338 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81dc0c9c60 con 0x7f81dc0d5ad0 2024-05-09T04:34:29.700 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.698+0000 7f81d3e00640 1 -- 172.21.15.145:0/2112057338 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d4014870 con 0x7f81dc0d5ad0 2024-05-09T04:34:29.700 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.698+0000 7f81e1c00640 1 -- 172.21.15.145:0/2112057338 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81dc0ca220 con 0x7f81dc0d5ad0 2024-05-09T04:34:29.700 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.699+0000 7f81e1c00640 1 -- 172.21.15.145:0/2112057338 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81dc06a310 con 0x7f81dc0d5ad0 2024-05-09T04:34:29.701 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.701+0000 7f81d3e00640 1 -- 172.21.15.145:0/2112057338 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f81d4005d10 con 0x7f81dc0d5ad0 2024-05-09T04:34:29.701 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.702+0000 7f81d3e00640 1 --2- 172.21.15.145:0/2112057338 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f81bc0762d0 0x7f81bc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:29.701 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.702+0000 7f81d3e00640 1 -- 172.21.15.145:0/2112057338 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f81d4098130 con 0x7f81dc0d5ad0 2024-05-09T04:34:29.701 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.702+0000 7f81dbe00640 1 --2- 172.21.15.145:0/2112057338 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f81bc0762d0 0x7f81bc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:29.702 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.703+0000 7f81dbe00640 1 --2- 172.21.15.145:0/2112057338 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f81bc0762d0 0x7f81bc078790 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f81cc001680 tx=0x7f81cc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:29.705 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.706+0000 7f81d3e00640 1 -- 172.21.15.145:0/2112057338 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f81d4022070 con 0x7f81dc0d5ad0 2024-05-09T04:34:29.958 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.958+0000 7f81e1c00640 1 -- 172.21.15.145:0/2112057338 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f81dc162a80 con 0x7f81dc0d5ad0 2024-05-09T04:34:29.959 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.959+0000 7f81d3e00640 1 -- 172.21.15.145:0/2112057338 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5187 (secure 0 0 0) 0x7f81d4010040 con 0x7f81dc0d5ad0 2024-05-09T04:34:29.959 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:34:29.959 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/1284540932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":1284540932},{"type":"v1","addr":"172.21.15.145:6835","nonce":1284540932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/1308656248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1308656248},{"type":"v1","addr":"172.21.15.145:6837","nonce":1308656248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3323136057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3323136057},{"type":"v1","addr":"172.21.15.192:6833","nonce":3323136057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T04:33:51.628921+0000","modified":"2024-05-09T04:33:54.699907+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.192:6835/1113351558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1113351558},{"type":"v1","addr":"172.21.15.192:6835","nonce":1113351558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4270,"qdb_cluster":[4270]},"id":1}]} 2024-05-09T04:34:29.959 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 8 2024-05-09T04:34:29.962 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.962+0000 7f81e1c00640 1 -- 172.21.15.145:0/2112057338 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f81bc0762d0 msgr2=0x7f81bc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:29.962 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.962+0000 7f81e1c00640 1 --2- 172.21.15.145:0/2112057338 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f81bc0762d0 0x7f81bc078790 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f81cc001680 tx=0x7f81cc009040 comp rx=0 tx=0).stop 2024-05-09T04:34:29.963 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.962+0000 7f81e1c00640 1 -- 172.21.15.145:0/2112057338 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f81dc0d5ad0 msgr2=0x7f81dc15d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:29.963 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.962+0000 7f81e1c00640 1 --2- 172.21.15.145:0/2112057338 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f81dc0d5ad0 0x7f81dc15d490 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f81d4009dd0 tx=0x7f81d400c490 comp rx=0 tx=0).stop 2024-05-09T04:34:29.963 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.962+0000 7f81e1c00640 1 -- 172.21.15.145:0/2112057338 shutdown_connections 2024-05-09T04:34:29.963 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.962+0000 7f81e1c00640 1 --2- 172.21.15.145:0/2112057338 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f81bc0762d0 0x7f81bc078790 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:29.963 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.962+0000 7f81e1c00640 1 --2- 172.21.15.145:0/2112057338 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f81dc0d5ad0 0x7f81dc15d490 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:29.963 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.962+0000 7f81e1c00640 1 --2- 172.21.15.145:0/2112057338 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f81dc0d3180 0x7f81dc15f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:29.963 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.962+0000 7f81e1c00640 1 --2- 172.21.15.145:0/2112057338 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81dc0b4760 0x7f81dc1583f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:29.963 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.962+0000 7f81e1c00640 1 -- 172.21.15.145:0/2112057338 >> 172.21.15.145:0/2112057338 conn(0x7f81dc0d4fb0 msgr2=0x7f81dc0b3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:29.963 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.963+0000 7f81e1c00640 1 -- 172.21.15.145:0/2112057338 shutdown_connections 2024-05-09T04:34:29.963 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:29.963+0000 7f81e1c00640 1 -- 172.21.15.145:0/2112057338 wait complete. 2024-05-09T04:34:29.981 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T04:34:30.176 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.176+0000 7f4aa4200640 1 Processor -- start 2024-05-09T04:34:30.176 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.176+0000 7f4aa4200640 1 -- start start 2024-05-09T04:34:30.178 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.178+0000 7f4aa4200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4a9c145880 0x7f4a9c0affc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:30.179 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.178+0000 7f4aa4200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f4a9c06ef70 0x7f4a9c0b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:30.179 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.178+0000 7f4aa4200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a9c072350 con 0x7f4a9c148c20 2024-05-09T04:34:30.179 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.178+0000 7f4aa4200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f4a9c072f60 con 0x7f4a9c06ef70 2024-05-09T04:34:30.179 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.178+0000 7f4aa4200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f4a9c072630 con 0x7f4a9c145880 2024-05-09T04:34:30.179 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.179+0000 7f4aa1800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4a9c145880 0x7f4a9c0affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:30.179 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.179+0000 7f4aa1800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4a9c145880 0x7f4a9c0affc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.145:33874/0 (socket says 172.21.15.145:33874) 2024-05-09T04:34:30.179 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.179+0000 7f4aa1800640 1 -- 172.21.15.145:0/963604821 learned_addr learned my addr 172.21.15.145:0/963604821 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:30.179 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.179+0000 7f4aa2200640 1 --2- 172.21.15.145:0/963604821 >> v2:172.21.15.192:3300/0 conn(0x7f4a9c06ef70 0x7f4a9c0b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:30.179 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.180+0000 7f4aa1800640 1 -- 172.21.15.145:0/963604821 >> v2:172.21.15.192:3300/0 conn(0x7f4a9c06ef70 msgr2=0x7f4a9c0b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:30.180 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.180+0000 7f4aa1800640 1 --2- 172.21.15.145:0/963604821 >> v2:172.21.15.192:3300/0 conn(0x7f4a9c06ef70 0x7f4a9c0b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:30.180 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.180+0000 7f4aa1800640 1 -- 172.21.15.145:0/963604821 >> v1:172.21.15.192:6789/0 conn(0x7f4a9c148c20 legacy=0x7f4a9c06e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:30.180 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.180+0000 7f4aa1800640 1 -- 172.21.15.145:0/963604821 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a9c072910 con 0x7f4a9c145880 2024-05-09T04:34:30.180 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.181+0000 7f4aa1800640 1 --2- 172.21.15.145:0/963604821 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4a9c145880 0x7f4a9c0affc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4a90009870 tx=0x7f4a9002a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d24a6fe7e0cb7942 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:30.181 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.181+0000 7f4a9be00640 1 -- 172.21.15.145:0/963604821 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a90037070 con 0x7f4a9c145880 2024-05-09T04:34:30.181 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.181+0000 7f4a9be00640 1 -- 172.21.15.145:0/963604821 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a90032440 con 0x7f4a9c145880 2024-05-09T04:34:30.181 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.181+0000 7f4a9be00640 1 -- 172.21.15.145:0/963604821 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a90032810 con 0x7f4a9c145880 2024-05-09T04:34:30.182 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.182+0000 7f4aa4200640 1 -- 172.21.15.145:0/963604821 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4a9c145880 msgr2=0x7f4a9c0affc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:30.182 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.182+0000 7f4aa4200640 1 --2- 172.21.15.145:0/963604821 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4a9c145880 0x7f4a9c0affc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4a90009870 tx=0x7f4a9002a1c0 comp rx=0 tx=0).stop 2024-05-09T04:34:30.182 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.182+0000 7f4aa4200640 1 -- 172.21.15.145:0/963604821 shutdown_connections 2024-05-09T04:34:30.182 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.182+0000 7f4aa4200640 1 --2- 172.21.15.145:0/963604821 >> v2:172.21.15.192:3300/0 conn(0x7f4a9c06ef70 0x7f4a9c0b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:30.182 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.182+0000 7f4aa4200640 1 --2- 172.21.15.145:0/963604821 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4a9c145880 0x7f4a9c0affc0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:30.182 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.182+0000 7f4aa4200640 1 -- 172.21.15.145:0/963604821 >> 172.21.15.145:0/963604821 conn(0x7f4a9c06c870 msgr2=0x7f4a9c06cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:30.182 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.182+0000 7f4aa4200640 1 -- 172.21.15.145:0/963604821 shutdown_connections 2024-05-09T04:34:30.182 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.182+0000 7f4aa4200640 1 -- 172.21.15.145:0/963604821 wait complete. 2024-05-09T04:34:30.183 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.184+0000 7f4aa4200640 1 Processor -- start 2024-05-09T04:34:30.184 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.184+0000 7f4aa4200640 1 -- start start 2024-05-09T04:34:30.186 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.185+0000 7f4aa4200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4a9c06ef70 0x7f4a9c163120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:30.186 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.185+0000 7f4aa4200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4a9c145880 0x7f4a9c0c5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:30.186 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.185+0000 7f4aa4200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a9c148c20 0x7f4a9c0c76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:30.186 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.185+0000 7f4aa4200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9c0b56f0 con 0x7f4a9c145880 2024-05-09T04:34:30.186 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.185+0000 7f4aa4200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9c145c60 con 0x7f4a9c148c20 2024-05-09T04:34:30.186 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.185+0000 7f4aa4200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f4a9c0b5870 con 0x7f4a9c06ef70 2024-05-09T04:34:30.186 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.185+0000 7f4aa0e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4a9c145880 0x7f4a9c0c5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:30.186 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.185+0000 7f4aa0e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4a9c145880 0x7f4a9c0c5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:40650/0 (socket says 172.21.15.145:40650) 2024-05-09T04:34:30.186 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.185+0000 7f4aa0e00640 1 -- 172.21.15.145:0/1517756387 learned_addr learned my addr 172.21.15.145:0/1517756387 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:30.186 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.186+0000 7f4aa1800640 1 --2- 172.21.15.145:0/1517756387 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4a9c06ef70 0x7f4a9c163120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:30.187 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.186+0000 7f4aa0e00640 1 -- 172.21.15.145:0/1517756387 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4a9c06ef70 msgr2=0x7f4a9c163120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:30.187 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.186+0000 7f4aa0e00640 1 --2- 172.21.15.145:0/1517756387 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4a9c06ef70 0x7f4a9c163120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:30.187 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.186+0000 7f4aa0e00640 1 -- 172.21.15.145:0/1517756387 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a9c148c20 msgr2=0x7f4a9c0c76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:30.187 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.186+0000 7f4aa0e00640 1 --2- 172.21.15.145:0/1517756387 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a9c148c20 0x7f4a9c0c76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:30.187 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.186+0000 7f4aa0e00640 1 -- 172.21.15.145:0/1517756387 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a9c0c6650 con 0x7f4a9c145880 2024-05-09T04:34:30.187 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.186+0000 7f4aa0e00640 1 --2- 172.21.15.145:0/1517756387 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4a9c145880 0x7f4a9c0c5ee0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4a8800c3a0 tx=0x7f4a8800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:30.187 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.186+0000 7f4a99600640 1 -- 172.21.15.145:0/1517756387 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a88017070 con 0x7f4a9c145880 2024-05-09T04:34:30.187 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.186+0000 7f4a99600640 1 -- 172.21.15.145:0/1517756387 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a880052d0 con 0x7f4a9c145880 2024-05-09T04:34:30.187 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.186+0000 7f4a99600640 1 -- 172.21.15.145:0/1517756387 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a88003830 con 0x7f4a9c145880 2024-05-09T04:34:30.187 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.186+0000 7f4aa4200640 1 -- 172.21.15.145:0/1517756387 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a9c0c6930 con 0x7f4a9c145880 2024-05-09T04:34:30.187 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.186+0000 7f4aa4200640 1 -- 172.21.15.145:0/1517756387 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a9c0c6ef0 con 0x7f4a9c145880 2024-05-09T04:34:30.188 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.187+0000 7f4aa4200640 1 -- 172.21.15.145:0/1517756387 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a9c06a310 con 0x7f4a9c145880 2024-05-09T04:34:30.188 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.189+0000 7f4a99600640 1 -- 172.21.15.145:0/1517756387 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4a8801a070 con 0x7f4a9c145880 2024-05-09T04:34:30.189 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.189+0000 7f4a99600640 1 --2- 172.21.15.145:0/1517756387 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f4a7c0762d0 0x7f4a7c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:30.189 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.190+0000 7f4a99600640 1 -- 172.21.15.145:0/1517756387 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4a880968e0 con 0x7f4a9c145880 2024-05-09T04:34:30.189 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.190+0000 7f4aa1800640 1 --2- 172.21.15.145:0/1517756387 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f4a7c0762d0 0x7f4a7c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:30.190 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.190+0000 7f4aa1800640 1 --2- 172.21.15.145:0/1517756387 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f4a7c0762d0 0x7f4a7c078790 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4a90034000 tx=0x7f4a9002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:30.193 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.194+0000 7f4a99600640 1 -- 172.21.15.145:0/1517756387 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f4a880601a0 con 0x7f4a9c145880 2024-05-09T04:34:30.400 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.400+0000 7f4aa4200640 1 -- 172.21.15.145:0/1517756387 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4a9c06ebb0 con 0x7f4a9c145880 2024-05-09T04:34:30.406 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.407+0000 7f4a99600640 1 -- 172.21.15.145:0/1517756387 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16653 (secure 0 0 0) 0x7f4a88068190 con 0x7f4a9c145880 2024-05-09T04:34:30.408 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:34:30.408 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":20,"fsid":"957ee6c1-073b-44cf-9b93-9491b3fdfc1d","created":"2024-05-09T04:33:26.544367+0000","modified":"2024-05-09T04:33:53.178020+0000","last_up_change":"2024-05-09T04:33:39.039318+0000","last_in_change":"2024-05-09T04:33:29.902361+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T04:33:39.795344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T04:33:49.651743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T04:33:50.596065+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22e550ad-847b-4952-80e6-63d859ed1a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6801","nonce":3182572912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6803","nonce":3182572912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6807","nonce":3182572912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6805","nonce":3182572912}]},"public_addr":"172.21.15.145:6801/3182572912","cluster_addr":"172.21.15.145:6803/3182572912","heartbeat_back_addr":"172.21.15.145:6807/3182572912","heartbeat_front_addr":"172.21.15.145:6805/3182572912","state":["exists","up"]},{"osd":1,"uuid":"0f6b9fca-0c41-4145-8d74-598ddb42f2a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6825","nonce":1845225555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6827","nonce":1845225555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6831","nonce":1845225555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6829","nonce":1845225555}]},"public_addr":"172.21.15.145:6825/1845225555","cluster_addr":"172.21.15.145:6827/1845225555","heartbeat_back_addr":"172.21.15.145:6831/1845225555","heartbeat_front_addr":"172.21.15.145:6829/1845225555","state":["exists","up"]},{"osd":2,"uuid":"b3b1f67e-5790-483f-8610-e05a5bce4c30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6817","nonce":2168958993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6819","nonce":2168958993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6823","nonce":2168958993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6821","nonce":2168958993}]},"public_addr":"172.21.15.145:6817/2168958993","cluster_addr":"172.21.15.145:6819/2168958993","heartbeat_back_addr":"172.21.15.145:6823/2168958993","heartbeat_front_addr":"172.21.15.145:6821/2168958993","state":["exists","up"]},{"osd":3,"uuid":"884787a5-e037-4d31-9aae-743030eba4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6809","nonce":337009826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6811","nonce":337009826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6815","nonce":337009826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6813","nonce":337009826}]},"public_addr":"172.21.15.145:6809/337009826","cluster_addr":"172.21.15.145:6811/337009826","heartbeat_back_addr":"172.21.15.145:6815/337009826","heartbeat_front_addr":"172.21.15.145:6813/337009826","state":["exists","up"]},{"osd":4,"uuid":"0759fc1e-c5fc-4a2f-86bb-e8eccd83f833","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6818","nonce":3117700995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6821","nonce":3117700995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6827","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6829","nonce":3117700995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6823","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6825","nonce":3117700995}]},"public_addr":"172.21.15.192:6818/3117700995","cluster_addr":"172.21.15.192:6821/3117700995","heartbeat_back_addr":"172.21.15.192:6829/3117700995","heartbeat_front_addr":"172.21.15.192:6825/3117700995","state":["exists","up"]},{"osd":5,"uuid":"9443cc22-9379-46e3-9173-98b592bf2c16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6801","nonce":1328373268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6803","nonce":1328373268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6807","nonce":1328373268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6805","nonce":1328373268}]},"public_addr":"172.21.15.192:6801/1328373268","cluster_addr":"172.21.15.192:6803/1328373268","heartbeat_back_addr":"172.21.15.192:6807/1328373268","heartbeat_front_addr":"172.21.15.192:6805/1328373268","state":["exists","up"]},{"osd":6,"uuid":"d05a3311-505a-45b5-85b9-aa3113e96987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6817","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6820","nonce":1868011057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6824","nonce":1868011057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6831","nonce":1868011057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6828","nonce":1868011057}]},"public_addr":"172.21.15.192:6820/1868011057","cluster_addr":"172.21.15.192:6824/1868011057","heartbeat_back_addr":"172.21.15.192:6831/1868011057","heartbeat_front_addr":"172.21.15.192:6828/1868011057","state":["exists","up"]},{"osd":7,"uuid":"c7c23dbc-2029-43ec-9f53-25f3d58019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6809","nonce":3368512157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6811","nonce":3368512157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6815","nonce":3368512157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6813","nonce":3368512157}]},"public_addr":"172.21.15.192:6809/3368512157","cluster_addr":"172.21.15.192:6811/3368512157","heartbeat_back_addr":"172.21.15.192:6815/3368512157","heartbeat_front_addr":"172.21.15.192:6813/3368512157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.091188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.032567+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:34.797468+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.228839+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.622400+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.473103+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.566092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.349395+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T04:34:30.410 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.410+0000 7f4aa4200640 1 -- 172.21.15.145:0/1517756387 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f4a7c0762d0 msgr2=0x7f4a7c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:30.410 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.410+0000 7f4aa4200640 1 --2- 172.21.15.145:0/1517756387 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f4a7c0762d0 0x7f4a7c078790 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4a90034000 tx=0x7f4a9002e000 comp rx=0 tx=0).stop 2024-05-09T04:34:30.410 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.410+0000 7f4aa4200640 1 -- 172.21.15.145:0/1517756387 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4a9c145880 msgr2=0x7f4a9c0c5ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:30.410 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.410+0000 7f4aa4200640 1 --2- 172.21.15.145:0/1517756387 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4a9c145880 0x7f4a9c0c5ee0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4a8800c3a0 tx=0x7f4a8800c860 comp rx=0 tx=0).stop 2024-05-09T04:34:30.410 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.410+0000 7f4aa4200640 1 -- 172.21.15.145:0/1517756387 shutdown_connections 2024-05-09T04:34:30.410 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.410+0000 7f4aa4200640 1 --2- 172.21.15.145:0/1517756387 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f4a7c0762d0 0x7f4a7c078790 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:30.410 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.410+0000 7f4aa4200640 1 --2- 172.21.15.145:0/1517756387 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4a9c148c20 0x7f4a9c0c76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:30.411 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.410+0000 7f4aa4200640 1 --2- 172.21.15.145:0/1517756387 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4a9c145880 0x7f4a9c0c5ee0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:30.411 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.410+0000 7f4aa4200640 1 --2- 172.21.15.145:0/1517756387 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4a9c06ef70 0x7f4a9c163120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:30.411 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.410+0000 7f4aa4200640 1 -- 172.21.15.145:0/1517756387 >> 172.21.15.145:0/1517756387 conn(0x7f4a9c06c870 msgr2=0x7f4a9c0b3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:30.411 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.410+0000 7f4aa4200640 1 -- 172.21.15.145:0/1517756387 shutdown_connections 2024-05-09T04:34:30.411 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.410+0000 7f4aa4200640 1 -- 172.21.15.145:0/1517756387 wait complete. 2024-05-09T04:34:30.428 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-09T04:34:30.428 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T04:34:30.629 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.629+0000 7f8e33600640 1 Processor -- start 2024-05-09T04:34:30.629 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.629+0000 7f8e33600640 1 -- start start 2024-05-09T04:34:30.631 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.631+0000 7f8e33600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e2c06ef70 0x7f8e2c0affc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:30.631 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.631+0000 7f8e33600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f8e2c148c20 0x7f8e2c0b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:30.631 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.631+0000 7f8e33600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e2c072350 con 0x7f8e2c145880 2024-05-09T04:34:30.631 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.631+0000 7f8e33600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f8e2c072f60 con 0x7f8e2c148c20 2024-05-09T04:34:30.631 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.631+0000 7f8e33600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8e2c072630 con 0x7f8e2c06ef70 2024-05-09T04:34:30.631 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.631+0000 7f8e30c00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f8e2c145880 0x7f8e2c06e860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:44896/0 (socket says 172.21.15.145:44896) 2024-05-09T04:34:30.631 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.632+0000 7f8e30c00640 1 -- 172.21.15.145:0/180392849 learned_addr learned my addr 172.21.15.145:0/180392849 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:34:30.635 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.632+0000 7f8e2b400640 1 -- 172.21.15.145:0/180392849 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3603806067 0 0) 0x7f8e2c072350 con 0x7f8e2c145880 2024-05-09T04:34:30.635 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.632+0000 7f8e2b400640 1 -- 172.21.15.145:0/180392849 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8e10003520 con 0x7f8e2c145880 2024-05-09T04:34:30.635 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.632+0000 7f8e2be00640 1 --2- 172.21.15.145:0/180392849 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e2c06ef70 0x7f8e2c0affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:30.635 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.632+0000 7f8e31600640 1 --2- 172.21.15.145:0/180392849 >> v2:172.21.15.192:3300/0 conn(0x7f8e2c148c20 0x7f8e2c0b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:30.636 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.632+0000 7f8e2b400640 1 -- 172.21.15.145:0/180392849 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2414417964 0 0) 0x7f8e10003520 con 0x7f8e2c145880 2024-05-09T04:34:30.636 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.632+0000 7f8e2b400640 1 -- 172.21.15.145:0/180392849 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e2c06ef70 msgr2=0x7f8e2c0affc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:30.636 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.632+0000 7f8e2b400640 1 --2- 172.21.15.145:0/180392849 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e2c06ef70 0x7f8e2c0affc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:30.636 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.632+0000 7f8e2b400640 1 -- 172.21.15.145:0/180392849 >> v2:172.21.15.192:3300/0 conn(0x7f8e2c148c20 msgr2=0x7f8e2c0b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:30.636 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.632+0000 7f8e2b400640 1 --2- 172.21.15.145:0/180392849 >> v2:172.21.15.192:3300/0 conn(0x7f8e2c148c20 0x7f8e2c0b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:30.636 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.632+0000 7f8e2b400640 1 -- 172.21.15.145:0/180392849 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e2c072910 con 0x7f8e2c145880 2024-05-09T04:34:30.636 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.632+0000 7f8e2b400640 1 -- 172.21.15.145:0/180392849 <== mon.0 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3474413261 0 0) 0x7f8e1c0016f0 con 0x7f8e2c145880 2024-05-09T04:34:30.636 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.633+0000 7f8e2b400640 1 -- 172.21.15.145:0/180392849 >> v1:172.21.15.145:6789/0 conn(0x7f8e2c145880 legacy=0x7f8e2c06e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:30.636 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.633+0000 7f8e2b400640 1 --2- 172.21.15.145:0/180392849 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e2c06ef70 0x7f8e10003a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:30.636 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.633+0000 7f8e2b400640 1 --2- 172.21.15.145:0/180392849 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e100040d0 0x7f8e10004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:30.637 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.633+0000 7f8e2b400640 1 --2- 172.21.15.145:0/180392849 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e10004d40 0x7f8e100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:30.637 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.633+0000 7f8e2b400640 1 -- 172.21.15.145:0/180392849 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2c072630 con 0x7f8e10004d40 2024-05-09T04:34:30.637 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.633+0000 7f8e2b400640 1 -- 172.21.15.145:0/180392849 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1000bbd0 con 0x7f8e100040d0 2024-05-09T04:34:30.637 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.633+0000 7f8e2b400640 1 -- 172.21.15.145:0/180392849 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8e1000bdd0 con 0x7f8e2c06ef70 2024-05-09T04:34:30.637 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.633+0000 7f8e31600640 1 --2- 172.21.15.145:0/180392849 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e10004d40 0x7f8e100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:30.637 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.633+0000 7f8e30c00640 1 --2- 172.21.15.145:0/180392849 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e2c06ef70 0x7f8e10003a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:30.637 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.633+0000 7f8e2be00640 1 --2- 172.21.15.145:0/180392849 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e100040d0 0x7f8e10004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:30.637 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.633+0000 7f8e31600640 1 -- 172.21.15.145:0/180392849 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e2c06ef70 msgr2=0x7f8e10003a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:30.637 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.633+0000 7f8e31600640 1 --2- 172.21.15.145:0/180392849 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e2c06ef70 0x7f8e10003a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:30.637 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.633+0000 7f8e31600640 1 -- 172.21.15.145:0/180392849 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e100040d0 msgr2=0x7f8e10004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:30.637 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.633+0000 7f8e31600640 1 --2- 172.21.15.145:0/180392849 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e100040d0 0x7f8e10004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:30.637 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.633+0000 7f8e31600640 1 -- 172.21.15.145:0/180392849 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e10003520 con 0x7f8e10004d40 2024-05-09T04:34:30.638 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.633+0000 7f8e2be00640 1 --2- 172.21.15.145:0/180392849 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e100040d0 0x7f8e10004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:34:30.638 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.634+0000 7f8e31600640 1 --2- 172.21.15.145:0/180392849 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e10004d40 0x7f8e100051c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8e24007360 tx=0x7f8e10005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10510776ae3e1e24 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:30.638 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.635+0000 7f8e2b400640 1 -- 172.21.15.145:0/180392849 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e24009be0 con 0x7f8e10004d40 2024-05-09T04:34:30.638 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.635+0000 7f8e2b400640 1 -- 172.21.15.145:0/180392849 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e2402e620 con 0x7f8e10004d40 2024-05-09T04:34:30.638 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.635+0000 7f8e2b400640 1 -- 172.21.15.145:0/180392849 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e2402e9f0 con 0x7f8e10004d40 2024-05-09T04:34:30.638 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.635+0000 7f8e33600640 1 -- 172.21.15.145:0/180392849 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e10004d40 msgr2=0x7f8e100051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:30.638 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.635+0000 7f8e33600640 1 --2- 172.21.15.145:0/180392849 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e10004d40 0x7f8e100051c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8e24007360 tx=0x7f8e10005830 comp rx=0 tx=0).stop 2024-05-09T04:34:30.638 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.635+0000 7f8e31600640 1 -- 172.21.15.145:0/180392849 reap_dead start 2024-05-09T04:34:30.638 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.635+0000 7f8e33600640 1 -- 172.21.15.145:0/180392849 shutdown_connections 2024-05-09T04:34:30.638 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.635+0000 7f8e33600640 1 -- 172.21.15.145:0/180392849 >> 172.21.15.145:0/180392849 conn(0x7f8e2c06c870 msgr2=0x7f8e2c06cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:30.638 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.635+0000 7f8e33600640 1 -- 172.21.15.145:0/180392849 shutdown_connections 2024-05-09T04:34:30.639 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.636+0000 7f8e33600640 1 -- 172.21.15.145:0/180392849 wait complete. 2024-05-09T04:34:30.641 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.637+0000 7f8e33600640 1 Processor -- start 2024-05-09T04:34:30.641 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.637+0000 7f8e33600640 1 -- start start 2024-05-09T04:34:30.641 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.638+0000 7f8e33600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e10004d40 0x7f8e2c15e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:30.641 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.638+0000 7f8e33600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e2c13a230 0x7f8e2c162280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:30.641 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.638+0000 7f8e33600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e2c15ed40 0x7f8e2c073610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:30.641 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.638+0000 7f8e33600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2c0affc0 con 0x7f8e2c15ed40 2024-05-09T04:34:30.641 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.638+0000 7f8e33600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2c145c60 con 0x7f8e2c13a230 2024-05-09T04:34:30.641 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.638+0000 7f8e33600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8e2c0b0140 con 0x7f8e10004d40 2024-05-09T04:34:30.641 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.639+0000 7f8e31600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e2c15ed40 0x7f8e2c073610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:30.641 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.639+0000 7f8e31600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e2c15ed40 0x7f8e2c073610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:40678/0 (socket says 172.21.15.145:40678) 2024-05-09T04:34:30.641 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.639+0000 7f8e31600640 1 -- 172.21.15.145:0/26410551 learned_addr learned my addr 172.21.15.145:0/26410551 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:30.642 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.639+0000 7f8e2be00640 1 --2- 172.21.15.145:0/26410551 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e2c13a230 0x7f8e2c162280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:30.642 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.639+0000 7f8e30c00640 1 --2- 172.21.15.145:0/26410551 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e10004d40 0x7f8e2c15e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:30.642 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.639+0000 7f8e31600640 1 -- 172.21.15.145:0/26410551 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e10004d40 msgr2=0x7f8e2c15e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:30.642 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.639+0000 7f8e31600640 1 --2- 172.21.15.145:0/26410551 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e10004d40 0x7f8e2c15e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:30.642 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.639+0000 7f8e31600640 1 -- 172.21.15.145:0/26410551 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e2c13a230 msgr2=0x7f8e2c162280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:30.642 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.639+0000 7f8e31600640 1 --2- 172.21.15.145:0/26410551 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e2c13a230 0x7f8e2c162280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:30.642 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.639+0000 7f8e31600640 1 -- 172.21.15.145:0/26410551 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e2c13e4a0 con 0x7f8e2c15ed40 2024-05-09T04:34:30.642 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.640+0000 7f8e31600640 1 --2- 172.21.15.145:0/26410551 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e2c15ed40 0x7f8e2c073610 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8e2402b0a0 tx=0x7f8e24004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:30.642 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.640+0000 7f8e28c00640 1 -- 172.21.15.145:0/26410551 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e24009be0 con 0x7f8e2c15ed40 2024-05-09T04:34:30.642 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.640+0000 7f8e28c00640 1 -- 172.21.15.145:0/26410551 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e24045070 con 0x7f8e2c15ed40 2024-05-09T04:34:30.642 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.640+0000 7f8e33600640 1 -- 172.21.15.145:0/26410551 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e2c13e780 con 0x7f8e2c15ed40 2024-05-09T04:34:30.642 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.640+0000 7f8e28c00640 1 -- 172.21.15.145:0/26410551 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e24036a80 con 0x7f8e2c15ed40 2024-05-09T04:34:30.642 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.640+0000 7f8e33600640 1 -- 172.21.15.145:0/26410551 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e2c13ed40 con 0x7f8e2c15ed40 2024-05-09T04:34:30.643 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.641+0000 7f8e33600640 1 -- 172.21.15.145:0/26410551 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e2c06a310 con 0x7f8e2c15ed40 2024-05-09T04:34:30.643 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.642+0000 7f8e28c00640 1 -- 172.21.15.145:0/26410551 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8e24036c20 con 0x7f8e2c15ed40 2024-05-09T04:34:30.643 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.643+0000 7f8e28c00640 1 --2- 172.21.15.145:0/26410551 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f8e08076300 0x7f8e080787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:30.643 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.643+0000 7f8e28c00640 1 -- 172.21.15.145:0/26410551 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8e240b84d0 con 0x7f8e2c15ed40 2024-05-09T04:34:30.644 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.643+0000 7f8e30c00640 1 --2- 172.21.15.145:0/26410551 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f8e08076300 0x7f8e080787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:30.644 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.644+0000 7f8e30c00640 1 --2- 172.21.15.145:0/26410551 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f8e08076300 0x7f8e080787c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c001c50 tx=0x7f8e1c001be0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:30.647 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.648+0000 7f8e28c00640 1 -- 172.21.15.145:0/26410551 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8e2403d070 con 0x7f8e2c15ed40 2024-05-09T04:34:30.890 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.890+0000 7f8e33600640 1 -- 172.21.15.145:0/26410551 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8e2c13ef10 con 0x7f8e2c15ed40 2024-05-09T04:34:30.891 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.892+0000 7f8e28c00640 1 -- 172.21.15.145:0/26410551 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5187 (secure 0 0 0) 0x7f8e24032030 con 0x7f8e2c15ed40 2024-05-09T04:34:30.892 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:34:30.892 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/1284540932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":1284540932},{"type":"v1","addr":"172.21.15.145:6835","nonce":1284540932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/1308656248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1308656248},{"type":"v1","addr":"172.21.15.145:6837","nonce":1308656248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3323136057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3323136057},{"type":"v1","addr":"172.21.15.192:6833","nonce":3323136057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T04:33:51.628921+0000","modified":"2024-05-09T04:33:54.699907+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.192:6835/1113351558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1113351558},{"type":"v1","addr":"172.21.15.192:6835","nonce":1113351558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4270,"qdb_cluster":[4270]},"id":1}]} 2024-05-09T04:34:30.892 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 8 2024-05-09T04:34:30.894 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.894+0000 7f8e33600640 1 -- 172.21.15.145:0/26410551 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f8e08076300 msgr2=0x7f8e080787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:30.894 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.894+0000 7f8e33600640 1 --2- 172.21.15.145:0/26410551 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f8e08076300 0x7f8e080787c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c001c50 tx=0x7f8e1c001be0 comp rx=0 tx=0).stop 2024-05-09T04:34:30.894 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.894+0000 7f8e33600640 1 -- 172.21.15.145:0/26410551 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e2c15ed40 msgr2=0x7f8e2c073610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:30.894 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.894+0000 7f8e33600640 1 --2- 172.21.15.145:0/26410551 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e2c15ed40 0x7f8e2c073610 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8e2402b0a0 tx=0x7f8e24004ab0 comp rx=0 tx=0).stop 2024-05-09T04:34:30.894 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.894+0000 7f8e33600640 1 -- 172.21.15.145:0/26410551 shutdown_connections 2024-05-09T04:34:30.894 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.894+0000 7f8e33600640 1 --2- 172.21.15.145:0/26410551 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f8e08076300 0x7f8e080787c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:30.895 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.894+0000 7f8e33600640 1 --2- 172.21.15.145:0/26410551 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8e2c15ed40 0x7f8e2c073610 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:30.895 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.894+0000 7f8e33600640 1 --2- 172.21.15.145:0/26410551 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e2c13a230 0x7f8e2c162280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:30.895 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.894+0000 7f8e33600640 1 --2- 172.21.15.145:0/26410551 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e10004d40 0x7f8e2c15e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:30.895 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.894+0000 7f8e33600640 1 -- 172.21.15.145:0/26410551 >> 172.21.15.145:0/26410551 conn(0x7f8e2c06c870 msgr2=0x7f8e2c06e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:30.895 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.894+0000 7f8e33600640 1 -- 172.21.15.145:0/26410551 shutdown_connections 2024-05-09T04:34:30.895 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:30.894+0000 7f8e33600640 1 -- 172.21.15.145:0/26410551 wait complete. 2024-05-09T04:34:30.911 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T04:34:31.102 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.102+0000 7f8d1f200640 1 Processor -- start 2024-05-09T04:34:31.102 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.103+0000 7f8d1f200640 1 -- start start 2024-05-09T04:34:31.104 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.104+0000 7f8d1f200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f8d201572c0 0x7f8d2015deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:31.104 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.104+0000 7f8d1f200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d200ae710 0x7f8d20161220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:31.104 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.104+0000 7f8d1f200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f8d200cff30 0x7f8d200d0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:31.104 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.104+0000 7f8d1f200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f8d200b6290 con 0x7f8d200cff30 2024-05-09T04:34:31.104 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.104+0000 7f8d1f200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f8d200d08c0 con 0x7f8d201572c0 2024-05-09T04:34:31.104 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.104+0000 7f8d1f200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8d200aecd0 con 0x7f8d200ae710 2024-05-09T04:34:31.105 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.104+0000 7f8d1e800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f8d200cff30 0x7f8d200d0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:31.105 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.105+0000 7f8d1e800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f8d200cff30 0x7f8d200d0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:51018/0 (socket says 172.21.15.145:51018) 2024-05-09T04:34:31.105 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.105+0000 7f8d1e800640 1 -- 172.21.15.145:0/3155052694 learned_addr learned my addr 172.21.15.145:0/3155052694 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:31.106 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.105+0000 7f8d17e00640 1 --2- 172.21.15.145:0/3155052694 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d200ae710 0x7f8d20161220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:31.106 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.105+0000 7f8d1de00640 1 --2- 172.21.15.145:0/3155052694 >> v2:172.21.15.192:3300/0 conn(0x7f8d201572c0 0x7f8d2015deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:31.106 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.105+0000 7f8d1e800640 1 -- 172.21.15.145:0/3155052694 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d200ae710 msgr2=0x7f8d20161220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:31.106 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.105+0000 7f8d1e800640 1 --2- 172.21.15.145:0/3155052694 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d200ae710 0x7f8d20161220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:31.106 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.105+0000 7f8d1e800640 1 -- 172.21.15.145:0/3155052694 >> v2:172.21.15.192:3300/0 conn(0x7f8d201572c0 msgr2=0x7f8d2015deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:31.106 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.105+0000 7f8d1e800640 1 --2- 172.21.15.145:0/3155052694 >> v2:172.21.15.192:3300/0 conn(0x7f8d201572c0 0x7f8d2015deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:31.106 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.105+0000 7f8d1e800640 1 -- 172.21.15.145:0/3155052694 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d200aefb0 con 0x7f8d200cff30 2024-05-09T04:34:31.106 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.105+0000 7f8d1de00640 1 --2- 172.21.15.145:0/3155052694 >> v2:172.21.15.192:3300/0 conn(0x7f8d201572c0 0x7f8d2015deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:34:31.107 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.106+0000 7f8d1e800640 1 --2- 172.21.15.145:0/3155052694 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8d200cff30 0x7f8d200d0380 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8d2006ffc0 tx=0x7f8d1802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe9bdbcd5a8b8934 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:31.107 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.107+0000 7f8d1d400640 1 -- 172.21.15.145:0/3155052694 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d18036070 con 0x7f8d200cff30 2024-05-09T04:34:31.108 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.107+0000 7f8d1d400640 1 -- 172.21.15.145:0/3155052694 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d18031600 con 0x7f8d200cff30 2024-05-09T04:34:31.108 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.107+0000 7f8d1d400640 1 -- 172.21.15.145:0/3155052694 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d180319d0 con 0x7f8d200cff30 2024-05-09T04:34:31.108 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.107+0000 7f8d1f200640 1 -- 172.21.15.145:0/3155052694 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8d200cff30 msgr2=0x7f8d200d0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:31.108 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.107+0000 7f8d1f200640 1 --2- 172.21.15.145:0/3155052694 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8d200cff30 0x7f8d200d0380 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8d2006ffc0 tx=0x7f8d1802a6a0 comp rx=0 tx=0).stop 2024-05-09T04:34:31.108 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.108+0000 7f8d1f200640 1 -- 172.21.15.145:0/3155052694 shutdown_connections 2024-05-09T04:34:31.108 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.108+0000 7f8d1f200640 1 --2- 172.21.15.145:0/3155052694 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8d200cff30 0x7f8d200d0380 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:31.108 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.108+0000 7f8d1f200640 1 --2- 172.21.15.145:0/3155052694 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d200ae710 0x7f8d20161220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:31.108 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.108+0000 7f8d1f200640 1 --2- 172.21.15.145:0/3155052694 >> v2:172.21.15.192:3300/0 conn(0x7f8d201572c0 0x7f8d2015deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:31.108 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.108+0000 7f8d1f200640 1 -- 172.21.15.145:0/3155052694 >> 172.21.15.145:0/3155052694 conn(0x7f8d2006c8a0 msgr2=0x7f8d2006ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:31.108 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.108+0000 7f8d1f200640 1 -- 172.21.15.145:0/3155052694 shutdown_connections 2024-05-09T04:34:31.108 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.108+0000 7f8d1f200640 1 -- 172.21.15.145:0/3155052694 wait complete. 2024-05-09T04:34:31.109 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.110+0000 7f8d1f200640 1 Processor -- start 2024-05-09T04:34:31.110 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.110+0000 7f8d1f200640 1 -- start start 2024-05-09T04:34:31.112 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.111+0000 7f8d1f200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d200ae710 0x7f8d2014a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:31.112 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.111+0000 7f8d1f200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d200cff30 0x7f8d2006f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:31.113 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.111+0000 7f8d1f200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8d201572c0 0x7f8d20157e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:31.113 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.111+0000 7f8d1f200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8d20160d00 con 0x7f8d201572c0 2024-05-09T04:34:31.113 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.111+0000 7f8d1f200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8d20160a20 con 0x7f8d200cff30 2024-05-09T04:34:31.113 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.111+0000 7f8d1f200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8d20161760 con 0x7f8d200ae710 2024-05-09T04:34:31.113 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.111+0000 7f8d1e800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8d201572c0 0x7f8d20157e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:31.113 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.112+0000 7f8d1e800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8d201572c0 0x7f8d20157e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:51020/0 (socket says 172.21.15.145:51020) 2024-05-09T04:34:31.113 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.112+0000 7f8d1e800640 1 -- 172.21.15.145:0/2887355579 learned_addr learned my addr 172.21.15.145:0/2887355579 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:31.113 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.112+0000 7f8d1de00640 1 --2- 172.21.15.145:0/2887355579 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d200ae710 0x7f8d2014a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:31.113 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.112+0000 7f8d17e00640 1 --2- 172.21.15.145:0/2887355579 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d200cff30 0x7f8d2006f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:31.113 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.112+0000 7f8d1e800640 1 -- 172.21.15.145:0/2887355579 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d200ae710 msgr2=0x7f8d2014a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:31.113 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.112+0000 7f8d1e800640 1 --2- 172.21.15.145:0/2887355579 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d200ae710 0x7f8d2014a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:31.113 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.112+0000 7f8d1e800640 1 -- 172.21.15.145:0/2887355579 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d200cff30 msgr2=0x7f8d2006f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:31.113 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.112+0000 7f8d1e800640 1 --2- 172.21.15.145:0/2887355579 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d200cff30 0x7f8d2006f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:31.113 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.112+0000 7f8d1e800640 1 -- 172.21.15.145:0/2887355579 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d200b1650 con 0x7f8d201572c0 2024-05-09T04:34:31.114 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.112+0000 7f8d17e00640 1 --2- 172.21.15.145:0/2887355579 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d200cff30 0x7f8d2006f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:34:31.114 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.112+0000 7f8d1e800640 1 --2- 172.21.15.145:0/2887355579 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8d201572c0 0x7f8d20157e90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8d18009f40 tx=0x7f8d18004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:31.114 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.112+0000 7f8d16000640 1 -- 172.21.15.145:0/2887355579 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d18036050 con 0x7f8d201572c0 2024-05-09T04:34:31.114 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.113+0000 7f8d16000640 1 -- 172.21.15.145:0/2887355579 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d18042070 con 0x7f8d201572c0 2024-05-09T04:34:31.114 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.113+0000 7f8d1f200640 1 -- 172.21.15.145:0/2887355579 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d200b1930 con 0x7f8d201572c0 2024-05-09T04:34:31.114 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.113+0000 7f8d16000640 1 -- 172.21.15.145:0/2887355579 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d18009510 con 0x7f8d201572c0 2024-05-09T04:34:31.114 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.113+0000 7f8d1f200640 1 -- 172.21.15.145:0/2887355579 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d200b1ef0 con 0x7f8d201572c0 2024-05-09T04:34:31.115 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.114+0000 7f8d1f200640 1 -- 172.21.15.145:0/2887355579 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d2006a310 con 0x7f8d201572c0 2024-05-09T04:34:31.115 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.115+0000 7f8d16000640 1 -- 172.21.15.145:0/2887355579 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8d18007bd0 con 0x7f8d201572c0 2024-05-09T04:34:31.115 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.116+0000 7f8d16000640 1 --2- 172.21.15.145:0/2887355579 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f8cfc0762d0 0x7f8cfc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:31.115 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.116+0000 7f8d16000640 1 -- 172.21.15.145:0/2887355579 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8d180b85e0 con 0x7f8d201572c0 2024-05-09T04:34:31.116 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.116+0000 7f8d1de00640 1 --2- 172.21.15.145:0/2887355579 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f8cfc0762d0 0x7f8cfc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:31.116 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.117+0000 7f8d1de00640 1 --2- 172.21.15.145:0/2887355579 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f8cfc0762d0 0x7f8cfc078790 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8d10006fd0 tx=0x7f8d10008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:31.120 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.120+0000 7f8d16000640 1 -- 172.21.15.145:0/2887355579 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8d18081e10 con 0x7f8d201572c0 2024-05-09T04:34:31.365 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.366+0000 7f8d1f200640 1 -- 172.21.15.145:0/2887355579 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8d200b45d0 con 0x7f8d201572c0 2024-05-09T04:34:31.366 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.366+0000 7f8d16000640 1 -- 172.21.15.145:0/2887355579 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5187 (secure 0 0 0) 0x7f8d18081710 con 0x7f8d201572c0 2024-05-09T04:34:31.366 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:34:31.367 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/1284540932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":1284540932},{"type":"v1","addr":"172.21.15.145:6835","nonce":1284540932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/1308656248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":1308656248},{"type":"v1","addr":"172.21.15.145:6837","nonce":1308656248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6833/3323136057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3323136057},{"type":"v1","addr":"172.21.15.192:6833","nonce":3323136057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T04:33:51.628921+0000","modified":"2024-05-09T04:33:54.699907+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.192:6835/1113351558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1113351558},{"type":"v1","addr":"172.21.15.192:6835","nonce":1113351558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4270,"qdb_cluster":[4270]},"id":1}]} 2024-05-09T04:34:31.367 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 8 2024-05-09T04:34:31.369 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.369+0000 7f8d1f200640 1 -- 172.21.15.145:0/2887355579 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f8cfc0762d0 msgr2=0x7f8cfc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:31.369 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.369+0000 7f8d1f200640 1 --2- 172.21.15.145:0/2887355579 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f8cfc0762d0 0x7f8cfc078790 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8d10006fd0 tx=0x7f8d10008040 comp rx=0 tx=0).stop 2024-05-09T04:34:31.369 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.369+0000 7f8d1f200640 1 -- 172.21.15.145:0/2887355579 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8d201572c0 msgr2=0x7f8d20157e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:31.369 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.369+0000 7f8d1f200640 1 --2- 172.21.15.145:0/2887355579 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8d201572c0 0x7f8d20157e90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8d18009f40 tx=0x7f8d18004a40 comp rx=0 tx=0).stop 2024-05-09T04:34:31.369 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.369+0000 7f8d1f200640 1 -- 172.21.15.145:0/2887355579 shutdown_connections 2024-05-09T04:34:31.370 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.369+0000 7f8d1f200640 1 --2- 172.21.15.145:0/2887355579 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f8cfc0762d0 0x7f8cfc078790 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:31.370 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.369+0000 7f8d1f200640 1 --2- 172.21.15.145:0/2887355579 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8d201572c0 0x7f8d20157e90 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:31.370 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.369+0000 7f8d1f200640 1 --2- 172.21.15.145:0/2887355579 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d200cff30 0x7f8d2006f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:31.370 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.369+0000 7f8d1f200640 1 --2- 172.21.15.145:0/2887355579 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d200ae710 0x7f8d2014a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:31.370 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.369+0000 7f8d1f200640 1 -- 172.21.15.145:0/2887355579 >> 172.21.15.145:0/2887355579 conn(0x7f8d2006c8a0 msgr2=0x7f8d2015e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:31.370 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.370+0000 7f8d1f200640 1 -- 172.21.15.145:0/2887355579 shutdown_connections 2024-05-09T04:34:31.370 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.370+0000 7f8d1f200640 1 -- 172.21.15.145:0/2887355579 wait complete. 2024-05-09T04:34:31.388 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T04:34:31.584 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.584+0000 7fd9c5a00640 1 Processor -- start 2024-05-09T04:34:31.584 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.584+0000 7fd9c5a00640 1 -- start start 2024-05-09T04:34:31.585 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.586+0000 7fd9c5a00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd9c015fbc0 0x7fd9c0142200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:31.586 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.586+0000 7fd9c5a00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9c013f340 con 0x7fd9c015ffa0 2024-05-09T04:34:31.586 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.586+0000 7fd9c5a00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9c00cffe0 con 0x7fd9c00726b0 2024-05-09T04:34:31.586 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.586+0000 7fd9c5a00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fd9c0072220 con 0x7fd9c015fbc0 2024-05-09T04:34:31.586 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.586+0000 7fd9be000640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7fd9c015ffa0 0x7fd9c00cf8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:59076/0 (socket says 172.21.15.145:59076) 2024-05-09T04:34:31.586 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.586+0000 7fd9be000640 1 -- 172.21.15.145:0/3085430136 learned_addr learned my addr 172.21.15.145:0/3085430136 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:34:31.588 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.587+0000 7fd9bea00640 1 --2- 172.21.15.145:0/3085430136 >> v2:172.21.15.192:3300/0 conn(0x7fd9c015fbc0 0x7fd9c0142200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:31.588 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.587+0000 7fd9bd600640 1 -- 172.21.15.145:0/3085430136 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3671347693 0 0) 0x7fd9ac001110 con 0x7fd9c015ffa0 2024-05-09T04:34:31.588 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.587+0000 7fd9bd600640 1 -- 172.21.15.145:0/3085430136 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd9a4005730 con 0x7fd9c015ffa0 2024-05-09T04:34:31.588 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.587+0000 7fd9bd600640 1 -- 172.21.15.145:0/3085430136 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 458188355 0 0) 0x7fd9a4005730 con 0x7fd9c015ffa0 2024-05-09T04:34:31.588 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.587+0000 7fd9bd600640 1 -- 172.21.15.145:0/3085430136 >> v2:172.21.15.192:3300/0 conn(0x7fd9c015fbc0 msgr2=0x7fd9c0142200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:31.589 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.587+0000 7fd9bd600640 1 --2- 172.21.15.145:0/3085430136 >> v2:172.21.15.192:3300/0 conn(0x7fd9c015fbc0 0x7fd9c0142200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:31.589 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.587+0000 7fd9bd600640 1 -- 172.21.15.145:0/3085430136 >> v1:172.21.15.192:6789/0 conn(0x7fd9c00726b0 legacy=0x7fd9c0072b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:31.589 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.587+0000 7fd9bd600640 1 -- 172.21.15.145:0/3085430136 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9c00d02b0 con 0x7fd9c015ffa0 2024-05-09T04:34:31.589 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.587+0000 7fd9bd600640 1 -- 172.21.15.145:0/3085430136 <== mon.0 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3474413261 0 0) 0x7fd9ac0018c0 con 0x7fd9c015ffa0 2024-05-09T04:34:31.589 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.587+0000 7fd9bd600640 1 -- 172.21.15.145:0/3085430136 >> v1:172.21.15.145:6789/0 conn(0x7fd9c015ffa0 legacy=0x7fd9c00cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:31.589 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.587+0000 7fd9bd600640 1 --2- 172.21.15.145:0/3085430136 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9a4005cb0 0x7fd9a40060d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:31.589 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.587+0000 7fd9bd600640 1 --2- 172.21.15.145:0/3085430136 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd9a4006a10 0x7fd9a4006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:31.589 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.587+0000 7fd9bd600640 1 --2- 172.21.15.145:0/3085430136 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9a40075d0 0x7fd9a400dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:31.589 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.587+0000 7fd9bd600640 1 -- 172.21.15.145:0/3085430136 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c0072220 con 0x7fd9a4005cb0 2024-05-09T04:34:31.589 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.587+0000 7fd9bd600640 1 -- 172.21.15.145:0/3085430136 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a400e4b0 con 0x7fd9a40075d0 2024-05-09T04:34:31.590 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.587+0000 7fd9bd600640 1 -- 172.21.15.145:0/3085430136 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd9a400e6b0 con 0x7fd9a4006a10 2024-05-09T04:34:31.590 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.587+0000 7fd9bea00640 1 --2- 172.21.15.145:0/3085430136 >> v2:172.21.15.192:3300/0 conn(0x7fd9c015fbc0 0x7fd9c0142200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:34:31.590 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.588+0000 7fd9bea00640 1 --2- 172.21.15.145:0/3085430136 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9a4005cb0 0x7fd9a40060d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:31.590 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.588+0000 7fd9be000640 1 --2- 172.21.15.145:0/3085430136 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd9a4006a10 0x7fd9a4006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:31.590 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.588+0000 7fd9bf400640 1 --2- 172.21.15.145:0/3085430136 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9a40075d0 0x7fd9a400dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:31.590 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.588+0000 7fd9bea00640 1 -- 172.21.15.145:0/3085430136 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd9a4006a10 msgr2=0x7fd9a4006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:31.590 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.588+0000 7fd9bea00640 1 --2- 172.21.15.145:0/3085430136 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd9a4006a10 0x7fd9a4006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:31.590 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.588+0000 7fd9bea00640 1 -- 172.21.15.145:0/3085430136 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9a40075d0 msgr2=0x7fd9a400dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:31.590 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.588+0000 7fd9bea00640 1 --2- 172.21.15.145:0/3085430136 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9a40075d0 0x7fd9a400dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:31.590 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.588+0000 7fd9bea00640 1 -- 172.21.15.145:0/3085430136 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9a4005730 con 0x7fd9a4005cb0 2024-05-09T04:34:31.590 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.588+0000 7fd9bf400640 1 -- 172.21.15.145:0/3085430136 reap_dead start 2024-05-09T04:34:31.592 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.589+0000 7fd9bea00640 1 --2- 172.21.15.145:0/3085430136 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9a4005cb0 0x7fd9a40060d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd9b00098a0 tx=0x7fd9a4007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abc30ef0770a2065 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:31.592 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.589+0000 7fd9bd600640 1 -- 172.21.15.145:0/3085430136 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b0037070 con 0x7fd9a4005cb0 2024-05-09T04:34:31.592 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.589+0000 7fd9bd600640 1 -- 172.21.15.145:0/3085430136 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9b002aca0 con 0x7fd9a4005cb0 2024-05-09T04:34:31.592 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.589+0000 7fd9bd600640 1 -- 172.21.15.145:0/3085430136 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b000f040 con 0x7fd9a4005cb0 2024-05-09T04:34:31.592 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.590+0000 7fd9c5a00640 1 -- 172.21.15.145:0/3085430136 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9a4005cb0 msgr2=0x7fd9a40060d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:31.592 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.590+0000 7fd9c5a00640 1 --2- 172.21.15.145:0/3085430136 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9a4005cb0 0x7fd9a40060d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd9b00098a0 tx=0x7fd9a4007a40 comp rx=0 tx=0).stop 2024-05-09T04:34:31.593 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.590+0000 7fd9c5a00640 1 -- 172.21.15.145:0/3085430136 shutdown_connections 2024-05-09T04:34:31.593 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.590+0000 7fd9c5a00640 1 --2- 172.21.15.145:0/3085430136 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9a4005cb0 0x7fd9a40060d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:31.593 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.590+0000 7fd9c5a00640 1 -- 172.21.15.145:0/3085430136 >> 172.21.15.145:0/3085430136 conn(0x7fd9c006c870 msgr2=0x7fd9c006cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:31.593 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.590+0000 7fd9c5a00640 1 -- 172.21.15.145:0/3085430136 shutdown_connections 2024-05-09T04:34:31.593 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.590+0000 7fd9c5a00640 1 -- 172.21.15.145:0/3085430136 wait complete. 2024-05-09T04:34:31.593 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.592+0000 7fd9c5a00640 1 Processor -- start 2024-05-09T04:34:31.593 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.592+0000 7fd9c5a00640 1 -- start start 2024-05-09T04:34:31.595 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.593+0000 7fd9c5a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9a4005cb0 0x7fd9c0152b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:31.595 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.593+0000 7fd9c5a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9c00b4fc0 0x7fd9c00b5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:31.595 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.593+0000 7fd9c5a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd9c0139b00 0x7fd9c0139f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:31.595 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.593+0000 7fd9c5a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c00d0b50 con 0x7fd9a4005cb0 2024-05-09T04:34:31.595 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.593+0000 7fd9c5a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c00d0870 con 0x7fd9c00b4fc0 2024-05-09T04:34:31.595 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.593+0000 7fd9c5a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd9c0160430 con 0x7fd9c0139b00 2024-05-09T04:34:31.595 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.593+0000 7fd9bea00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9a4005cb0 0x7fd9c0152b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:31.595 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.593+0000 7fd9bea00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9a4005cb0 0x7fd9c0152b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:51042/0 (socket says 172.21.15.145:51042) 2024-05-09T04:34:31.595 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.593+0000 7fd9bea00640 1 -- 172.21.15.145:0/1098082586 learned_addr learned my addr 172.21.15.145:0/1098082586 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:31.595 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.593+0000 7fd9bea00640 1 -- 172.21.15.145:0/1098082586 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd9c0139b00 msgr2=0x7fd9c0139f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:31.596 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.593+0000 7fd9bf400640 1 --2- 172.21.15.145:0/1098082586 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd9c0139b00 0x7fd9c0139f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:31.596 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.593+0000 7fd9bea00640 1 --2- 172.21.15.145:0/1098082586 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd9c0139b00 0x7fd9c0139f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:31.596 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.593+0000 7fd9bea00640 1 -- 172.21.15.145:0/1098082586 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9c00b4fc0 msgr2=0x7fd9c00b5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:31.596 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.593+0000 7fd9bea00640 1 --2- 172.21.15.145:0/1098082586 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9c00b4fc0 0x7fd9c00b5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:31.596 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.593+0000 7fd9bea00640 1 -- 172.21.15.145:0/1098082586 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9c00af1a0 con 0x7fd9a4005cb0 2024-05-09T04:34:31.597 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.594+0000 7fd9bea00640 1 --2- 172.21.15.145:0/1098082586 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9a4005cb0 0x7fd9c0152b80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd9b002a790 tx=0x7fd9b0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:31.597 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.594+0000 7fd9b6a00640 1 -- 172.21.15.145:0/1098082586 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b0037050 con 0x7fd9a4005cb0 2024-05-09T04:34:31.597 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.594+0000 7fd9b6a00640 1 -- 172.21.15.145:0/1098082586 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9b0043070 con 0x7fd9a4005cb0 2024-05-09T04:34:31.597 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.594+0000 7fd9b6a00640 1 -- 172.21.15.145:0/1098082586 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b003c830 con 0x7fd9a4005cb0 2024-05-09T04:34:31.597 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.594+0000 7fd9c5a00640 1 -- 172.21.15.145:0/1098082586 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9c00af480 con 0x7fd9a4005cb0 2024-05-09T04:34:31.597 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.594+0000 7fd9c5a00640 1 -- 172.21.15.145:0/1098082586 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9c00afa40 con 0x7fd9a4005cb0 2024-05-09T04:34:31.597 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.595+0000 7fd9c5a00640 1 -- 172.21.15.145:0/1098082586 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9c006a310 con 0x7fd9a4005cb0 2024-05-09T04:34:31.598 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.596+0000 7fd9b6a00640 1 -- 172.21.15.145:0/1098082586 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd9b003c9d0 con 0x7fd9a4005cb0 2024-05-09T04:34:31.598 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.597+0000 7fd9b6a00640 1 --2- 172.21.15.145:0/1098082586 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fd9a00762d0 0x7fd9a0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:31.598 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.597+0000 7fd9b6a00640 1 -- 172.21.15.145:0/1098082586 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd9b00b7f60 con 0x7fd9a4005cb0 2024-05-09T04:34:31.598 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.598+0000 7fd9be000640 1 --2- 172.21.15.145:0/1098082586 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fd9a00762d0 0x7fd9a0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:31.598 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.598+0000 7fd9be000640 1 --2- 172.21.15.145:0/1098082586 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fd9a00762d0 0x7fd9a0078790 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac007a30 tx=0x7fd9ac001dd0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:31.602 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.602+0000 7fd9b6a00640 1 -- 172.21.15.145:0/1098082586 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd9b0081820 con 0x7fd9a4005cb0 2024-05-09T04:34:31.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.810+0000 7fd9c5a00640 1 -- 172.21.15.145:0/1098082586 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd9c00afc10 con 0x7fd9a4005cb0 2024-05-09T04:34:31.812 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.812+0000 7fd9b6a00640 1 -- 172.21.15.145:0/1098082586 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16653 (secure 0 0 0) 0x7fd9b000f000 con 0x7fd9a4005cb0 2024-05-09T04:34:31.813 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:34:31.813 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":20,"fsid":"957ee6c1-073b-44cf-9b93-9491b3fdfc1d","created":"2024-05-09T04:33:26.544367+0000","modified":"2024-05-09T04:33:53.178020+0000","last_up_change":"2024-05-09T04:33:39.039318+0000","last_in_change":"2024-05-09T04:33:29.902361+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T04:33:39.795344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T04:33:49.651743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T04:33:50.596065+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22e550ad-847b-4952-80e6-63d859ed1a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6801","nonce":3182572912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6803","nonce":3182572912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6807","nonce":3182572912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6805","nonce":3182572912}]},"public_addr":"172.21.15.145:6801/3182572912","cluster_addr":"172.21.15.145:6803/3182572912","heartbeat_back_addr":"172.21.15.145:6807/3182572912","heartbeat_front_addr":"172.21.15.145:6805/3182572912","state":["exists","up"]},{"osd":1,"uuid":"0f6b9fca-0c41-4145-8d74-598ddb42f2a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6825","nonce":1845225555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6827","nonce":1845225555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6831","nonce":1845225555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6829","nonce":1845225555}]},"public_addr":"172.21.15.145:6825/1845225555","cluster_addr":"172.21.15.145:6827/1845225555","heartbeat_back_addr":"172.21.15.145:6831/1845225555","heartbeat_front_addr":"172.21.15.145:6829/1845225555","state":["exists","up"]},{"osd":2,"uuid":"b3b1f67e-5790-483f-8610-e05a5bce4c30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6817","nonce":2168958993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6819","nonce":2168958993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6823","nonce":2168958993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6821","nonce":2168958993}]},"public_addr":"172.21.15.145:6817/2168958993","cluster_addr":"172.21.15.145:6819/2168958993","heartbeat_back_addr":"172.21.15.145:6823/2168958993","heartbeat_front_addr":"172.21.15.145:6821/2168958993","state":["exists","up"]},{"osd":3,"uuid":"884787a5-e037-4d31-9aae-743030eba4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6809","nonce":337009826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6811","nonce":337009826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6815","nonce":337009826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6813","nonce":337009826}]},"public_addr":"172.21.15.145:6809/337009826","cluster_addr":"172.21.15.145:6811/337009826","heartbeat_back_addr":"172.21.15.145:6815/337009826","heartbeat_front_addr":"172.21.15.145:6813/337009826","state":["exists","up"]},{"osd":4,"uuid":"0759fc1e-c5fc-4a2f-86bb-e8eccd83f833","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6818","nonce":3117700995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6821","nonce":3117700995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6827","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6829","nonce":3117700995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6823","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6825","nonce":3117700995}]},"public_addr":"172.21.15.192:6818/3117700995","cluster_addr":"172.21.15.192:6821/3117700995","heartbeat_back_addr":"172.21.15.192:6829/3117700995","heartbeat_front_addr":"172.21.15.192:6825/3117700995","state":["exists","up"]},{"osd":5,"uuid":"9443cc22-9379-46e3-9173-98b592bf2c16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6801","nonce":1328373268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6803","nonce":1328373268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6807","nonce":1328373268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6805","nonce":1328373268}]},"public_addr":"172.21.15.192:6801/1328373268","cluster_addr":"172.21.15.192:6803/1328373268","heartbeat_back_addr":"172.21.15.192:6807/1328373268","heartbeat_front_addr":"172.21.15.192:6805/1328373268","state":["exists","up"]},{"osd":6,"uuid":"d05a3311-505a-45b5-85b9-aa3113e96987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6817","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6820","nonce":1868011057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6824","nonce":1868011057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6831","nonce":1868011057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6828","nonce":1868011057}]},"public_addr":"172.21.15.192:6820/1868011057","cluster_addr":"172.21.15.192:6824/1868011057","heartbeat_back_addr":"172.21.15.192:6831/1868011057","heartbeat_front_addr":"172.21.15.192:6828/1868011057","state":["exists","up"]},{"osd":7,"uuid":"c7c23dbc-2029-43ec-9f53-25f3d58019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6809","nonce":3368512157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6811","nonce":3368512157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6815","nonce":3368512157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6813","nonce":3368512157}]},"public_addr":"172.21.15.192:6809/3368512157","cluster_addr":"172.21.15.192:6811/3368512157","heartbeat_back_addr":"172.21.15.192:6815/3368512157","heartbeat_front_addr":"172.21.15.192:6813/3368512157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.091188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.032567+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:34.797468+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.228839+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.622400+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.473103+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.566092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.349395+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T04:34:31.815 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.815+0000 7fd9c5a00640 1 -- 172.21.15.145:0/1098082586 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fd9a00762d0 msgr2=0x7fd9a0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:31.815 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.815+0000 7fd9c5a00640 1 --2- 172.21.15.145:0/1098082586 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fd9a00762d0 0x7fd9a0078790 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac007a30 tx=0x7fd9ac001dd0 comp rx=0 tx=0).stop 2024-05-09T04:34:31.815 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.815+0000 7fd9c5a00640 1 -- 172.21.15.145:0/1098082586 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9a4005cb0 msgr2=0x7fd9c0152b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:31.815 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.815+0000 7fd9c5a00640 1 --2- 172.21.15.145:0/1098082586 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9a4005cb0 0x7fd9c0152b80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd9b002a790 tx=0x7fd9b0004730 comp rx=0 tx=0).stop 2024-05-09T04:34:31.815 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.815+0000 7fd9c5a00640 1 -- 172.21.15.145:0/1098082586 shutdown_connections 2024-05-09T04:34:31.815 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.815+0000 7fd9c5a00640 1 --2- 172.21.15.145:0/1098082586 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fd9a00762d0 0x7fd9a0078790 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:31.815 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.815+0000 7fd9c5a00640 1 --2- 172.21.15.145:0/1098082586 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd9c0139b00 0x7fd9c0139f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:31.815 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.815+0000 7fd9c5a00640 1 --2- 172.21.15.145:0/1098082586 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9c00b4fc0 0x7fd9c00b5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:31.815 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.815+0000 7fd9c5a00640 1 --2- 172.21.15.145:0/1098082586 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd9a4005cb0 0x7fd9c0152b80 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:31.815 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.815+0000 7fd9c5a00640 1 -- 172.21.15.145:0/1098082586 >> 172.21.15.145:0/1098082586 conn(0x7fd9c006c870 msgr2=0x7fd9c00b33a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:31.815 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.815+0000 7fd9c5a00640 1 -- 172.21.15.145:0/1098082586 shutdown_connections 2024-05-09T04:34:31.815 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:31.815+0000 7fd9c5a00640 1 -- 172.21.15.145:0/1098082586 wait complete. 2024-05-09T04:34:31.834 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-09T04:34:32.022 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.022+0000 7ff83d200640 1 Processor -- start 2024-05-09T04:34:32.023 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.023+0000 7ff83d200640 1 -- start start 2024-05-09T04:34:32.024 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.024+0000 7ff83d200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff838162c00 0x7ff8380d58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:32.024 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.024+0000 7ff83d200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff8380b5fe0 0x7ff8380d1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:32.024 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.024+0000 7ff83d200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8380d3780 con 0x7ff8380d1f50 2024-05-09T04:34:32.025 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.024+0000 7ff83d200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7ff8380d23a0 con 0x7ff838162c00 2024-05-09T04:34:32.025 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.024+0000 7ff83d200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff8380d5ec0 con 0x7ff8380b5fe0 2024-05-09T04:34:32.025 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.025+0000 7ff835e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff838162c00 0x7ff8380d58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:32.025 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.025+0000 7ff835e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff838162c00 0x7ff8380d58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:51050/0 (socket says 172.21.15.145:51050) 2024-05-09T04:34:32.025 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.025+0000 7ff835e00640 1 -- 172.21.15.145:0/3165240189 learned_addr learned my addr 172.21.15.145:0/3165240189 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:32.026 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.025+0000 7ff835400640 1 --2- 172.21.15.145:0/3165240189 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff8380b5fe0 0x7ff8380d1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:32.026 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.025+0000 7ff835e00640 1 -- 172.21.15.145:0/3165240189 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff8380b5fe0 msgr2=0x7ff8380d1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:32.026 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.025+0000 7ff835e00640 1 --2- 172.21.15.145:0/3165240189 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff8380b5fe0 0x7ff8380d1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:32.026 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.025+0000 7ff835e00640 1 -- 172.21.15.145:0/3165240189 >> v1:172.21.15.192:6789/0 conn(0x7ff8380d1f50 legacy=0x7ff8381630e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:32.026 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.026+0000 7ff835e00640 1 -- 172.21.15.145:0/3165240189 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8380d3a00 con 0x7ff838162c00 2024-05-09T04:34:32.026 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.027+0000 7ff835e00640 1 --2- 172.21.15.145:0/3165240189 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff838162c00 0x7ff8380d58e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff820009870 tx=0x7ff82002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80c00ccefe13f072 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:32.027 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.027+0000 7ff834a00640 1 -- 172.21.15.145:0/3165240189 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff820037070 con 0x7ff838162c00 2024-05-09T04:34:32.027 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.027+0000 7ff834a00640 1 -- 172.21.15.145:0/3165240189 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff820032440 con 0x7ff838162c00 2024-05-09T04:34:32.027 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.027+0000 7ff834a00640 1 -- 172.21.15.145:0/3165240189 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff820032810 con 0x7ff838162c00 2024-05-09T04:34:32.028 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.028+0000 7ff83d200640 1 -- 172.21.15.145:0/3165240189 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff838162c00 msgr2=0x7ff8380d58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:32.028 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.028+0000 7ff83d200640 1 --2- 172.21.15.145:0/3165240189 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff838162c00 0x7ff8380d58e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff820009870 tx=0x7ff82002a1e0 comp rx=0 tx=0).stop 2024-05-09T04:34:32.028 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.028+0000 7ff83d200640 1 -- 172.21.15.145:0/3165240189 shutdown_connections 2024-05-09T04:34:32.028 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.028+0000 7ff83d200640 1 --2- 172.21.15.145:0/3165240189 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff8380b5fe0 0x7ff8380d1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:32.028 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.028+0000 7ff83d200640 1 --2- 172.21.15.145:0/3165240189 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff838162c00 0x7ff8380d58e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:32.028 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.028+0000 7ff83d200640 1 -- 172.21.15.145:0/3165240189 >> 172.21.15.145:0/3165240189 conn(0x7ff83809e510 msgr2=0x7ff8380d51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:32.028 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.028+0000 7ff83d200640 1 -- 172.21.15.145:0/3165240189 shutdown_connections 2024-05-09T04:34:32.028 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.028+0000 7ff83d200640 1 -- 172.21.15.145:0/3165240189 wait complete. 2024-05-09T04:34:32.029 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.030+0000 7ff83d200640 1 Processor -- start 2024-05-09T04:34:32.029 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.030+0000 7ff83d200640 1 -- start start 2024-05-09T04:34:32.031 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.031+0000 7ff83d200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff8380b5fe0 0x7ff83815aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:32.031 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.031+0000 7ff83d200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff8380d1f50 0x7ff838161510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:32.031 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.031+0000 7ff83d200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff838162c00 0x7ff8380b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:32.031 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.031+0000 7ff83d200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff8380b4950 con 0x7ff8380d1f50 2024-05-09T04:34:32.031 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.031+0000 7ff83d200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff8380d3fc0 con 0x7ff838162c00 2024-05-09T04:34:32.031 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.031+0000 7ff83d200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff8380d1040 con 0x7ff8380b5fe0 2024-05-09T04:34:32.031 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.031+0000 7ff835400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff8380d1f50 0x7ff838161510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:32.031 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.031+0000 7ff835400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff8380d1f50 0x7ff838161510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:51066/0 (socket says 172.21.15.145:51066) 2024-05-09T04:34:32.031 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.031+0000 7ff835400640 1 -- 172.21.15.145:0/2792546796 learned_addr learned my addr 172.21.15.145:0/2792546796 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:32.033 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.032+0000 7ff835e00640 1 --2- 172.21.15.145:0/2792546796 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff8380b5fe0 0x7ff83815aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:32.033 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.032+0000 7ff835400640 1 -- 172.21.15.145:0/2792546796 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff8380b5fe0 msgr2=0x7ff83815aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:32.033 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.032+0000 7ff835400640 1 --2- 172.21.15.145:0/2792546796 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff8380b5fe0 0x7ff83815aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:32.033 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.032+0000 7ff835400640 1 -- 172.21.15.145:0/2792546796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff838162c00 msgr2=0x7ff8380b4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:32.033 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.032+0000 7ff836800640 1 --2- 172.21.15.145:0/2792546796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff838162c00 0x7ff8380b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:32.033 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.032+0000 7ff835400640 1 --2- 172.21.15.145:0/2792546796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff838162c00 0x7ff8380b4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:32.033 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.032+0000 7ff835400640 1 -- 172.21.15.145:0/2792546796 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff83815afb0 con 0x7ff8380d1f50 2024-05-09T04:34:32.033 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.032+0000 7ff836800640 1 --2- 172.21.15.145:0/2792546796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff838162c00 0x7ff8380b4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:34:32.033 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.032+0000 7ff835400640 1 --2- 172.21.15.145:0/2792546796 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff8380d1f50 0x7ff838161510 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff828007e60 tx=0x7ff82800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:32.034 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.032+0000 7ff82e000640 1 -- 172.21.15.145:0/2792546796 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff828014620 con 0x7ff8380d1f50 2024-05-09T04:34:32.034 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.032+0000 7ff82e000640 1 -- 172.21.15.145:0/2792546796 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff828019070 con 0x7ff8380d1f50 2024-05-09T04:34:32.034 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.032+0000 7ff82e000640 1 -- 172.21.15.145:0/2792546796 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff828014a20 con 0x7ff8380d1f50 2024-05-09T04:34:32.034 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.032+0000 7ff83d200640 1 -- 172.21.15.145:0/2792546796 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff83815b4f0 con 0x7ff8380d1f50 2024-05-09T04:34:32.034 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.033+0000 7ff83d200640 1 -- 172.21.15.145:0/2792546796 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8381f5fb0 con 0x7ff8380d1f50 2024-05-09T04:34:32.034 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.034+0000 7ff83d200640 1 -- 172.21.15.145:0/2792546796 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff83806a310 con 0x7ff8380d1f50 2024-05-09T04:34:32.034 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.035+0000 7ff82e000640 1 -- 172.21.15.145:0/2792546796 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff828014bc0 con 0x7ff8380d1f50 2024-05-09T04:34:32.035 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.036+0000 7ff82e000640 1 --2- 172.21.15.145:0/2792546796 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff8140762d0 0x7ff814078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:32.035 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.036+0000 7ff82e000640 1 -- 172.21.15.145:0/2792546796 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff828097f90 con 0x7ff8380d1f50 2024-05-09T04:34:32.036 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.036+0000 7ff835e00640 1 --2- 172.21.15.145:0/2792546796 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff8140762d0 0x7ff814078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:32.036 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.037+0000 7ff835e00640 1 --2- 172.21.15.145:0/2792546796 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff8140762d0 0x7ff814078790 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff820034000 tx=0x7ff82002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:32.040 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.040+0000 7ff82e000640 1 -- 172.21.15.145:0/2792546796 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff828061850 con 0x7ff8380d1f50 2024-05-09T04:34:32.293 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.293+0000 7ff83d200640 1 -- 172.21.15.145:0/2792546796 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ff8380cd2f0 con 0x7ff8380d1f50 2024-05-09T04:34:32.293 INFO:tasks.ceph.mon.a.smithi145.stderr:2024-05-09T04:34:32.293+0000 7f50dee00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T04:34:32.297 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.297+0000 7ff82e000640 1 -- 172.21.15.145:0/2792546796 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) ==== 161+0+0 (secure 0 0 0) 0x7ff828061150 con 0x7ff8380d1f50 2024-05-09T04:34:32.297 INFO:teuthology.orchestra.run.smithi145.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-09T04:34:32.303 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.300+0000 7ff83d200640 1 -- 172.21.15.145:0/2792546796 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff8140762d0 msgr2=0x7ff814078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:32.303 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.300+0000 7ff83d200640 1 --2- 172.21.15.145:0/2792546796 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff8140762d0 0x7ff814078790 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff820034000 tx=0x7ff82002e000 comp rx=0 tx=0).stop 2024-05-09T04:34:32.303 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.300+0000 7ff83d200640 1 -- 172.21.15.145:0/2792546796 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff8380d1f50 msgr2=0x7ff838161510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:32.303 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.300+0000 7ff83d200640 1 --2- 172.21.15.145:0/2792546796 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff8380d1f50 0x7ff838161510 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff828007e60 tx=0x7ff82800e4b0 comp rx=0 tx=0).stop 2024-05-09T04:34:32.303 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.300+0000 7ff83d200640 1 -- 172.21.15.145:0/2792546796 shutdown_connections 2024-05-09T04:34:32.303 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.300+0000 7ff83d200640 1 --2- 172.21.15.145:0/2792546796 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff8140762d0 0x7ff814078790 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:32.303 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.300+0000 7ff83d200640 1 --2- 172.21.15.145:0/2792546796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff838162c00 0x7ff8380b4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:32.303 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.300+0000 7ff83d200640 1 --2- 172.21.15.145:0/2792546796 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff8380d1f50 0x7ff838161510 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:32.303 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.300+0000 7ff83d200640 1 --2- 172.21.15.145:0/2792546796 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff8380b5fe0 0x7ff83815aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:32.303 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.300+0000 7ff83d200640 1 -- 172.21.15.145:0/2792546796 >> 172.21.15.145:0/2792546796 conn(0x7ff83809e510 msgr2=0x7ff83815b910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:32.303 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.300+0000 7ff83d200640 1 -- 172.21.15.145:0/2792546796 shutdown_connections 2024-05-09T04:34:32.303 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.300+0000 7ff83d200640 1 -- 172.21.15.145:0/2792546796 wait complete. 2024-05-09T04:34:32.323 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-09T04:34:32.327 INFO:tasks.ceph.mds.d.smithi192.stderr: -2516> 2024-05-09T04:33:56.957+0000 7f441a800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-09T04:34:32.394 INFO:tasks.ceph.mds.d.smithi192.stdout:starting mds.d at 2024-05-09T04:34:32.501 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.502+0000 7f267a400640 1 Processor -- start 2024-05-09T04:34:32.502 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.502+0000 7f267a400640 1 -- start start 2024-05-09T04:34:32.503 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.503+0000 7f267a400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f26740ce640 0x7f267406f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:32.503 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.503+0000 7f267a400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f267406fb10 0x7f26740b4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:32.503 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.503+0000 7f267a400640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f267415ffa0 con 0x7f2674144c00 2024-05-09T04:34:32.503 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.503+0000 7f267a400640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f267406ff40 con 0x7f26740ce640 2024-05-09T04:34:32.503 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.503+0000 7f267a400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f2674160280 con 0x7f267406fb10 2024-05-09T04:34:32.503 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.504+0000 7f266be00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f26740ce640 0x7f267406f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:32.504 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.504+0000 7f2673e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f267406fb10 0x7f26740b4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:32.504 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.504+0000 7f266be00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f26740ce640 0x7f267406f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.145:33452/0 (socket says 172.21.15.145:33452) 2024-05-09T04:34:32.504 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.504+0000 7f266be00640 1 -- 172.21.15.145:0/2628966403 learned_addr learned my addr 172.21.15.145:0/2628966403 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:32.505 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.505+0000 7f266be00640 1 -- 172.21.15.145:0/2628966403 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f267406fb10 msgr2=0x7f26740b4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:32.505 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.505+0000 7f266be00640 1 --2- 172.21.15.145:0/2628966403 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f267406fb10 0x7f26740b4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:32.505 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.505+0000 7f266be00640 1 -- 172.21.15.145:0/2628966403 >> v1:172.21.15.192:6789/0 conn(0x7f2674144c00 legacy=0x7f2674145440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:32.505 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.505+0000 7f266be00640 1 -- 172.21.15.145:0/2628966403 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2674160560 con 0x7f26740ce640 2024-05-09T04:34:32.506 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.506+0000 7f266be00640 1 --2- 172.21.15.145:0/2628966403 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26740ce640 0x7f267406f5d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f265c009870 tx=0x7f265c02a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eec672d612247aeb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:32.506 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.507+0000 7f2672a00640 1 -- 172.21.15.145:0/2628966403 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265c037070 con 0x7f26740ce640 2024-05-09T04:34:32.506 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.507+0000 7f2672a00640 1 -- 172.21.15.145:0/2628966403 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f265c032440 con 0x7f26740ce640 2024-05-09T04:34:32.507 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.507+0000 7f2672a00640 1 -- 172.21.15.145:0/2628966403 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265c032810 con 0x7f26740ce640 2024-05-09T04:34:32.507 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.507+0000 7f267a400640 1 -- 172.21.15.145:0/2628966403 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26740ce640 msgr2=0x7f267406f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:32.507 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.507+0000 7f267a400640 1 --2- 172.21.15.145:0/2628966403 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26740ce640 0x7f267406f5d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f265c009870 tx=0x7f265c02a250 comp rx=0 tx=0).stop 2024-05-09T04:34:32.508 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.507+0000 7f267a400640 1 -- 172.21.15.145:0/2628966403 shutdown_connections 2024-05-09T04:34:32.508 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.507+0000 7f267a400640 1 --2- 172.21.15.145:0/2628966403 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f267406fb10 0x7f26740b4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:32.508 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.507+0000 7f267a400640 1 --2- 172.21.15.145:0/2628966403 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f26740ce640 0x7f267406f5d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:32.508 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.507+0000 7f267a400640 1 -- 172.21.15.145:0/2628966403 >> 172.21.15.145:0/2628966403 conn(0x7f267406cab0 msgr2=0x7f26740d0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:32.508 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.507+0000 7f267a400640 1 -- 172.21.15.145:0/2628966403 shutdown_connections 2024-05-09T04:34:32.508 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.507+0000 7f267a400640 1 -- 172.21.15.145:0/2628966403 wait complete. 2024-05-09T04:34:32.508 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.509+0000 7f267a400640 1 Processor -- start 2024-05-09T04:34:32.509 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.509+0000 7f267a400640 1 -- start start 2024-05-09T04:34:32.511 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.510+0000 7f267a400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f267406fb10 0x7f26741f9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:32.511 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.510+0000 7f267a400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f26740ce640 0x7f267413ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:32.511 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.510+0000 7f267a400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2674144c00 0x7f26740cfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:32.511 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.510+0000 7f267a400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f267415fc30 con 0x7f267406fb10 2024-05-09T04:34:32.511 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.510+0000 7f267a400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2674160b20 con 0x7f2674144c00 2024-05-09T04:34:32.511 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.510+0000 7f267a400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f2674145b50 con 0x7f26740ce640 2024-05-09T04:34:32.511 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.510+0000 7f2673400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f267406fb10 0x7f26741f9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:32.511 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.510+0000 7f2673400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f267406fb10 0x7f26741f9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:51074/0 (socket says 172.21.15.145:51074) 2024-05-09T04:34:32.511 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.510+0000 7f2673400640 1 -- 172.21.15.145:0/3531864868 learned_addr learned my addr 172.21.15.145:0/3531864868 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:32.511 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.511+0000 7f2673e00640 1 --2- 172.21.15.145:0/3531864868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2674144c00 0x7f26740cfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:32.511 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.511+0000 7f266be00640 1 --2- 172.21.15.145:0/3531864868 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f26740ce640 0x7f267413ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:32.511 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.511+0000 7f2673400640 1 -- 172.21.15.145:0/3531864868 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f26740ce640 msgr2=0x7f267413ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:32.511 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.511+0000 7f2673400640 1 --2- 172.21.15.145:0/3531864868 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f26740ce640 0x7f267413ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:32.511 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.511+0000 7f2673400640 1 -- 172.21.15.145:0/3531864868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2674144c00 msgr2=0x7f26740cfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:32.511 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.511+0000 7f2673400640 1 --2- 172.21.15.145:0/3531864868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2674144c00 0x7f26740cfdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:32.512 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.511+0000 7f2673400640 1 -- 172.21.15.145:0/3531864868 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26740ced10 con 0x7f267406fb10 2024-05-09T04:34:32.512 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.511+0000 7f266be00640 1 --2- 172.21.15.145:0/3531864868 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f26740ce640 0x7f267413ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:34:32.512 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.511+0000 7f2673400640 1 --2- 172.21.15.145:0/3531864868 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f267406fb10 0x7f26741f9c30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f266400c3d0 tx=0x7f266400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:32.512 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.511+0000 7f266b400640 1 -- 172.21.15.145:0/3531864868 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2664017070 con 0x7f267406fb10 2024-05-09T04:34:32.512 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.511+0000 7f266b400640 1 -- 172.21.15.145:0/3531864868 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26640052d0 con 0x7f267406fb10 2024-05-09T04:34:32.512 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.511+0000 7f266b400640 1 -- 172.21.15.145:0/3531864868 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2664003830 con 0x7f267406fb10 2024-05-09T04:34:32.513 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.511+0000 7f267a400640 1 -- 172.21.15.145:0/3531864868 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26740ceff0 con 0x7f267406fb10 2024-05-09T04:34:32.513 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.512+0000 7f267a400640 1 -- 172.21.15.145:0/3531864868 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26740cf5b0 con 0x7f267406fb10 2024-05-09T04:34:32.513 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.513+0000 7f267a400640 1 -- 172.21.15.145:0/3531864868 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f267406a310 con 0x7f267406fb10 2024-05-09T04:34:32.513 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.514+0000 7f266b400640 1 -- 172.21.15.145:0/3531864868 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f266401a070 con 0x7f267406fb10 2024-05-09T04:34:32.514 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.515+0000 7f266b400640 1 --2- 172.21.15.145:0/3531864868 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f26500762d0 0x7f2650078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:32.515 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.515+0000 7f266b400640 1 -- 172.21.15.145:0/3531864868 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5911+0+0 (secure 0 0 0) 0x7f2664096af0 con 0x7f267406fb10 2024-05-09T04:34:32.515 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.515+0000 7f266be00640 1 --2- 172.21.15.145:0/3531864868 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f26500762d0 0x7f2650078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:32.515 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.516+0000 7f266be00640 1 --2- 172.21.15.145:0/3531864868 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f26500762d0 0x7f2650078790 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f265c034000 tx=0x7f265c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:32.518 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.519+0000 7f266b400640 1 -- 172.21.15.145:0/3531864868 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f26640602e0 con 0x7f267406fb10 2024-05-09T04:34:32.744 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:32.744+0000 7f267a400640 1 -- 172.21.15.145:0/3531864868 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f26740b5b10 con 0x7f267406fb10 2024-05-09T04:34:33.300 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.300+0000 7f266b400640 1 -- 172.21.15.145:0/3531864868 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) ==== 104+0+0 (secure 0 0 0) 0x7f266405fbe0 con 0x7f267406fb10 2024-05-09T04:34:33.305 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.305+0000 7f267a400640 1 -- 172.21.15.145:0/3531864868 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f26500762d0 msgr2=0x7f2650078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:33.305 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.305+0000 7f267a400640 1 --2- 172.21.15.145:0/3531864868 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f26500762d0 0x7f2650078790 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f265c034000 tx=0x7f265c02e000 comp rx=0 tx=0).stop 2024-05-09T04:34:33.305 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.305+0000 7f267a400640 1 -- 172.21.15.145:0/3531864868 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f267406fb10 msgr2=0x7f26741f9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:33.305 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.305+0000 7f267a400640 1 --2- 172.21.15.145:0/3531864868 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f267406fb10 0x7f26741f9c30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f266400c3d0 tx=0x7f266400c890 comp rx=0 tx=0).stop 2024-05-09T04:34:33.305 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.305+0000 7f267a400640 1 -- 172.21.15.145:0/3531864868 shutdown_connections 2024-05-09T04:34:33.305 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.305+0000 7f267a400640 1 --2- 172.21.15.145:0/3531864868 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f26500762d0 0x7f2650078790 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:33.306 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.305+0000 7f267a400640 1 --2- 172.21.15.145:0/3531864868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2674144c00 0x7f26740cfdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:33.306 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.305+0000 7f267a400640 1 --2- 172.21.15.145:0/3531864868 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f26740ce640 0x7f267413ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:33.306 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.305+0000 7f267a400640 1 --2- 172.21.15.145:0/3531864868 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f267406fb10 0x7f26741f9c30 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:33.306 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.305+0000 7f267a400640 1 -- 172.21.15.145:0/3531864868 >> 172.21.15.145:0/3531864868 conn(0x7f267406cab0 msgr2=0x7f267415ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:33.306 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.305+0000 7f267a400640 1 -- 172.21.15.145:0/3531864868 shutdown_connections 2024-05-09T04:34:33.306 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.306+0000 7f267a400640 1 -- 172.21.15.145:0/3531864868 wait complete. 2024-05-09T04:34:33.334 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-09T04:34:33.520 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.520+0000 7f9ac1c00640 1 Processor -- start 2024-05-09T04:34:33.520 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.521+0000 7f9ac1c00640 1 -- start start 2024-05-09T04:34:33.522 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.522+0000 7f9ac1c00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f9abc15f2f0 0x7f9abc0d08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:33.522 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.522+0000 7f9ac1c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9abc0d2170 0x7f9abc0ba290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:33.522 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.522+0000 7f9ac1c00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9abc0d30d0 con 0x7f9abc0d0e10 2024-05-09T04:34:33.522 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.522+0000 7f9ac1c00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f9abc0d33b0 con 0x7f9abc15f2f0 2024-05-09T04:34:33.523 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.522+0000 7f9ac1c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9abc0d3690 con 0x7f9abc0d2170 2024-05-09T04:34:33.523 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.522+0000 7f9abaa00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f9abc15f2f0 0x7f9abc0d08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:33.523 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.522+0000 7f9abaa00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f9abc15f2f0 0x7f9abc0d08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:51086/0 (socket says 172.21.15.145:51086) 2024-05-09T04:34:33.523 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.522+0000 7f9abaa00640 1 -- 172.21.15.145:0/4229470666 learned_addr learned my addr 172.21.15.145:0/4229470666 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:33.523 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.523+0000 7f9abaa00640 1 -- 172.21.15.145:0/4229470666 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9abc0d2170 msgr2=0x7f9abc0ba290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:33.523 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.523+0000 7f9abaa00640 1 --2- 172.21.15.145:0/4229470666 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9abc0d2170 0x7f9abc0ba290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:33.523 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.523+0000 7f9abaa00640 1 -- 172.21.15.145:0/4229470666 >> v1:172.21.15.192:6789/0 conn(0x7f9abc0d0e10 legacy=0x7f9abc0d1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:33.523 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.523+0000 7f9abaa00640 1 -- 172.21.15.145:0/4229470666 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9abc0d3970 con 0x7f9abc15f2f0 2024-05-09T04:34:33.524 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.524+0000 7f9abaa00640 1 --2- 172.21.15.145:0/4229470666 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9abc15f2f0 0x7f9abc0d08d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9aac009e50 tx=0x7f9aac02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68c4c18877ea7eab server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:33.524 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.524+0000 7f9ab9600640 1 -- 172.21.15.145:0/4229470666 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aac036070 con 0x7f9abc15f2f0 2024-05-09T04:34:33.524 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.524+0000 7f9ab9600640 1 -- 172.21.15.145:0/4229470666 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9aac031600 con 0x7f9abc15f2f0 2024-05-09T04:34:33.525 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.524+0000 7f9ab9600640 1 -- 172.21.15.145:0/4229470666 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aac0319d0 con 0x7f9abc15f2f0 2024-05-09T04:34:33.525 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.524+0000 7f9ac1c00640 1 -- 172.21.15.145:0/4229470666 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9abc15f2f0 msgr2=0x7f9abc0d08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:33.525 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.524+0000 7f9ac1c00640 1 --2- 172.21.15.145:0/4229470666 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9abc15f2f0 0x7f9abc0d08d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9aac009e50 tx=0x7f9aac02a680 comp rx=0 tx=0).stop 2024-05-09T04:34:33.525 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.525+0000 7f9ac1c00640 1 -- 172.21.15.145:0/4229470666 shutdown_connections 2024-05-09T04:34:33.525 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.525+0000 7f9ac1c00640 1 --2- 172.21.15.145:0/4229470666 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9abc0d2170 0x7f9abc0ba290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:33.525 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.525+0000 7f9ac1c00640 1 --2- 172.21.15.145:0/4229470666 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9abc15f2f0 0x7f9abc0d08d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:33.525 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.525+0000 7f9ac1c00640 1 -- 172.21.15.145:0/4229470666 >> 172.21.15.145:0/4229470666 conn(0x7f9abc09e4d0 msgr2=0x7f9abc0d5190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:33.525 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.525+0000 7f9ac1c00640 1 -- 172.21.15.145:0/4229470666 shutdown_connections 2024-05-09T04:34:33.525 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.525+0000 7f9ac1c00640 1 -- 172.21.15.145:0/4229470666 wait complete. 2024-05-09T04:34:33.526 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.527+0000 7f9ac1c00640 1 Processor -- start 2024-05-09T04:34:33.526 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.527+0000 7f9ac1c00640 1 -- start start 2024-05-09T04:34:33.529 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.528+0000 7f9ac1c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9abc0d0e10 0x7f9abc15b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:33.529 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.528+0000 7f9ac1c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9abc0d2170 0x7f9abc158750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:33.529 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.528+0000 7f9ac1c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9abc15f2f0 0x7f9abc0b8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:33.529 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.528+0000 7f9ac1c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9abc0d2d60 con 0x7f9abc0d2170 2024-05-09T04:34:33.529 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.528+0000 7f9ac1c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9abc0d4150 con 0x7f9abc15f2f0 2024-05-09T04:34:33.529 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.528+0000 7f9ac1c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9abc15f6d0 con 0x7f9abc0d0e10 2024-05-09T04:34:33.529 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.528+0000 7f9aba000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9abc0d2170 0x7f9abc158750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:33.530 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.528+0000 7f9aba000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9abc0d2170 0x7f9abc158750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:51100/0 (socket says 172.21.15.145:51100) 2024-05-09T04:34:33.530 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.528+0000 7f9aba000640 1 -- 172.21.15.145:0/2068653679 learned_addr learned my addr 172.21.15.145:0/2068653679 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:33.530 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.529+0000 7f9abaa00640 1 --2- 172.21.15.145:0/2068653679 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9abc0d0e10 0x7f9abc15b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:33.530 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.529+0000 7f9abb400640 1 --2- 172.21.15.145:0/2068653679 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9abc15f2f0 0x7f9abc0b8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:33.530 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.529+0000 7f9aba000640 1 -- 172.21.15.145:0/2068653679 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9abc0d0e10 msgr2=0x7f9abc15b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:33.530 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.529+0000 7f9aba000640 1 --2- 172.21.15.145:0/2068653679 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9abc0d0e10 0x7f9abc15b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:33.530 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.529+0000 7f9aba000640 1 -- 172.21.15.145:0/2068653679 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9abc15f2f0 msgr2=0x7f9abc0b8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:33.530 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.529+0000 7f9aba000640 1 --2- 172.21.15.145:0/2068653679 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9abc15f2f0 0x7f9abc0b8650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:33.530 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.529+0000 7f9aba000640 1 -- 172.21.15.145:0/2068653679 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9abc0b98b0 con 0x7f9abc0d2170 2024-05-09T04:34:33.530 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.529+0000 7f9aba000640 1 --2- 172.21.15.145:0/2068653679 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9abc0d2170 0x7f9abc158750 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9aa400c440 tx=0x7f9aa400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:33.530 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.529+0000 7f9ab2a00640 1 -- 172.21.15.145:0/2068653679 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa4017070 con 0x7f9abc0d2170 2024-05-09T04:34:33.530 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.529+0000 7f9ab2a00640 1 -- 172.21.15.145:0/2068653679 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9aa40035f0 con 0x7f9abc0d2170 2024-05-09T04:34:33.530 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.529+0000 7f9ab2a00640 1 -- 172.21.15.145:0/2068653679 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa40039e0 con 0x7f9abc0d2170 2024-05-09T04:34:33.530 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.529+0000 7f9ac1c00640 1 -- 172.21.15.145:0/2068653679 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9abc15bcc0 con 0x7f9abc0d2170 2024-05-09T04:34:33.530 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.530+0000 7f9ac1c00640 1 -- 172.21.15.145:0/2068653679 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9abc0b7e50 con 0x7f9abc0d2170 2024-05-09T04:34:33.531 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.531+0000 7f9ac1c00640 1 -- 172.21.15.145:0/2068653679 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9abc06a310 con 0x7f9abc0d2170 2024-05-09T04:34:33.531 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.532+0000 7f9ab2a00640 1 -- 172.21.15.145:0/2068653679 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9aa401a070 con 0x7f9abc0d2170 2024-05-09T04:34:33.532 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.533+0000 7f9ab2a00640 1 --2- 172.21.15.145:0/2068653679 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f9a980762d0 0x7f9a98078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:33.532 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.533+0000 7f9ab2a00640 1 -- 172.21.15.145:0/2068653679 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5911+0+0 (secure 0 0 0) 0x7f9aa4096b30 con 0x7f9abc0d2170 2024-05-09T04:34:33.532 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.533+0000 7f9abaa00640 1 --2- 172.21.15.145:0/2068653679 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f9a980762d0 0x7f9a98078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:33.533 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.534+0000 7f9abaa00640 1 --2- 172.21.15.145:0/2068653679 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f9a980762d0 0x7f9a98078790 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9aac033000 tx=0x7f9aac009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:33.538 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.538+0000 7f9ab2a00640 1 -- 172.21.15.145:0/2068653679 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f9aa4060320 con 0x7f9abc0d2170 2024-05-09T04:34:33.744 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:33.744+0000 7f9ac1c00640 1 -- 172.21.15.145:0/2068653679 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f9abc0d2810 con 0x7f9abc0d2170 2024-05-09T04:34:33.763 DEBUG:teuthology.orchestra.run.smithi145:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T04:34:33.768 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T04:34:34.427 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.427+0000 7f9ab2a00640 1 -- 172.21.15.145:0/2068653679 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v23) ==== 181+0+0 (secure 0 0 0) 0x7f9aa40010c0 con 0x7f9abc0d2170 2024-05-09T04:34:34.427 INFO:teuthology.orchestra.run.smithi145.stderr:pool 'cephfs_metadata' removed 2024-05-09T04:34:34.439 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.435+0000 7f9ac1c00640 1 -- 172.21.15.145:0/2068653679 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f9a980762d0 msgr2=0x7f9a98078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:34.439 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.435+0000 7f9ac1c00640 1 --2- 172.21.15.145:0/2068653679 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f9a980762d0 0x7f9a98078790 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9aac033000 tx=0x7f9aac009000 comp rx=0 tx=0).stop 2024-05-09T04:34:34.439 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.435+0000 7f9ac1c00640 1 -- 172.21.15.145:0/2068653679 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9abc0d2170 msgr2=0x7f9abc158750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:34.439 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.435+0000 7f9ac1c00640 1 --2- 172.21.15.145:0/2068653679 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9abc0d2170 0x7f9abc158750 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9aa400c440 tx=0x7f9aa400c900 comp rx=0 tx=0).stop 2024-05-09T04:34:34.439 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.436+0000 7f9ac1c00640 1 -- 172.21.15.145:0/2068653679 shutdown_connections 2024-05-09T04:34:34.440 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.436+0000 7f9ac1c00640 1 --2- 172.21.15.145:0/2068653679 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f9a980762d0 0x7f9a98078790 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:34.440 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.436+0000 7f9ac1c00640 1 --2- 172.21.15.145:0/2068653679 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9abc15f2f0 0x7f9abc0b8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:34.440 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.436+0000 7f9ac1c00640 1 --2- 172.21.15.145:0/2068653679 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9abc0d2170 0x7f9abc158750 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:34.440 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.436+0000 7f9ac1c00640 1 --2- 172.21.15.145:0/2068653679 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9abc0d0e10 0x7f9abc15b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:34.440 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.436+0000 7f9ac1c00640 1 -- 172.21.15.145:0/2068653679 >> 172.21.15.145:0/2068653679 conn(0x7f9abc09e4d0 msgr2=0x7f9abc0ce4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:34.440 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.439+0000 7f9ac1c00640 1 -- 172.21.15.145:0/2068653679 shutdown_connections 2024-05-09T04:34:34.440 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.440+0000 7f9ac1c00640 1 -- 172.21.15.145:0/2068653679 wait complete. 2024-05-09T04:34:34.462 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-09T04:34:34.647 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.647+0000 7fc1da400640 1 Processor -- start 2024-05-09T04:34:34.648 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.648+0000 7fc1da400640 1 -- start start 2024-05-09T04:34:34.650 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.649+0000 7fc1da400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc1d4144c50 0x7fc1d40b3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:34.650 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.649+0000 7fc1da400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fc1d406e510 0x7fc1d406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:34.650 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.649+0000 7fc1da400640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1d41455f0 con 0x7fc1d406fb20 2024-05-09T04:34:34.650 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.649+0000 7fc1da400640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fc1d406efd0 con 0x7fc1d406e510 2024-05-09T04:34:34.650 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.649+0000 7fc1da400640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fc1d406f2b0 con 0x7fc1d4144c50 2024-05-09T04:34:34.650 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.649+0000 7fc1d3e00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7fc1d406fb20 0x7fc1d406ff30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:59086/0 (socket says 172.21.15.145:59086) 2024-05-09T04:34:34.650 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.649+0000 7fc1d3e00640 1 -- 172.21.15.145:0/2080866673 learned_addr learned my addr 172.21.15.145:0/2080866673 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:34:34.650 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.650+0000 7fc1cbe00640 1 --2- 172.21.15.145:0/2080866673 >> v2:172.21.15.145:3300/0 conn(0x7fc1d406e510 0x7fc1d406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:34.650 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.650+0000 7fc1d3400640 1 --2- 172.21.15.145:0/2080866673 >> v2:172.21.15.192:3300/0 conn(0x7fc1d4144c50 0x7fc1d40b3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:34.651 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.650+0000 7fc1d2a00640 1 -- 172.21.15.145:0/2080866673 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1346794491 0 0) 0x7fc1d41455f0 con 0x7fc1d406fb20 2024-05-09T04:34:34.651 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.650+0000 7fc1d2a00640 1 -- 172.21.15.145:0/2080866673 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc1b8003520 con 0x7fc1d406fb20 2024-05-09T04:34:34.651 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.650+0000 7fc1cbe00640 1 -- 172.21.15.145:0/2080866673 >> v2:172.21.15.192:3300/0 conn(0x7fc1d4144c50 msgr2=0x7fc1d40b3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:34.651 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.650+0000 7fc1cbe00640 1 --2- 172.21.15.145:0/2080866673 >> v2:172.21.15.192:3300/0 conn(0x7fc1d4144c50 0x7fc1d40b3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:34.651 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.650+0000 7fc1cbe00640 1 -- 172.21.15.145:0/2080866673 >> v1:172.21.15.145:6789/0 conn(0x7fc1d406fb20 legacy=0x7fc1d406ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:34.651 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.650+0000 7fc1cbe00640 1 -- 172.21.15.145:0/2080866673 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1d406f590 con 0x7fc1d406e510 2024-05-09T04:34:34.651 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.652+0000 7fc1cbe00640 1 --2- 172.21.15.145:0/2080866673 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1d406e510 0x7fc1d406e9a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc1d40d0db0 tx=0x7fc1bc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fa9dac7920df7f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:34.652 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.652+0000 7fc1d2a00640 1 -- 172.21.15.145:0/2080866673 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc037070 con 0x7fc1d406e510 2024-05-09T04:34:34.652 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.652+0000 7fc1d2a00640 1 -- 172.21.15.145:0/2080866673 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1bc032440 con 0x7fc1d406e510 2024-05-09T04:34:34.652 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.652+0000 7fc1d2a00640 1 -- 172.21.15.145:0/2080866673 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc0327f0 con 0x7fc1d406e510 2024-05-09T04:34:34.652 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.653+0000 7fc1da400640 1 -- 172.21.15.145:0/2080866673 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1d406e510 msgr2=0x7fc1d406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:34.653 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.653+0000 7fc1da400640 1 --2- 172.21.15.145:0/2080866673 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1d406e510 0x7fc1d406e9a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc1d40d0db0 tx=0x7fc1bc02a1a0 comp rx=0 tx=0).stop 2024-05-09T04:34:34.653 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.653+0000 7fc1da400640 1 -- 172.21.15.145:0/2080866673 shutdown_connections 2024-05-09T04:34:34.653 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.653+0000 7fc1da400640 1 --2- 172.21.15.145:0/2080866673 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1d406e510 0x7fc1d406e9a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:34.653 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.653+0000 7fc1da400640 1 --2- 172.21.15.145:0/2080866673 >> v2:172.21.15.192:3300/0 conn(0x7fc1d4144c50 0x7fc1d40b3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:34.653 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.653+0000 7fc1da400640 1 -- 172.21.15.145:0/2080866673 >> 172.21.15.145:0/2080866673 conn(0x7fc1d406c8d0 msgr2=0x7fc1d406cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:34.653 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.653+0000 7fc1da400640 1 -- 172.21.15.145:0/2080866673 shutdown_connections 2024-05-09T04:34:34.653 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.653+0000 7fc1da400640 1 -- 172.21.15.145:0/2080866673 wait complete. 2024-05-09T04:34:34.655 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.655+0000 7fc1da400640 1 Processor -- start 2024-05-09T04:34:34.655 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.655+0000 7fc1da400640 1 -- start start 2024-05-09T04:34:34.657 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.656+0000 7fc1da400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1d406e510 0x7fc1d40b0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:34.657 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.656+0000 7fc1da400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1d406fb20 0x7fc1d415b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:34.658 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.656+0000 7fc1da400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc1d4144c50 0x7fc1d415fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:34.658 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.656+0000 7fc1da400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d40b4440 con 0x7fc1d406e510 2024-05-09T04:34:34.658 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.656+0000 7fc1da400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d4073460 con 0x7fc1d406fb20 2024-05-09T04:34:34.658 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.656+0000 7fc1da400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc1d4145030 con 0x7fc1d4144c50 2024-05-09T04:34:34.658 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.657+0000 7fc1d3400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1d406e510 0x7fc1d40b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:34.658 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.657+0000 7fc1d3400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1d406e510 0x7fc1d40b0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:51120/0 (socket says 172.21.15.145:51120) 2024-05-09T04:34:34.658 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.657+0000 7fc1d3400640 1 -- 172.21.15.145:0/732331938 learned_addr learned my addr 172.21.15.145:0/732331938 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:34.658 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.657+0000 7fc1d3e00640 1 --2- 172.21.15.145:0/732331938 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc1d4144c50 0x7fc1d415fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:34.658 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.657+0000 7fc1cbe00640 1 --2- 172.21.15.145:0/732331938 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1d406fb20 0x7fc1d415b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:34.659 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.657+0000 7fc1d3400640 1 -- 172.21.15.145:0/732331938 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc1d4144c50 msgr2=0x7fc1d415fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:34.659 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.657+0000 7fc1d3400640 1 --2- 172.21.15.145:0/732331938 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc1d4144c50 0x7fc1d415fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:34.659 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.657+0000 7fc1d3400640 1 -- 172.21.15.145:0/732331938 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1d406fb20 msgr2=0x7fc1d415b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:34.659 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.657+0000 7fc1d3400640 1 --2- 172.21.15.145:0/732331938 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1d406fb20 0x7fc1d415b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:34.659 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.657+0000 7fc1d3400640 1 -- 172.21.15.145:0/732331938 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1d40b0df0 con 0x7fc1d406e510 2024-05-09T04:34:34.659 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.657+0000 7fc1cbe00640 1 --2- 172.21.15.145:0/732331938 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1d406fb20 0x7fc1d415b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:34:34.659 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.657+0000 7fc1d3400640 1 --2- 172.21.15.145:0/732331938 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1d406e510 0x7fc1d40b0750 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc1c400d620 tx=0x7fc1c400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:34.659 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.658+0000 7fc1cb400640 1 -- 172.21.15.145:0/732331938 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c400b620 con 0x7fc1d406e510 2024-05-09T04:34:34.659 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.658+0000 7fc1cb400640 1 -- 172.21.15.145:0/732331938 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1c4019070 con 0x7fc1d406e510 2024-05-09T04:34:34.659 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.658+0000 7fc1cb400640 1 -- 172.21.15.145:0/732331938 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c400ba00 con 0x7fc1d406e510 2024-05-09T04:34:34.659 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.658+0000 7fc1da400640 1 -- 172.21.15.145:0/732331938 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1d40b10d0 con 0x7fc1d406e510 2024-05-09T04:34:34.659 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.658+0000 7fc1da400640 1 -- 172.21.15.145:0/732331938 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1d40b1690 con 0x7fc1d406e510 2024-05-09T04:34:34.660 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.659+0000 7fc1da400640 1 -- 172.21.15.145:0/732331938 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1d406a310 con 0x7fc1d406e510 2024-05-09T04:34:34.660 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.660+0000 7fc1cb400640 1 -- 172.21.15.145:0/732331938 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc1c400bba0 con 0x7fc1d406e510 2024-05-09T04:34:34.661 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.661+0000 7fc1cb400640 1 --2- 172.21.15.145:0/732331938 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fc1b00762d0 0x7fc1b0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:34.661 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.661+0000 7fc1cb400640 1 -- 172.21.15.145:0/732331938 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5473+0+0 (secure 0 0 0) 0x7fc1c4097880 con 0x7fc1d406e510 2024-05-09T04:34:34.661 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.661+0000 7fc1cbe00640 1 --2- 172.21.15.145:0/732331938 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fc1b00762d0 0x7fc1b0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:34.662 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.662+0000 7fc1cbe00640 1 --2- 172.21.15.145:0/732331938 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fc1b00762d0 0x7fc1b0078790 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc034000 tx=0x7fc1bc02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:34.667 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.667+0000 7fc1cb400640 1 -- 172.21.15.145:0/732331938 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc1c4061350 con 0x7fc1d406e510 2024-05-09T04:34:34.875 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:34.875+0000 7fc1da400640 1 -- 172.21.15.145:0/732331938 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc1d40ae9c0 con 0x7fc1d406e510 2024-05-09T04:34:35.431 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.431+0000 7fc1cb400640 1 -- 172.21.15.145:0/732331938 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v24) ==== 169+0+0 (secure 0 0 0) 0x7fc1c4060c50 con 0x7fc1d406e510 2024-05-09T04:34:35.431 INFO:teuthology.orchestra.run.smithi145.stderr:pool 'cephfs_data' removed 2024-05-09T04:34:35.434 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.434+0000 7fc1da400640 1 -- 172.21.15.145:0/732331938 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fc1b00762d0 msgr2=0x7fc1b0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:35.434 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.434+0000 7fc1da400640 1 --2- 172.21.15.145:0/732331938 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fc1b00762d0 0x7fc1b0078790 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc034000 tx=0x7fc1bc02e000 comp rx=0 tx=0).stop 2024-05-09T04:34:35.434 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.434+0000 7fc1da400640 1 -- 172.21.15.145:0/732331938 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1d406e510 msgr2=0x7fc1d40b0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:35.434 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.434+0000 7fc1da400640 1 --2- 172.21.15.145:0/732331938 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1d406e510 0x7fc1d40b0750 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc1c400d620 tx=0x7fc1c400dae0 comp rx=0 tx=0).stop 2024-05-09T04:34:35.435 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.435+0000 7fc1da400640 1 -- 172.21.15.145:0/732331938 shutdown_connections 2024-05-09T04:34:35.435 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.435+0000 7fc1da400640 1 --2- 172.21.15.145:0/732331938 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fc1b00762d0 0x7fc1b0078790 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:35.435 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.435+0000 7fc1da400640 1 --2- 172.21.15.145:0/732331938 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc1d4144c50 0x7fc1d415fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:35.435 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.435+0000 7fc1da400640 1 --2- 172.21.15.145:0/732331938 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc1d406fb20 0x7fc1d415b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:35.435 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.435+0000 7fc1da400640 1 --2- 172.21.15.145:0/732331938 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc1d406e510 0x7fc1d40b0750 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:35.435 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.435+0000 7fc1da400640 1 -- 172.21.15.145:0/732331938 >> 172.21.15.145:0/732331938 conn(0x7fc1d406c8d0 msgr2=0x7fc1d413f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:35.435 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.435+0000 7fc1da400640 1 -- 172.21.15.145:0/732331938 shutdown_connections 2024-05-09T04:34:35.436 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.435+0000 7fc1da400640 1 -- 172.21.15.145:0/732331938 wait complete. 2024-05-09T04:34:35.463 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-09T04:34:35.463 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-09T04:34:35.464 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-09T04:34:35.464 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T04:34:35.464 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-09T04:34:35.464 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-09T04:34:35.464 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-09T04:34:35.464 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T04:34:35.464 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-09T04:34:35.464 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-09T04:34:35.465 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-09T04:34:35.465 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T04:34:35.465 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-09T04:34:35.465 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-09T04:34:35.465 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-09T04:34:35.465 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T04:34:35.485 INFO:tasks.ceph.mds.a:Stopped 2024-05-09T04:34:35.485 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-09T04:34:35.486 INFO:tasks.ceph.mds.c:Stopped 2024-05-09T04:34:35.486 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-09T04:34:35.487 INFO:tasks.ceph.mds.d:Stopped 2024-05-09T04:34:35.487 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-09T04:34:35.488 INFO:tasks.ceph.mds.b:Stopped 2024-05-09T04:34:35.488 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-09T04:34:35.533 INFO:tasks.ceph.mds.a:Started 2024-05-09T04:34:35.533 DEBUG:teuthology.parallel:result is None 2024-05-09T04:34:35.534 INFO:tasks.ceph.mds.d:Started 2024-05-09T04:34:35.534 DEBUG:teuthology.parallel:result is None 2024-05-09T04:34:35.539 INFO:tasks.ceph.mds.c:Started 2024-05-09T04:34:35.539 DEBUG:teuthology.parallel:result is None 2024-05-09T04:34:35.540 INFO:tasks.ceph.mds.b:Started 2024-05-09T04:34:35.540 DEBUG:teuthology.parallel:result is None 2024-05-09T04:34:35.540 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-09T04:34:35.673 INFO:tasks.ceph.mds.b.smithi192.stdout:starting mds.b at 2024-05-09T04:34:35.674 INFO:tasks.ceph.mds.d.smithi192.stdout:starting mds.d at 2024-05-09T04:34:35.686 INFO:tasks.ceph.mds.c.smithi145.stdout:starting mds.c at 2024-05-09T04:34:35.687 INFO:tasks.ceph.mds.a.smithi145.stdout:starting mds.a at 2024-05-09T04:34:35.725 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.725+0000 7f5d9d600640 1 Processor -- start 2024-05-09T04:34:35.725 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.725+0000 7f5d9d600640 1 -- start start 2024-05-09T04:34:35.726 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.727+0000 7f5d9d600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f5d98071380 0x7f5d980b2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:35.727 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.727+0000 7f5d9d600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d980b2210 con 0x7f5d9815f650 2024-05-09T04:34:35.727 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.727+0000 7f5d9d600640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d980705c0 con 0x7f5d9809ba10 2024-05-09T04:34:35.727 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.727+0000 7f5d9d600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f5d980708a0 con 0x7f5d98071380 2024-05-09T04:34:35.727 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.727+0000 7f5d8fe00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f5d9815f650 0x7f5d980d10f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:59098/0 (socket says 172.21.15.145:59098) 2024-05-09T04:34:35.727 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.727+0000 7f5d8fe00640 1 -- 172.21.15.145:0/4157845816 learned_addr learned my addr 172.21.15.145:0/4157845816 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:34:35.727 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.727+0000 7f5d97400640 1 --2- 172.21.15.145:0/4157845816 >> v2:172.21.15.192:3300/0 conn(0x7f5d98071380 0x7f5d980b2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:35.728 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.727+0000 7f5d96000640 1 -- 172.21.15.145:0/4157845816 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1147221142 0 0) 0x7f5d980705c0 con 0x7f5d9809ba10 2024-05-09T04:34:35.728 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.727+0000 7f5d96000640 1 -- 172.21.15.145:0/4157845816 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5d7c003520 con 0x7f5d9809ba10 2024-05-09T04:34:35.728 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.727+0000 7f5d96000640 1 -- 172.21.15.145:0/4157845816 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2553942904 0 0) 0x7f5d980b2210 con 0x7f5d9815f650 2024-05-09T04:34:35.728 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.727+0000 7f5d96000640 1 -- 172.21.15.145:0/4157845816 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5d980705c0 con 0x7f5d9815f650 2024-05-09T04:34:35.728 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.728+0000 7f5d97400640 1 -- 172.21.15.145:0/4157845816 >> v1:172.21.15.192:6789/0 conn(0x7f5d9809ba10 legacy=0x7f5d9809bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:35.728 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.728+0000 7f5d97400640 1 -- 172.21.15.145:0/4157845816 >> v1:172.21.15.145:6789/0 conn(0x7f5d9815f650 legacy=0x7f5d980d10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:35.728 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.728+0000 7f5d97400640 1 -- 172.21.15.145:0/4157845816 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d98070b80 con 0x7f5d98071380 2024-05-09T04:34:35.729 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.729+0000 7f5d97400640 1 --2- 172.21.15.145:0/4157845816 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d98071380 0x7f5d980b2440 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5d90009d90 tx=0x7f5d9002a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b203eced7762c8f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:35.729 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.729+0000 7f5d96000640 1 -- 172.21.15.145:0/4157845816 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d90007740 con 0x7f5d98071380 2024-05-09T04:34:35.729 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.729+0000 7f5d96000640 1 -- 172.21.15.145:0/4157845816 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d900078e0 con 0x7f5d98071380 2024-05-09T04:34:35.730 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.729+0000 7f5d96000640 1 -- 172.21.15.145:0/4157845816 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d90007cb0 con 0x7f5d98071380 2024-05-09T04:34:35.730 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.729+0000 7f5d9d600640 1 -- 172.21.15.145:0/4157845816 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d98071380 msgr2=0x7f5d980b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:35.730 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.729+0000 7f5d9d600640 1 --2- 172.21.15.145:0/4157845816 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d98071380 0x7f5d980b2440 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5d90009d90 tx=0x7f5d9002a6f0 comp rx=0 tx=0).stop 2024-05-09T04:34:35.730 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.730+0000 7f5d9d600640 1 -- 172.21.15.145:0/4157845816 shutdown_connections 2024-05-09T04:34:35.730 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.730+0000 7f5d9d600640 1 --2- 172.21.15.145:0/4157845816 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d98071380 0x7f5d980b2440 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:35.730 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.730+0000 7f5d9d600640 1 -- 172.21.15.145:0/4157845816 >> 172.21.15.145:0/4157845816 conn(0x7f5d980b4a50 msgr2=0x7f5d980b4e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:35.730 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.730+0000 7f5d9d600640 1 -- 172.21.15.145:0/4157845816 shutdown_connections 2024-05-09T04:34:35.730 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.730+0000 7f5d9d600640 1 -- 172.21.15.145:0/4157845816 wait complete. 2024-05-09T04:34:35.731 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.731+0000 7f5d9d600640 1 Processor -- start 2024-05-09T04:34:35.731 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.732+0000 7f5d9d600640 1 -- start start 2024-05-09T04:34:35.733 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.732+0000 7f5d9d600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5d98071380 0x7f5d980c7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:35.733 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.732+0000 7f5d9d600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5d9809ba10 0x7f5d980c6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:35.733 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.732+0000 7f5d9d600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d98155b40 0x7f5d981563b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:35.733 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.732+0000 7f5d9d600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5d98160590 con 0x7f5d9809ba10 2024-05-09T04:34:35.733 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.732+0000 7f5d9d600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5d98160410 con 0x7f5d98155b40 2024-05-09T04:34:35.733 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.732+0000 7f5d9d600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f5d98160710 con 0x7f5d98071380 2024-05-09T04:34:35.733 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.733+0000 7f5d8fe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5d9809ba10 0x7f5d980c6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:35.733 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.733+0000 7f5d8fe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5d9809ba10 0x7f5d980c6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:51148/0 (socket says 172.21.15.145:51148) 2024-05-09T04:34:35.734 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.733+0000 7f5d8fe00640 1 -- 172.21.15.145:0/4085198759 learned_addr learned my addr 172.21.15.145:0/4085198759 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:35.734 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.733+0000 7f5d96a00640 1 --2- 172.21.15.145:0/4085198759 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5d98071380 0x7f5d980c7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:35.734 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.733+0000 7f5d97400640 1 --2- 172.21.15.145:0/4085198759 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d98155b40 0x7f5d981563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:35.734 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.733+0000 7f5d8fe00640 1 -- 172.21.15.145:0/4085198759 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5d98071380 msgr2=0x7f5d980c7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:35.734 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.733+0000 7f5d8fe00640 1 --2- 172.21.15.145:0/4085198759 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5d98071380 0x7f5d980c7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:35.734 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.733+0000 7f5d8fe00640 1 -- 172.21.15.145:0/4085198759 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d98155b40 msgr2=0x7f5d981563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:35.734 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.733+0000 7f5d8fe00640 1 --2- 172.21.15.145:0/4085198759 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d98155b40 0x7f5d981563b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:35.734 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.733+0000 7f5d8fe00640 1 -- 172.21.15.145:0/4085198759 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d98163220 con 0x7f5d9809ba10 2024-05-09T04:34:35.734 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.733+0000 7f5d97400640 1 --2- 172.21.15.145:0/4085198759 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d98155b40 0x7f5d981563b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:34:35.734 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.733+0000 7f5d8fe00640 1 --2- 172.21.15.145:0/4085198759 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5d9809ba10 0x7f5d980c6a70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5d8000c420 tx=0x7f5d8000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:35.735 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.734+0000 7f5d8ea00640 1 -- 172.21.15.145:0/4085198759 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d80017070 con 0x7f5d9809ba10 2024-05-09T04:34:35.735 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.734+0000 7f5d8ea00640 1 -- 172.21.15.145:0/4085198759 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d80005ce0 con 0x7f5d9809ba10 2024-05-09T04:34:35.735 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.734+0000 7f5d8ea00640 1 -- 172.21.15.145:0/4085198759 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d800072d0 con 0x7f5d9809ba10 2024-05-09T04:34:35.735 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.734+0000 7f5d9d600640 1 -- 172.21.15.145:0/4085198759 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d98163490 con 0x7f5d9809ba10 2024-05-09T04:34:35.735 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.734+0000 7f5d9d600640 1 -- 172.21.15.145:0/4085198759 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d980c7a10 con 0x7f5d9809ba10 2024-05-09T04:34:35.735 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.735+0000 7f5d9d600640 1 -- 172.21.15.145:0/4085198759 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d9806a310 con 0x7f5d9809ba10 2024-05-09T04:34:35.736 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.736+0000 7f5d8ea00640 1 -- 172.21.15.145:0/4085198759 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5d8001a070 con 0x7f5d9809ba10 2024-05-09T04:34:35.737 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.737+0000 7f5d8ea00640 1 --2- 172.21.15.145:0/4085198759 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f5d740762d0 0x7f5d74078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:35.737 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.737+0000 7f5d8ea00640 1 -- 172.21.15.145:0/4085198759 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 4943+0+0 (secure 0 0 0) 0x7f5d80096640 con 0x7f5d9809ba10 2024-05-09T04:34:35.737 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.737+0000 7f5d96a00640 1 --2- 172.21.15.145:0/4085198759 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f5d740762d0 0x7f5d74078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:35.738 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.738+0000 7f5d96a00640 1 --2- 172.21.15.145:0/4085198759 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f5d740762d0 0x7f5d74078790 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5d88000c00 tx=0x7f5d88009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:35.741 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.742+0000 7f5d8ea00640 1 -- 172.21.15.145:0/4085198759 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5d8005fbb0 con 0x7f5d9809ba10 2024-05-09T04:34:35.973 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:35.973+0000 7f5d9d600640 1 -- 172.21.15.145:0/4085198759 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f5d980d2a80 con 0x7f5d9809ba10 2024-05-09T04:34:36.467 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.466+0000 7f5d8ea00640 1 -- 172.21.15.145:0/4085198759 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v26) ==== 97+0+0 (secure 0 0 0) 0x7f5d80063eb0 con 0x7f5d9809ba10 2024-05-09T04:34:36.467 INFO:teuthology.orchestra.run.smithi145.stderr: removed all blocklist entries 2024-05-09T04:34:36.471 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.471+0000 7f5d9d600640 1 -- 172.21.15.145:0/4085198759 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f5d740762d0 msgr2=0x7f5d74078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:36.471 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.471+0000 7f5d9d600640 1 --2- 172.21.15.145:0/4085198759 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f5d740762d0 0x7f5d74078790 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5d88000c00 tx=0x7f5d88009040 comp rx=0 tx=0).stop 2024-05-09T04:34:36.471 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.471+0000 7f5d9d600640 1 -- 172.21.15.145:0/4085198759 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5d9809ba10 msgr2=0x7f5d980c6a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:36.471 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.471+0000 7f5d9d600640 1 --2- 172.21.15.145:0/4085198759 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5d9809ba10 0x7f5d980c6a70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5d8000c420 tx=0x7f5d8000c8e0 comp rx=0 tx=0).stop 2024-05-09T04:34:36.471 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.471+0000 7f5d9d600640 1 -- 172.21.15.145:0/4085198759 shutdown_connections 2024-05-09T04:34:36.471 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.471+0000 7f5d9d600640 1 --2- 172.21.15.145:0/4085198759 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f5d740762d0 0x7f5d74078790 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:36.472 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.471+0000 7f5d9d600640 1 --2- 172.21.15.145:0/4085198759 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d98155b40 0x7f5d981563b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:36.472 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.471+0000 7f5d9d600640 1 --2- 172.21.15.145:0/4085198759 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5d9809ba10 0x7f5d980c6a70 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:36.472 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.471+0000 7f5d9d600640 1 --2- 172.21.15.145:0/4085198759 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5d98071380 0x7f5d980c7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:36.472 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.471+0000 7f5d9d600640 1 -- 172.21.15.145:0/4085198759 >> 172.21.15.145:0/4085198759 conn(0x7f5d980b4a50 msgr2=0x7f5d98075400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:36.472 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.472+0000 7f5d9d600640 1 -- 172.21.15.145:0/4085198759 shutdown_connections 2024-05-09T04:34:36.472 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.472+0000 7f5d9d600640 1 -- 172.21.15.145:0/4085198759 wait complete. 2024-05-09T04:34:36.493 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-09T04:34:36.689 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.689+0000 7efcb8800640 1 Processor -- start 2024-05-09T04:34:36.689 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.689+0000 7efcb8800640 1 -- start start 2024-05-09T04:34:36.691 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.691+0000 7efcb8800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7efcb0162cc0 0x7efcb00b5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:36.691 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.691+0000 7efcb8800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7efcb00d2150 0x7efcb00d0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:36.691 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.691+0000 7efcb8800640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcb00cfd10 con 0x7efcb00d5910 2024-05-09T04:34:36.691 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.691+0000 7efcb8800640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7efcb00d4120 con 0x7efcb0162cc0 2024-05-09T04:34:36.691 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.691+0000 7efcb8800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7efcb00d6080 con 0x7efcb00d2150 2024-05-09T04:34:36.691 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.691+0000 7efcb5e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7efcb0162cc0 0x7efcb00b5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:36.691 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.691+0000 7efcb5e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7efcb0162cc0 0x7efcb00b5fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:51160/0 (socket says 172.21.15.145:51160) 2024-05-09T04:34:36.691 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.691+0000 7efcb5e00640 1 -- 172.21.15.145:0/1878756519 learned_addr learned my addr 172.21.15.145:0/1878756519 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:36.692 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.692+0000 7efcb6800640 1 --2- 172.21.15.145:0/1878756519 >> v2:172.21.15.192:3300/0 conn(0x7efcb00d2150 0x7efcb00d0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:36.692 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.692+0000 7efcb5e00640 1 -- 172.21.15.145:0/1878756519 >> v2:172.21.15.192:3300/0 conn(0x7efcb00d2150 msgr2=0x7efcb00d0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:36.692 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.692+0000 7efcb5e00640 1 --2- 172.21.15.145:0/1878756519 >> v2:172.21.15.192:3300/0 conn(0x7efcb00d2150 0x7efcb00d0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:36.692 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.692+0000 7efcb5e00640 1 -- 172.21.15.145:0/1878756519 >> v1:172.21.15.192:6789/0 conn(0x7efcb00d5910 legacy=0x7efcb00d1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:36.692 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.692+0000 7efcb5e00640 1 -- 172.21.15.145:0/1878756519 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcb00d3880 con 0x7efcb0162cc0 2024-05-09T04:34:36.692 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.692+0000 7efcb6800640 1 --2- 172.21.15.145:0/1878756519 >> v2:172.21.15.192:3300/0 conn(0x7efcb00d2150 0x7efcb00d0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:34:36.694 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.693+0000 7efcb5e00640 1 --2- 172.21.15.145:0/1878756519 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efcb0162cc0 0x7efcb00b5fe0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7efcb00c8a30 tx=0x7efca802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8337f6f191c73d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:36.694 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.693+0000 7efcb4a00640 1 -- 172.21.15.145:0/1878756519 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca8036070 con 0x7efcb0162cc0 2024-05-09T04:34:36.694 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.693+0000 7efcb4a00640 1 -- 172.21.15.145:0/1878756519 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efca8031600 con 0x7efcb0162cc0 2024-05-09T04:34:36.694 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.693+0000 7efcb4a00640 1 -- 172.21.15.145:0/1878756519 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca80319d0 con 0x7efcb0162cc0 2024-05-09T04:34:36.694 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.694+0000 7efcb8800640 1 -- 172.21.15.145:0/1878756519 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efcb0162cc0 msgr2=0x7efcb00b5fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:36.694 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.694+0000 7efcb8800640 1 --2- 172.21.15.145:0/1878756519 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efcb0162cc0 0x7efcb00b5fe0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7efcb00c8a30 tx=0x7efca802a610 comp rx=0 tx=0).stop 2024-05-09T04:34:36.694 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.694+0000 7efcb8800640 1 -- 172.21.15.145:0/1878756519 shutdown_connections 2024-05-09T04:34:36.694 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.694+0000 7efcb8800640 1 --2- 172.21.15.145:0/1878756519 >> v2:172.21.15.192:3300/0 conn(0x7efcb00d2150 0x7efcb00d0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:36.694 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.694+0000 7efcb8800640 1 --2- 172.21.15.145:0/1878756519 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efcb0162cc0 0x7efcb00b5fe0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:36.695 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.694+0000 7efcb8800640 1 -- 172.21.15.145:0/1878756519 >> 172.21.15.145:0/1878756519 conn(0x7efcb009e540 msgr2=0x7efcb00d5200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:36.695 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.694+0000 7efcb8800640 1 -- 172.21.15.145:0/1878756519 shutdown_connections 2024-05-09T04:34:36.695 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.694+0000 7efcb8800640 1 -- 172.21.15.145:0/1878756519 wait complete. 2024-05-09T04:34:36.696 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.696+0000 7efcb8800640 1 Processor -- start 2024-05-09T04:34:36.696 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.696+0000 7efcb8800640 1 -- start start 2024-05-09T04:34:36.698 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.697+0000 7efcb8800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efcb00d2150 0x7efcb00ba9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:36.698 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.697+0000 7efcb8800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efcb00d5910 0x7efcb0163420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:36.698 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.697+0000 7efcb8800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efcb0162cc0 0x7efcb0163960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:36.698 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.697+0000 7efcb8800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7efcb00d5d70 con 0x7efcb0162cc0 2024-05-09T04:34:36.698 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.697+0000 7efcb8800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7efcb00d3e40 con 0x7efcb00d2150 2024-05-09T04:34:36.698 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.697+0000 7efcb8800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7efcb00b4980 con 0x7efcb00d5910 2024-05-09T04:34:36.699 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.697+0000 7efcb6800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efcb0162cc0 0x7efcb0163960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:36.699 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.697+0000 7efcb6800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efcb0162cc0 0x7efcb0163960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:51168/0 (socket says 172.21.15.145:51168) 2024-05-09T04:34:36.699 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.697+0000 7efcb6800640 1 -- 172.21.15.145:0/150455370 learned_addr learned my addr 172.21.15.145:0/150455370 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:36.699 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.698+0000 7efcb5e00640 1 --2- 172.21.15.145:0/150455370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efcb00d2150 0x7efcb00ba9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:36.699 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.698+0000 7efcb6800640 1 -- 172.21.15.145:0/150455370 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efcb00d5910 msgr2=0x7efcb0163420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:36.699 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.698+0000 7efcb5400640 1 --2- 172.21.15.145:0/150455370 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efcb00d5910 0x7efcb0163420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:36.700 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.698+0000 7efcb6800640 1 --2- 172.21.15.145:0/150455370 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efcb00d5910 0x7efcb0163420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:36.700 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.698+0000 7efcb6800640 1 -- 172.21.15.145:0/150455370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efcb00d2150 msgr2=0x7efcb00ba9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:36.700 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.698+0000 7efcb6800640 1 --2- 172.21.15.145:0/150455370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efcb00d2150 0x7efcb00ba9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:36.700 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.698+0000 7efcb6800640 1 -- 172.21.15.145:0/150455370 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcb00ce770 con 0x7efcb0162cc0 2024-05-09T04:34:36.700 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.698+0000 7efcb6800640 1 --2- 172.21.15.145:0/150455370 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efcb0162cc0 0x7efcb0163960 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7efcac00c5a0 tx=0x7efcac00ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:36.701 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.698+0000 7efca6000640 1 -- 172.21.15.145:0/150455370 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcac015070 con 0x7efcb0162cc0 2024-05-09T04:34:36.701 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.698+0000 7efca6000640 1 -- 172.21.15.145:0/150455370 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcac004240 con 0x7efcb0162cc0 2024-05-09T04:34:36.701 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.698+0000 7efcb8800640 1 -- 172.21.15.145:0/150455370 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcb00cea50 con 0x7efcb0162cc0 2024-05-09T04:34:36.701 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.698+0000 7efca6000640 1 -- 172.21.15.145:0/150455370 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcac003610 con 0x7efcb0162cc0 2024-05-09T04:34:36.701 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.699+0000 7efcb8800640 1 -- 172.21.15.145:0/150455370 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcb00b7fa0 con 0x7efcb0162cc0 2024-05-09T04:34:36.701 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.700+0000 7efcb8800640 1 -- 172.21.15.145:0/150455370 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcb006a310 con 0x7efcb0162cc0 2024-05-09T04:34:36.701 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.701+0000 7efca6000640 1 -- 172.21.15.145:0/150455370 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7efcac0037b0 con 0x7efcb0162cc0 2024-05-09T04:34:36.702 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.702+0000 7efca6000640 1 --2- 172.21.15.145:0/150455370 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7efc940762d0 0x7efc94078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:36.702 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.702+0000 7efca6000640 1 -- 172.21.15.145:0/150455370 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7efcac0970f0 con 0x7efcb0162cc0 2024-05-09T04:34:36.702 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.702+0000 7efcb5e00640 1 --2- 172.21.15.145:0/150455370 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7efc940762d0 0x7efc94078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:36.702 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.703+0000 7efcb5e00640 1 --2- 172.21.15.145:0/150455370 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7efc940762d0 0x7efc94078790 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7efca8033000 tx=0x7efca8009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:36.706 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.707+0000 7efca6000640 1 -- 172.21.15.145:0/150455370 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7efcac060d00 con 0x7efcb0162cc0 2024-05-09T04:34:36.985 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.985+0000 7efcb8800640 1 -- 172.21.15.145:0/150455370 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7efcb00b8170 con 0x7efcb0162cc0 2024-05-09T04:34:36.986 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.986+0000 7efca6000640 1 -- 172.21.15.145:0/150455370 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) ==== 80+0+6645 (secure 0 0 0) 0x7efcac060600 con 0x7efcb0162cc0 2024-05-09T04:34:36.986 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:34:36.987 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-05-09T04:34:36.987 INFO:teuthology.orchestra.run.smithi145.stdout: "auth_dump": [ 2024-05-09T04:34:36.987 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-05-09T04:34:36.987 INFO:teuthology.orchestra.run.smithi145.stdout: "entity": "mds.a", 2024-05-09T04:34:36.987 INFO:teuthology.orchestra.run.smithi145.stdout: "key": "AQDlUTxmfnR3ERAAUsvH1ebffiEsH8dMX+Xe1w==", 2024-05-09T04:34:36.987 INFO:teuthology.orchestra.run.smithi145.stdout: "caps": { 2024-05-09T04:34:36.987 INFO:teuthology.orchestra.run.smithi145.stdout: "mds": "allow", 2024-05-09T04:34:36.987 INFO:teuthology.orchestra.run.smithi145.stdout: "mgr": "allow *", 2024-05-09T04:34:36.987 INFO:teuthology.orchestra.run.smithi145.stdout: "mon": "allow *", 2024-05-09T04:34:36.988 INFO:teuthology.orchestra.run.smithi145.stdout: "osd": "allow *" 2024-05-09T04:34:36.988 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-05-09T04:34:36.988 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:34:36.988 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-05-09T04:34:36.988 INFO:teuthology.orchestra.run.smithi145.stdout: "entity": "mds.b", 2024-05-09T04:34:36.988 INFO:teuthology.orchestra.run.smithi145.stdout: "key": "AQDlUTxm02GmLhAAI7JaX2VaRh08cXbDLpBHDg==", 2024-05-09T04:34:36.988 INFO:teuthology.orchestra.run.smithi145.stdout: "caps": { 2024-05-09T04:34:36.988 INFO:teuthology.orchestra.run.smithi145.stdout: "mds": "allow", 2024-05-09T04:34:36.988 INFO:teuthology.orchestra.run.smithi145.stdout: "mgr": "allow *", 2024-05-09T04:34:36.989 INFO:teuthology.orchestra.run.smithi145.stdout: "mon": "allow *", 2024-05-09T04:34:36.989 INFO:teuthology.orchestra.run.smithi145.stdout: "osd": "allow *" 2024-05-09T04:34:36.989 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-05-09T04:34:36.989 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:34:36.989 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-05-09T04:34:36.989 INFO:teuthology.orchestra.run.smithi145.stdout: "entity": "mds.c", 2024-05-09T04:34:36.989 INFO:teuthology.orchestra.run.smithi145.stdout: "key": "AQDlUTxm1//EHxAADR9WdmuEIMrbMzQDHi9ltQ==", 2024-05-09T04:34:36.990 INFO:teuthology.orchestra.run.smithi145.stdout: "caps": { 2024-05-09T04:34:36.990 INFO:teuthology.orchestra.run.smithi145.stdout: "mds": "allow", 2024-05-09T04:34:36.990 INFO:teuthology.orchestra.run.smithi145.stdout: "mgr": "allow *", 2024-05-09T04:34:36.990 INFO:teuthology.orchestra.run.smithi145.stdout: "mon": "allow *", 2024-05-09T04:34:36.990 INFO:teuthology.orchestra.run.smithi145.stdout: "osd": "allow *" 2024-05-09T04:34:36.990 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-05-09T04:34:36.990 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:34:36.990 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-05-09T04:34:36.990 INFO:teuthology.orchestra.run.smithi145.stdout: "entity": "mds.d", 2024-05-09T04:34:36.991 INFO:teuthology.orchestra.run.smithi145.stdout: "key": "AQDmUTxm5qxUARAA6NW1Rj3N4APnz2hYdHOpKw==", 2024-05-09T04:34:36.991 INFO:teuthology.orchestra.run.smithi145.stdout: "caps": { 2024-05-09T04:34:36.991 INFO:teuthology.orchestra.run.smithi145.stdout: "mds": "allow", 2024-05-09T04:34:36.991 INFO:teuthology.orchestra.run.smithi145.stdout: "mgr": "allow *", 2024-05-09T04:34:36.991 INFO:teuthology.orchestra.run.smithi145.stdout: "mon": "allow *", 2024-05-09T04:34:36.991 INFO:teuthology.orchestra.run.smithi145.stdout: "osd": "allow *" 2024-05-09T04:34:36.991 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-05-09T04:34:36.991 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:34:36.991 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-05-09T04:34:36.991 INFO:teuthology.orchestra.run.smithi145.stdout: "entity": "osd.0", 2024-05-09T04:34:36.991 INFO:teuthology.orchestra.run.smithi145.stdout: "key": "AQDpUTxms65yMhAARwU3xBmDigriIR7IGY00iQ==", 2024-05-09T04:34:36.991 INFO:teuthology.orchestra.run.smithi145.stdout: "caps": { 2024-05-09T04:34:36.991 INFO:teuthology.orchestra.run.smithi145.stdout: "mgr": "allow profile osd", 2024-05-09T04:34:36.991 INFO:teuthology.orchestra.run.smithi145.stdout: "mon": "allow profile osd", 2024-05-09T04:34:36.992 INFO:teuthology.orchestra.run.smithi145.stdout: "osd": "allow *" 2024-05-09T04:34:36.992 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-05-09T04:34:36.992 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:34:36.992 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-05-09T04:34:36.992 INFO:teuthology.orchestra.run.smithi145.stdout: "entity": "osd.1", 2024-05-09T04:34:36.992 INFO:teuthology.orchestra.run.smithi145.stdout: "key": "AQDtUTxmfJohERAAEVklN8QfeSfc8C8vZTlE2A==", 2024-05-09T04:34:36.992 INFO:teuthology.orchestra.run.smithi145.stdout: "caps": { 2024-05-09T04:34:36.992 INFO:teuthology.orchestra.run.smithi145.stdout: "mgr": "allow profile osd", 2024-05-09T04:34:36.992 INFO:teuthology.orchestra.run.smithi145.stdout: "mon": "allow profile osd", 2024-05-09T04:34:36.992 INFO:teuthology.orchestra.run.smithi145.stdout: "osd": "allow *" 2024-05-09T04:34:36.992 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-05-09T04:34:36.992 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:34:36.992 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-05-09T04:34:36.993 INFO:teuthology.orchestra.run.smithi145.stdout: "entity": "osd.2", 2024-05-09T04:34:36.993 INFO:teuthology.orchestra.run.smithi145.stdout: "key": "AQDwUTxmFvcpKhAARtXZ1qSYuJp6gZCTAj0AiA==", 2024-05-09T04:34:36.993 INFO:teuthology.orchestra.run.smithi145.stdout: "caps": { 2024-05-09T04:34:36.993 INFO:teuthology.orchestra.run.smithi145.stdout: "mgr": "allow profile osd", 2024-05-09T04:34:36.993 INFO:teuthology.orchestra.run.smithi145.stdout: "mon": "allow profile osd", 2024-05-09T04:34:36.993 INFO:teuthology.orchestra.run.smithi145.stdout: "osd": "allow *" 2024-05-09T04:34:36.993 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-05-09T04:34:36.993 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:34:36.993 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-05-09T04:34:36.993 INFO:teuthology.orchestra.run.smithi145.stdout: "entity": "osd.3", 2024-05-09T04:34:36.993 INFO:teuthology.orchestra.run.smithi145.stdout: "key": "AQDzUTxmqcp8NBAAWjK3tnEyljnJEu/wAx77TQ==", 2024-05-09T04:34:36.993 INFO:teuthology.orchestra.run.smithi145.stdout: "caps": { 2024-05-09T04:34:36.993 INFO:teuthology.orchestra.run.smithi145.stdout: "mgr": "allow profile osd", 2024-05-09T04:34:36.993 INFO:teuthology.orchestra.run.smithi145.stdout: "mon": "allow profile osd", 2024-05-09T04:34:36.993 INFO:teuthology.orchestra.run.smithi145.stdout: "osd": "allow *" 2024-05-09T04:34:36.993 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-05-09T04:34:36.994 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:34:36.994 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-05-09T04:34:36.994 INFO:teuthology.orchestra.run.smithi145.stdout: "entity": "osd.4", 2024-05-09T04:34:36.994 INFO:teuthology.orchestra.run.smithi145.stdout: "key": "AQD6UTxmtvWdDxAAe9r1F/szPZid+E9/Vuio7Q==", 2024-05-09T04:34:36.994 INFO:teuthology.orchestra.run.smithi145.stdout: "caps": { 2024-05-09T04:34:36.994 INFO:teuthology.orchestra.run.smithi145.stdout: "mgr": "allow profile osd", 2024-05-09T04:34:36.994 INFO:teuthology.orchestra.run.smithi145.stdout: "mon": "allow profile osd", 2024-05-09T04:34:36.994 INFO:teuthology.orchestra.run.smithi145.stdout: "osd": "allow *" 2024-05-09T04:34:36.994 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-05-09T04:34:36.994 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:34:36.994 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-05-09T04:34:36.994 INFO:teuthology.orchestra.run.smithi145.stdout: "entity": "osd.5", 2024-05-09T04:34:36.994 INFO:teuthology.orchestra.run.smithi145.stdout: "key": "AQD9UTxm0LE1GhAAIm98EcczdlqMnivEAjIsHQ==", 2024-05-09T04:34:36.994 INFO:teuthology.orchestra.run.smithi145.stdout: "caps": { 2024-05-09T04:34:36.995 INFO:teuthology.orchestra.run.smithi145.stdout: "mgr": "allow profile osd", 2024-05-09T04:34:36.995 INFO:teuthology.orchestra.run.smithi145.stdout: "mon": "allow profile osd", 2024-05-09T04:34:36.995 INFO:teuthology.orchestra.run.smithi145.stdout: "osd": "allow *" 2024-05-09T04:34:36.995 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-05-09T04:34:36.995 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:34:36.995 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-05-09T04:34:36.995 INFO:teuthology.orchestra.run.smithi145.stdout: "entity": "osd.6", 2024-05-09T04:34:36.995 INFO:teuthology.orchestra.run.smithi145.stdout: "key": "AQAAUjxmSToZJxAAYIKW6x8/ARj4Nnz1tKgGQw==", 2024-05-09T04:34:36.995 INFO:teuthology.orchestra.run.smithi145.stdout: "caps": { 2024-05-09T04:34:36.995 INFO:teuthology.orchestra.run.smithi145.stdout: "mgr": "allow profile osd", 2024-05-09T04:34:36.995 INFO:teuthology.orchestra.run.smithi145.stdout: "mon": "allow profile osd", 2024-05-09T04:34:36.995 INFO:teuthology.orchestra.run.smithi145.stdout: "osd": "allow *" 2024-05-09T04:34:36.995 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-05-09T04:34:36.995 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:34:36.995 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-05-09T04:34:36.995 INFO:teuthology.orchestra.run.smithi145.stdout: "entity": "osd.7", 2024-05-09T04:34:36.996 INFO:teuthology.orchestra.run.smithi145.stdout: "key": "AQADUjxmkEIcMxAAn5KshgT0Ugl/LXKcIssEcQ==", 2024-05-09T04:34:36.996 INFO:teuthology.orchestra.run.smithi145.stdout: "caps": { 2024-05-09T04:34:36.996 INFO:teuthology.orchestra.run.smithi145.stdout: "mgr": "allow profile osd", 2024-05-09T04:34:36.996 INFO:teuthology.orchestra.run.smithi145.stdout: "mon": "allow profile osd", 2024-05-09T04:34:36.996 INFO:teuthology.orchestra.run.smithi145.stdout: "osd": "allow *" 2024-05-09T04:34:36.996 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-05-09T04:34:36.996 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:34:36.996 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-05-09T04:34:36.996 INFO:teuthology.orchestra.run.smithi145.stdout: "entity": "client.0", 2024-05-09T04:34:36.996 INFO:teuthology.orchestra.run.smithi145.stdout: "key": "AQDmUTxmfB7QDRAAs0TGpMOXJSerMXVrobw+hA==", 2024-05-09T04:34:36.996 INFO:teuthology.orchestra.run.smithi145.stdout: "caps": { 2024-05-09T04:34:36.996 INFO:teuthology.orchestra.run.smithi145.stdout: "mds": "allow", 2024-05-09T04:34:36.996 INFO:teuthology.orchestra.run.smithi145.stdout: "mgr": "allow r", 2024-05-09T04:34:36.996 INFO:teuthology.orchestra.run.smithi145.stdout: "mon": "allow rw", 2024-05-09T04:34:36.996 INFO:teuthology.orchestra.run.smithi145.stdout: "osd": "allow rwx" 2024-05-09T04:34:36.996 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-05-09T04:34:36.997 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:34:36.997 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-05-09T04:34:36.997 INFO:teuthology.orchestra.run.smithi145.stdout: "entity": "client.1", 2024-05-09T04:34:36.997 INFO:teuthology.orchestra.run.smithi145.stdout: "key": "AQDmUTxmZoCtIBAAwylggR+NZPk4qXCIbv9i9g==", 2024-05-09T04:34:36.997 INFO:teuthology.orchestra.run.smithi145.stdout: "caps": { 2024-05-09T04:34:36.997 INFO:teuthology.orchestra.run.smithi145.stdout: "mds": "allow", 2024-05-09T04:34:36.997 INFO:teuthology.orchestra.run.smithi145.stdout: "mgr": "allow r", 2024-05-09T04:34:36.997 INFO:teuthology.orchestra.run.smithi145.stdout: "mon": "allow rw", 2024-05-09T04:34:36.997 INFO:teuthology.orchestra.run.smithi145.stdout: "osd": "allow rwx" 2024-05-09T04:34:36.997 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-05-09T04:34:36.997 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:34:36.997 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-05-09T04:34:36.997 INFO:teuthology.orchestra.run.smithi145.stdout: "entity": "client.2", 2024-05-09T04:34:36.997 INFO:teuthology.orchestra.run.smithi145.stdout: "key": "AQDmUTxmAn/iFhAAj7Wb0NYC8Z7KEMrsgrOkjA==", 2024-05-09T04:34:36.998 INFO:teuthology.orchestra.run.smithi145.stdout: "caps": { 2024-05-09T04:34:36.998 INFO:teuthology.orchestra.run.smithi145.stdout: "mds": "allow", 2024-05-09T04:34:36.998 INFO:teuthology.orchestra.run.smithi145.stdout: "mgr": "allow r", 2024-05-09T04:34:36.998 INFO:teuthology.orchestra.run.smithi145.stdout: "mon": "allow rw", 2024-05-09T04:34:36.998 INFO:teuthology.orchestra.run.smithi145.stdout: "osd": "allow rwx" 2024-05-09T04:34:36.998 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-05-09T04:34:36.998 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:34:36.998 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-05-09T04:34:36.998 INFO:teuthology.orchestra.run.smithi145.stdout: "entity": "client.3", 2024-05-09T04:34:36.998 INFO:teuthology.orchestra.run.smithi145.stdout: "key": "AQDmUTxmcUBaKhAA9XU8rgRmVEKjczR91U76hA==", 2024-05-09T04:34:36.998 INFO:teuthology.orchestra.run.smithi145.stdout: "caps": { 2024-05-09T04:34:36.998 INFO:teuthology.orchestra.run.smithi145.stdout: "mds": "allow", 2024-05-09T04:34:36.998 INFO:teuthology.orchestra.run.smithi145.stdout: "mgr": "allow r", 2024-05-09T04:34:36.998 INFO:teuthology.orchestra.run.smithi145.stdout: "mon": "allow rw", 2024-05-09T04:34:36.999 INFO:teuthology.orchestra.run.smithi145.stdout: "osd": "allow rwx" 2024-05-09T04:34:36.999 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-05-09T04:34:36.999 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:34:36.999 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-05-09T04:34:36.999 INFO:teuthology.orchestra.run.smithi145.stdout: "entity": "client.admin", 2024-05-09T04:34:36.999 INFO:teuthology.orchestra.run.smithi145.stdout: "key": "AQDkUTxmcfhNChAA3lkT8ApjnE77ulBRGugimg==", 2024-05-09T04:34:36.999 INFO:teuthology.orchestra.run.smithi145.stdout: "caps": { 2024-05-09T04:34:36.999 INFO:teuthology.orchestra.run.smithi145.stdout: "mds": "allow *", 2024-05-09T04:34:36.999 INFO:teuthology.orchestra.run.smithi145.stdout: "mgr": "allow *", 2024-05-09T04:34:36.999 INFO:teuthology.orchestra.run.smithi145.stdout: "mon": "allow *", 2024-05-09T04:34:36.999 INFO:teuthology.orchestra.run.smithi145.stdout: "osd": "allow *" 2024-05-09T04:34:36.999 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-05-09T04:34:36.999 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:34:36.999 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-05-09T04:34:36.999 INFO:teuthology.orchestra.run.smithi145.stdout: "entity": "client.bootstrap-mds", 2024-05-09T04:34:36.999 INFO:teuthology.orchestra.run.smithi145.stdout: "key": "AQAWUjxmCECJIBAANM3qiecrFtIiavwadtMQvg==", 2024-05-09T04:34:37.000 INFO:teuthology.orchestra.run.smithi145.stdout: "caps": { 2024-05-09T04:34:37.000 INFO:teuthology.orchestra.run.smithi145.stdout: "mon": "allow profile bootstrap-mds" 2024-05-09T04:34:37.000 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-05-09T04:34:37.000 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:34:37.000 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-05-09T04:34:37.000 INFO:teuthology.orchestra.run.smithi145.stdout: "entity": "client.bootstrap-mgr", 2024-05-09T04:34:37.000 INFO:teuthology.orchestra.run.smithi145.stdout: "key": "AQAWUjxmRVuJIBAA5vnc4P4QMlm7vIELhX/lBg==", 2024-05-09T04:34:37.000 INFO:teuthology.orchestra.run.smithi145.stdout: "caps": { 2024-05-09T04:34:37.000 INFO:teuthology.orchestra.run.smithi145.stdout: "mon": "allow profile bootstrap-mgr" 2024-05-09T04:34:37.000 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-05-09T04:34:37.000 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:34:37.000 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-05-09T04:34:37.000 INFO:teuthology.orchestra.run.smithi145.stdout: "entity": "client.bootstrap-osd", 2024-05-09T04:34:37.000 INFO:teuthology.orchestra.run.smithi145.stdout: "key": "AQAWUjxmVHaJIBAArSzO0AQufEFitX8VOzX/5A==", 2024-05-09T04:34:37.000 INFO:teuthology.orchestra.run.smithi145.stdout: "caps": { 2024-05-09T04:34:37.001 INFO:teuthology.orchestra.run.smithi145.stdout: "mon": "allow profile bootstrap-osd" 2024-05-09T04:34:37.001 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-05-09T04:34:37.001 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:34:37.001 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-05-09T04:34:37.001 INFO:teuthology.orchestra.run.smithi145.stdout: "entity": "client.bootstrap-rbd", 2024-05-09T04:34:37.001 INFO:teuthology.orchestra.run.smithi145.stdout: "key": "AQAWUjxmppOJIBAAZizP1AMHkalVGAAW782bXw==", 2024-05-09T04:34:37.001 INFO:teuthology.orchestra.run.smithi145.stdout: "caps": { 2024-05-09T04:34:37.001 INFO:teuthology.orchestra.run.smithi145.stdout: "mon": "allow profile bootstrap-rbd" 2024-05-09T04:34:37.001 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-05-09T04:34:37.001 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:34:37.001 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-05-09T04:34:37.001 INFO:teuthology.orchestra.run.smithi145.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-05-09T04:34:37.001 INFO:teuthology.orchestra.run.smithi145.stdout: "key": "AQAWUjxmeqyJIBAAzXuWhnOcxpNWovQu+WW0Ug==", 2024-05-09T04:34:37.001 INFO:teuthology.orchestra.run.smithi145.stdout: "caps": { 2024-05-09T04:34:37.001 INFO:teuthology.orchestra.run.smithi145.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-05-09T04:34:37.001 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-05-09T04:34:37.002 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:34:37.002 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-05-09T04:34:37.002 INFO:teuthology.orchestra.run.smithi145.stdout: "entity": "client.bootstrap-rgw", 2024-05-09T04:34:37.002 INFO:teuthology.orchestra.run.smithi145.stdout: "key": "AQAWUjxmAsWJIBAAerdz80bHiXcXAr4LjOV5/g==", 2024-05-09T04:34:37.002 INFO:teuthology.orchestra.run.smithi145.stdout: "caps": { 2024-05-09T04:34:37.002 INFO:teuthology.orchestra.run.smithi145.stdout: "mon": "allow profile bootstrap-rgw" 2024-05-09T04:34:37.002 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-05-09T04:34:37.002 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:34:37.002 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-05-09T04:34:37.002 INFO:teuthology.orchestra.run.smithi145.stdout: "entity": "mgr.x", 2024-05-09T04:34:37.002 INFO:teuthology.orchestra.run.smithi145.stdout: "key": "AQDlUTxmYdonBxAA9AoBwsKJCkAOsuTcY2zXHQ==", 2024-05-09T04:34:37.002 INFO:teuthology.orchestra.run.smithi145.stdout: "caps": { 2024-05-09T04:34:37.002 INFO:teuthology.orchestra.run.smithi145.stdout: "mds": "allow *", 2024-05-09T04:34:37.002 INFO:teuthology.orchestra.run.smithi145.stdout: "mon": "allow profile mgr", 2024-05-09T04:34:37.003 INFO:teuthology.orchestra.run.smithi145.stdout: "osd": "allow *" 2024-05-09T04:34:37.003 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-05-09T04:34:37.003 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:34:37.003 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-05-09T04:34:37.003 INFO:teuthology.orchestra.run.smithi145.stdout: "entity": "mgr.y", 2024-05-09T04:34:37.003 INFO:teuthology.orchestra.run.smithi145.stdout: "key": "AQDkUTxmSBenNxAAns0hdQXzIWDm3C86Jrc8gw==", 2024-05-09T04:34:37.003 INFO:teuthology.orchestra.run.smithi145.stdout: "caps": { 2024-05-09T04:34:37.003 INFO:teuthology.orchestra.run.smithi145.stdout: "mds": "allow *", 2024-05-09T04:34:37.003 INFO:teuthology.orchestra.run.smithi145.stdout: "mon": "allow profile mgr", 2024-05-09T04:34:37.003 INFO:teuthology.orchestra.run.smithi145.stdout: "osd": "allow *" 2024-05-09T04:34:37.003 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-05-09T04:34:37.003 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-05-09T04:34:37.003 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-05-09T04:34:37.003 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-05-09T04:34:37.005 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.990+0000 7efcb8800640 1 -- 172.21.15.145:0/150455370 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7efc940762d0 msgr2=0x7efc94078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:37.005 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.990+0000 7efcb8800640 1 --2- 172.21.15.145:0/150455370 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7efc940762d0 0x7efc94078790 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7efca8033000 tx=0x7efca8009000 comp rx=0 tx=0).stop 2024-05-09T04:34:37.005 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.990+0000 7efcb8800640 1 -- 172.21.15.145:0/150455370 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efcb0162cc0 msgr2=0x7efcb0163960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:37.005 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.990+0000 7efcb8800640 1 --2- 172.21.15.145:0/150455370 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efcb0162cc0 0x7efcb0163960 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7efcac00c5a0 tx=0x7efcac00ca60 comp rx=0 tx=0).stop 2024-05-09T04:34:37.005 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.990+0000 7efcb8800640 1 -- 172.21.15.145:0/150455370 shutdown_connections 2024-05-09T04:34:37.005 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.990+0000 7efcb8800640 1 --2- 172.21.15.145:0/150455370 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7efc940762d0 0x7efc94078790 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:37.005 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.990+0000 7efcb8800640 1 --2- 172.21.15.145:0/150455370 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efcb0162cc0 0x7efcb0163960 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:37.005 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.990+0000 7efcb8800640 1 --2- 172.21.15.145:0/150455370 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efcb00d5910 0x7efcb0163420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:37.005 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.990+0000 7efcb8800640 1 --2- 172.21.15.145:0/150455370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efcb00d2150 0x7efcb00ba9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:37.005 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.990+0000 7efcb8800640 1 -- 172.21.15.145:0/150455370 >> 172.21.15.145:0/150455370 conn(0x7efcb009e540 msgr2=0x7efcb00d0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:37.005 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.990+0000 7efcb8800640 1 -- 172.21.15.145:0/150455370 shutdown_connections 2024-05-09T04:34:37.005 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:36.990+0000 7efcb8800640 1 -- 172.21.15.145:0/150455370 wait complete. 2024-05-09T04:34:37.007 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-05-09T04:34:37.198 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.198+0000 7fd246e00640 1 Processor -- start 2024-05-09T04:34:37.198 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.198+0000 7fd246e00640 1 -- start start 2024-05-09T04:34:37.200 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.200+0000 7fd246e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd24015f3b0 0x7fd2400cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:37.200 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.200+0000 7fd246e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fd2400afc90 0x7fd2400ce330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:37.200 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.200+0000 7fd246e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd24015f790 con 0x7fd2400745e0 2024-05-09T04:34:37.200 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.200+0000 7fd246e00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fd2400b6440 con 0x7fd2400afc90 2024-05-09T04:34:37.200 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.200+0000 7fd246e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd2400b6160 con 0x7fd24015f3b0 2024-05-09T04:34:37.201 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.200+0000 7fd244e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fd2400afc90 0x7fd2400ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:37.201 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.201+0000 7fd244e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fd2400afc90 0x7fd2400ce330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:51184/0 (socket says 172.21.15.145:51184) 2024-05-09T04:34:37.201 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.201+0000 7fd244e00640 1 -- 172.21.15.145:0/2726453324 learned_addr learned my addr 172.21.15.145:0/2726453324 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:37.201 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.201+0000 7fd23fe00640 1 --2- 172.21.15.145:0/2726453324 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd24015f3b0 0x7fd2400cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:37.201 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.201+0000 7fd244e00640 1 -- 172.21.15.145:0/2726453324 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd24015f3b0 msgr2=0x7fd2400cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:37.201 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.201+0000 7fd244e00640 1 --2- 172.21.15.145:0/2726453324 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd24015f3b0 0x7fd2400cddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:37.201 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.201+0000 7fd244e00640 1 -- 172.21.15.145:0/2726453324 >> v1:172.21.15.192:6789/0 conn(0x7fd2400745e0 legacy=0x7fd2400ced50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:37.201 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.201+0000 7fd244e00640 1 -- 172.21.15.145:0/2726453324 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2400af1b0 con 0x7fd2400afc90 2024-05-09T04:34:37.202 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.202+0000 7fd244e00640 1 --2- 172.21.15.145:0/2726453324 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd2400afc90 0x7fd2400ce330 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd238009e50 tx=0x7fd23802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdedd32619ce8ca4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:37.202 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.203+0000 7fd23ea00640 1 -- 172.21.15.145:0/2726453324 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd238036070 con 0x7fd2400afc90 2024-05-09T04:34:37.202 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.203+0000 7fd23ea00640 1 -- 172.21.15.145:0/2726453324 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd238031600 con 0x7fd2400afc90 2024-05-09T04:34:37.202 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.203+0000 7fd23ea00640 1 -- 172.21.15.145:0/2726453324 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2380319d0 con 0x7fd2400afc90 2024-05-09T04:34:37.203 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.203+0000 7fd246e00640 1 -- 172.21.15.145:0/2726453324 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd2400afc90 msgr2=0x7fd2400ce330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:37.203 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.203+0000 7fd246e00640 1 --2- 172.21.15.145:0/2726453324 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd2400afc90 0x7fd2400ce330 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd238009e50 tx=0x7fd23802a680 comp rx=0 tx=0).stop 2024-05-09T04:34:37.204 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.204+0000 7fd246e00640 1 -- 172.21.15.145:0/2726453324 shutdown_connections 2024-05-09T04:34:37.204 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.204+0000 7fd246e00640 1 --2- 172.21.15.145:0/2726453324 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd2400afc90 0x7fd2400ce330 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:37.204 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.204+0000 7fd246e00640 1 --2- 172.21.15.145:0/2726453324 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd24015f3b0 0x7fd2400cddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:37.204 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.204+0000 7fd246e00640 1 -- 172.21.15.145:0/2726453324 >> 172.21.15.145:0/2726453324 conn(0x7fd24006c8d0 msgr2=0x7fd24006cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:37.204 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.204+0000 7fd246e00640 1 -- 172.21.15.145:0/2726453324 shutdown_connections 2024-05-09T04:34:37.204 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.204+0000 7fd246e00640 1 -- 172.21.15.145:0/2726453324 wait complete. 2024-05-09T04:34:37.205 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.205+0000 7fd246e00640 1 Processor -- start 2024-05-09T04:34:37.205 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.206+0000 7fd246e00640 1 -- start start 2024-05-09T04:34:37.207 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.206+0000 7fd246e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd2400745e0 0x7fd240155070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:37.207 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.206+0000 7fd246e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd2400afc90 0x7fd2401606f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:37.207 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.207+0000 7fd246e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd24015f3b0 0x7fd2401f6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:37.207 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.207+0000 7fd246e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd2400b5d30 con 0x7fd2400afc90 2024-05-09T04:34:37.207 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.207+0000 7fd246e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd2400af770 con 0x7fd24015f3b0 2024-05-09T04:34:37.207 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.207+0000 7fd246e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd2400b5eb0 con 0x7fd2400745e0 2024-05-09T04:34:37.207 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.207+0000 7fd23f400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd2400afc90 0x7fd2401606f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:37.207 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.207+0000 7fd23f400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd2400afc90 0x7fd2401606f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:51188/0 (socket says 172.21.15.145:51188) 2024-05-09T04:34:37.207 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.207+0000 7fd23f400640 1 -- 172.21.15.145:0/662107937 learned_addr learned my addr 172.21.15.145:0/662107937 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:37.208 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.207+0000 7fd23f400640 1 -- 172.21.15.145:0/662107937 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd2400745e0 msgr2=0x7fd240155070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:37.208 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.207+0000 7fd23f400640 1 --2- 172.21.15.145:0/662107937 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd2400745e0 0x7fd240155070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:37.208 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.207+0000 7fd23f400640 1 -- 172.21.15.145:0/662107937 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd24015f3b0 msgr2=0x7fd2401f6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:37.208 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.207+0000 7fd23f400640 1 --2- 172.21.15.145:0/662107937 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd24015f3b0 0x7fd2401f6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:37.208 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.207+0000 7fd23f400640 1 -- 172.21.15.145:0/662107937 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2401555b0 con 0x7fd2400afc90 2024-05-09T04:34:37.208 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.208+0000 7fd23f400640 1 --2- 172.21.15.145:0/662107937 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd2400afc90 0x7fd2401606f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd22c00c480 tx=0x7fd22c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:37.209 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.208+0000 7fd237e00640 1 -- 172.21.15.145:0/662107937 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd22c0052d0 con 0x7fd2400afc90 2024-05-09T04:34:37.209 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.208+0000 7fd237e00640 1 -- 172.21.15.145:0/662107937 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd22c005470 con 0x7fd2400afc90 2024-05-09T04:34:37.209 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.208+0000 7fd237e00640 1 -- 172.21.15.145:0/662107937 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd22c012870 con 0x7fd2400afc90 2024-05-09T04:34:37.209 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.208+0000 7fd246e00640 1 -- 172.21.15.145:0/662107937 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd24015a350 con 0x7fd2400afc90 2024-05-09T04:34:37.209 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.208+0000 7fd246e00640 1 -- 172.21.15.145:0/662107937 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd24015fef0 con 0x7fd2400afc90 2024-05-09T04:34:37.209 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.209+0000 7fd246e00640 1 -- 172.21.15.145:0/662107937 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd24006f9e0 con 0x7fd2400afc90 2024-05-09T04:34:37.210 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.210+0000 7fd237e00640 1 -- 172.21.15.145:0/662107937 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd22c012a10 con 0x7fd2400afc90 2024-05-09T04:34:37.211 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.211+0000 7fd237e00640 1 --2- 172.21.15.145:0/662107937 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fd2200762d0 0x7fd220078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:37.211 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.211+0000 7fd237e00640 1 -- 172.21.15.145:0/662107937 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7fd22c096f50 con 0x7fd2400afc90 2024-05-09T04:34:37.211 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.211+0000 7fd23fe00640 1 --2- 172.21.15.145:0/662107937 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fd2200762d0 0x7fd220078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:37.212 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.212+0000 7fd23fe00640 1 --2- 172.21.15.145:0/662107937 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fd2200762d0 0x7fd220078790 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd230006fd0 tx=0x7fd230008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:37.215 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.215+0000 7fd237e00640 1 -- 172.21.15.145:0/662107937 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd24006f9e0 con 0x7fd2400afc90 2024-05-09T04:34:37.511 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.512+0000 7fd246e00640 1 -- 172.21.15.145:0/662107937 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7fd24006a310 con 0x7fd2400afc90 2024-05-09T04:34:37.516 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.516+0000 7fd237e00640 1 -- 172.21.15.145:0/662107937 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7fd22c060bf0 con 0x7fd2400afc90 2024-05-09T04:34:37.519 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.519+0000 7fd246e00640 1 -- 172.21.15.145:0/662107937 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fd2200762d0 msgr2=0x7fd220078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:37.519 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.519+0000 7fd246e00640 1 --2- 172.21.15.145:0/662107937 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fd2200762d0 0x7fd220078790 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd230006fd0 tx=0x7fd230008040 comp rx=0 tx=0).stop 2024-05-09T04:34:37.519 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.519+0000 7fd246e00640 1 -- 172.21.15.145:0/662107937 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd2400afc90 msgr2=0x7fd2401606f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:37.519 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.519+0000 7fd246e00640 1 --2- 172.21.15.145:0/662107937 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd2400afc90 0x7fd2401606f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd22c00c480 tx=0x7fd22c00c940 comp rx=0 tx=0).stop 2024-05-09T04:34:37.519 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.519+0000 7fd246e00640 1 -- 172.21.15.145:0/662107937 shutdown_connections 2024-05-09T04:34:37.519 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.519+0000 7fd246e00640 1 --2- 172.21.15.145:0/662107937 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fd2200762d0 0x7fd220078790 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:37.520 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.519+0000 7fd246e00640 1 --2- 172.21.15.145:0/662107937 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd24015f3b0 0x7fd2401f6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:37.520 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.519+0000 7fd246e00640 1 --2- 172.21.15.145:0/662107937 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd2400afc90 0x7fd2401606f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:37.520 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.519+0000 7fd246e00640 1 --2- 172.21.15.145:0/662107937 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd2400745e0 0x7fd240155070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:37.520 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.519+0000 7fd246e00640 1 -- 172.21.15.145:0/662107937 >> 172.21.15.145:0/662107937 conn(0x7fd24006c8d0 msgr2=0x7fd240162d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:37.520 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.520+0000 7fd246e00640 1 -- 172.21.15.145:0/662107937 shutdown_connections 2024-05-09T04:34:37.520 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.520+0000 7fd246e00640 1 -- 172.21.15.145:0/662107937 wait complete. 2024-05-09T04:34:37.539 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-05-09T04:34:37.737 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.737+0000 7facd5c00640 1 Processor -- start 2024-05-09T04:34:37.737 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.738+0000 7facd5c00640 1 -- start start 2024-05-09T04:34:37.739 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.739+0000 7facd5c00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7facd00ce500 0x7facd0162f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:37.739 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.739+0000 7facd5c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7facd00745e0 0x7facd0163480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:37.739 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.739+0000 7facd5c00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facd015f790 con 0x7facd015f3b0 2024-05-09T04:34:37.739 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.739+0000 7facd5c00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7facd00b6440 con 0x7facd00ce500 2024-05-09T04:34:37.739 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.739+0000 7facd5c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7facd00b6160 con 0x7facd00745e0 2024-05-09T04:34:37.739 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.740+0000 7faccea00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7facd015f3b0 0x7facd00cddf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:59110/0 (socket says 172.21.15.145:59110) 2024-05-09T04:34:37.740 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.740+0000 7faccea00640 1 -- 172.21.15.145:0/282050864 learned_addr learned my addr 172.21.15.145:0/282050864 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:34:37.742 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.740+0000 7facce000640 1 --2- 172.21.15.145:0/282050864 >> v2:172.21.15.192:3300/0 conn(0x7facd00ce500 0x7facd0162f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:37.742 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.740+0000 7faccf400640 1 --2- 172.21.15.145:0/282050864 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7facd00745e0 0x7facd0163480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:37.742 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.740+0000 7faccd600640 1 -- 172.21.15.145:0/282050864 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2328721703 0 0) 0x7facd015f790 con 0x7facd015f3b0 2024-05-09T04:34:37.742 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.740+0000 7faccd600640 1 -- 172.21.15.145:0/282050864 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7facb4003520 con 0x7facd015f3b0 2024-05-09T04:34:37.742 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.740+0000 7faccd600640 1 -- 172.21.15.145:0/282050864 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3337982231 0 0) 0x7facb4003520 con 0x7facd015f3b0 2024-05-09T04:34:37.742 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.740+0000 7faccd600640 1 -- 172.21.15.145:0/282050864 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7facd00745e0 msgr2=0x7facd0163480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:37.742 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.740+0000 7faccd600640 1 --2- 172.21.15.145:0/282050864 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7facd00745e0 0x7facd0163480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:37.742 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.741+0000 7faccd600640 1 -- 172.21.15.145:0/282050864 >> v2:172.21.15.192:3300/0 conn(0x7facd00ce500 msgr2=0x7facd0162f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:37.742 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.741+0000 7faccd600640 1 --2- 172.21.15.145:0/282050864 >> v2:172.21.15.192:3300/0 conn(0x7facd00ce500 0x7facd0162f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:37.742 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.741+0000 7faccd600640 1 -- 172.21.15.145:0/282050864 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facd00af1b0 con 0x7facd015f3b0 2024-05-09T04:34:37.743 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.741+0000 7faccd600640 1 -- 172.21.15.145:0/282050864 <== mon.0 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3474413261 0 0) 0x7facc00016f0 con 0x7facd015f3b0 2024-05-09T04:34:37.743 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.741+0000 7faccd600640 1 -- 172.21.15.145:0/282050864 >> v1:172.21.15.145:6789/0 conn(0x7facd015f3b0 legacy=0x7facd00cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:37.743 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.741+0000 7faccd600640 1 --2- 172.21.15.145:0/282050864 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7facb4003a00 0x7facb4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:37.743 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.741+0000 7faccd600640 1 --2- 172.21.15.145:0/282050864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7facb4004800 0x7facb4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:37.743 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.741+0000 7faccd600640 1 --2- 172.21.15.145:0/282050864 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7facd00745e0 0x7facb40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:37.743 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.741+0000 7faccd600640 1 -- 172.21.15.145:0/282050864 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7facd00b6440 con 0x7facb4003a00 2024-05-09T04:34:37.743 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.741+0000 7faccd600640 1 -- 172.21.15.145:0/282050864 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7facd00b6160 con 0x7facb4004800 2024-05-09T04:34:37.743 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.741+0000 7faccd600640 1 -- 172.21.15.145:0/282050864 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7facb400ba70 con 0x7facd00745e0 2024-05-09T04:34:37.743 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.741+0000 7faccea00640 1 --2- 172.21.15.145:0/282050864 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7facb4003a00 0x7facb4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:37.743 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.741+0000 7faccea00640 1 -- 172.21.15.145:0/282050864 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7facd00745e0 msgr2=0x7facb40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:37.743 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.741+0000 7faccea00640 1 --2- 172.21.15.145:0/282050864 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7facd00745e0 0x7facb40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:37.743 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.741+0000 7faccea00640 1 -- 172.21.15.145:0/282050864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7facb4004800 msgr2=0x7facb4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:37.744 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.741+0000 7facce000640 1 --2- 172.21.15.145:0/282050864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7facb4004800 0x7facb4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:37.744 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.741+0000 7faccea00640 1 --2- 172.21.15.145:0/282050864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7facb4004800 0x7facb4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:37.744 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.741+0000 7faccea00640 1 -- 172.21.15.145:0/282050864 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facb4003520 con 0x7facb4003a00 2024-05-09T04:34:37.744 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.742+0000 7facce000640 1 --2- 172.21.15.145:0/282050864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7facb4004800 0x7facb4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:34:37.745 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.743+0000 7faccea00640 1 --2- 172.21.15.145:0/282050864 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7facb4003a00 0x7facb4003e20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7facc000c210 tx=0x7facb4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d74ce8569722d9d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:37.745 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.743+0000 7faccd600640 1 -- 172.21.15.145:0/282050864 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc0007d30 con 0x7facb4003a00 2024-05-09T04:34:37.745 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.743+0000 7faccd600640 1 -- 172.21.15.145:0/282050864 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facc002e070 con 0x7facb4003a00 2024-05-09T04:34:37.745 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.743+0000 7faccd600640 1 -- 172.21.15.145:0/282050864 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc0005d30 con 0x7facb4003a00 2024-05-09T04:34:37.745 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.743+0000 7facd5c00640 1 -- 172.21.15.145:0/282050864 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7facb4003a00 msgr2=0x7facb4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:37.745 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.743+0000 7facd5c00640 1 --2- 172.21.15.145:0/282050864 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7facb4003a00 0x7facb4003e20 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7facc000c210 tx=0x7facb4005830 comp rx=0 tx=0).stop 2024-05-09T04:34:37.745 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.743+0000 7faccf400640 1 -- 172.21.15.145:0/282050864 reap_dead start 2024-05-09T04:34:37.746 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.743+0000 7facd5c00640 1 -- 172.21.15.145:0/282050864 shutdown_connections 2024-05-09T04:34:37.746 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.743+0000 7facd5c00640 1 -- 172.21.15.145:0/282050864 >> 172.21.15.145:0/282050864 conn(0x7facd006c8d0 msgr2=0x7facd006cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:37.746 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.743+0000 7facd5c00640 1 -- 172.21.15.145:0/282050864 shutdown_connections 2024-05-09T04:34:37.746 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.744+0000 7facd5c00640 1 -- 172.21.15.145:0/282050864 wait complete. 2024-05-09T04:34:37.746 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.745+0000 7facd5c00640 1 Processor -- start 2024-05-09T04:34:37.746 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.745+0000 7facd5c00640 1 -- start start 2024-05-09T04:34:37.748 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.746+0000 7facd5c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7facb4003a00 0x7facd01416b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:37.748 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.746+0000 7facd5c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7facd015f9a0 0x7facd015b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:37.748 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.746+0000 7facd5c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7facd015bd60 0x7facd015dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:37.748 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.746+0000 7facd5c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7facd00b5d30 con 0x7facd015bd60 2024-05-09T04:34:37.748 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.746+0000 7facd5c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7facd00af770 con 0x7facd015f9a0 2024-05-09T04:34:37.748 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.746+0000 7facd5c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7facd00b5eb0 con 0x7facb4003a00 2024-05-09T04:34:37.748 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.747+0000 7faccf400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7facd015bd60 0x7facd015dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:37.749 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.747+0000 7faccf400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7facd015bd60 0x7facd015dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:51200/0 (socket says 172.21.15.145:51200) 2024-05-09T04:34:37.749 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.747+0000 7faccf400640 1 -- 172.21.15.145:0/2459380557 learned_addr learned my addr 172.21.15.145:0/2459380557 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:37.749 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.747+0000 7faccea00640 1 --2- 172.21.15.145:0/2459380557 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7facb4003a00 0x7facd01416b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:37.749 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.747+0000 7facce000640 1 --2- 172.21.15.145:0/2459380557 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7facd015f9a0 0x7facd015b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:37.749 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.747+0000 7faccf400640 1 -- 172.21.15.145:0/2459380557 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7facb4003a00 msgr2=0x7facd01416b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:37.749 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.747+0000 7faccf400640 1 --2- 172.21.15.145:0/2459380557 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7facb4003a00 0x7facd01416b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:37.749 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.747+0000 7faccf400640 1 -- 172.21.15.145:0/2459380557 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7facd015f9a0 msgr2=0x7facd015b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:37.749 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.747+0000 7faccf400640 1 --2- 172.21.15.145:0/2459380557 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7facd015f9a0 0x7facd015b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:37.749 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.747+0000 7faccf400640 1 -- 172.21.15.145:0/2459380557 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facd015fe30 con 0x7facd015bd60 2024-05-09T04:34:37.749 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.747+0000 7facce000640 1 --2- 172.21.15.145:0/2459380557 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7facd015f9a0 0x7facd015b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:34:37.749 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.747+0000 7faccf400640 1 --2- 172.21.15.145:0/2459380557 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7facd015bd60 0x7facd015dd30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7facd00aeb60 tx=0x7facc800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:37.749 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.748+0000 7facc6a00640 1 -- 172.21.15.145:0/2459380557 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc8018020 con 0x7facd015bd60 2024-05-09T04:34:37.749 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.748+0000 7facc6a00640 1 -- 172.21.15.145:0/2459380557 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facc800bb10 con 0x7facd015bd60 2024-05-09T04:34:37.749 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.748+0000 7facc6a00640 1 -- 172.21.15.145:0/2459380557 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc8014400 con 0x7facd015bd60 2024-05-09T04:34:37.749 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.748+0000 7facd5c00640 1 -- 172.21.15.145:0/2459380557 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facd0160110 con 0x7facd015bd60 2024-05-09T04:34:37.750 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.748+0000 7facd5c00640 1 -- 172.21.15.145:0/2459380557 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facd01606d0 con 0x7facd015bd60 2024-05-09T04:34:37.750 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.749+0000 7facd5c00640 1 -- 172.21.15.145:0/2459380557 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facd006a310 con 0x7facd015bd60 2024-05-09T04:34:37.750 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.750+0000 7facc6a00640 1 -- 172.21.15.145:0/2459380557 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7facc8005d10 con 0x7facd015bd60 2024-05-09T04:34:37.751 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.751+0000 7facc6a00640 1 --2- 172.21.15.145:0/2459380557 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7facac0762d0 0x7facac078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:37.751 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.752+0000 7facc6a00640 1 -- 172.21.15.145:0/2459380557 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7facc8097900 con 0x7facd015bd60 2024-05-09T04:34:37.751 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.752+0000 7faccea00640 1 --2- 172.21.15.145:0/2459380557 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7facac0762d0 0x7facac078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:37.752 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.752+0000 7faccea00640 1 --2- 172.21.15.145:0/2459380557 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7facac0762d0 0x7facac078790 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7facc0032000 tx=0x7facc0007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:37.756 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:37.757+0000 7facc6a00640 1 -- 172.21.15.145:0/2459380557 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7facc8022030 con 0x7facd015bd60 2024-05-09T04:34:38.043 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.043+0000 7facd5c00640 1 -- 172.21.15.145:0/2459380557 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7facd0141bf0 con 0x7facd015bd60 2024-05-09T04:34:38.048 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.048+0000 7facc6a00640 1 -- 172.21.15.145:0/2459380557 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7facc80615a0 con 0x7facd015bd60 2024-05-09T04:34:38.051 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.051+0000 7facd5c00640 1 -- 172.21.15.145:0/2459380557 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7facac0762d0 msgr2=0x7facac078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:38.051 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.051+0000 7facd5c00640 1 --2- 172.21.15.145:0/2459380557 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7facac0762d0 0x7facac078790 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7facc0032000 tx=0x7facc0007000 comp rx=0 tx=0).stop 2024-05-09T04:34:38.051 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.051+0000 7facd5c00640 1 -- 172.21.15.145:0/2459380557 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7facd015bd60 msgr2=0x7facd015dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:38.051 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.051+0000 7facd5c00640 1 --2- 172.21.15.145:0/2459380557 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7facd015bd60 0x7facd015dd30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7facd00aeb60 tx=0x7facc800b480 comp rx=0 tx=0).stop 2024-05-09T04:34:38.051 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.051+0000 7facd5c00640 1 -- 172.21.15.145:0/2459380557 shutdown_connections 2024-05-09T04:34:38.052 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.051+0000 7facd5c00640 1 --2- 172.21.15.145:0/2459380557 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7facac0762d0 0x7facac078790 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:38.052 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.051+0000 7facd5c00640 1 --2- 172.21.15.145:0/2459380557 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7facd015bd60 0x7facd015dd30 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:38.052 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.051+0000 7facd5c00640 1 --2- 172.21.15.145:0/2459380557 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7facd015f9a0 0x7facd015b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:38.052 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.051+0000 7facd5c00640 1 --2- 172.21.15.145:0/2459380557 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7facb4003a00 0x7facd01416b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:38.052 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.051+0000 7facd5c00640 1 -- 172.21.15.145:0/2459380557 >> 172.21.15.145:0/2459380557 conn(0x7facd006c8d0 msgr2=0x7facd00cddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:38.052 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.051+0000 7facd5c00640 1 -- 172.21.15.145:0/2459380557 shutdown_connections 2024-05-09T04:34:38.052 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.051+0000 7facd5c00640 1 -- 172.21.15.145:0/2459380557 wait complete. 2024-05-09T04:34:38.070 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-05-09T04:34:38.272 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.272+0000 7fd363c00640 1 Processor -- start 2024-05-09T04:34:38.272 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.272+0000 7fd363c00640 1 -- start start 2024-05-09T04:34:38.276 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.274+0000 7fd363c00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd35c1432b0 0x7fd35c168100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:38.276 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.274+0000 7fd363c00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd35c15d780 con 0x7fd35c140070 2024-05-09T04:34:38.276 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.274+0000 7fd363c00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd35c15da00 con 0x7fd35c0ce500 2024-05-09T04:34:38.276 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.274+0000 7fd363c00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fd35c143700 con 0x7fd35c1432b0 2024-05-09T04:34:38.276 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.274+0000 7fd361200640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7fd35c140070 0x7fd35c0cddf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:59122/0 (socket says 172.21.15.145:59122) 2024-05-09T04:34:38.276 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.274+0000 7fd361200640 1 -- 172.21.15.145:0/3969344985 learned_addr learned my addr 172.21.15.145:0/3969344985 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:34:38.277 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.275+0000 7fd35be00640 1 -- 172.21.15.145:0/3969344985 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3781202749 0 0) 0x7fd34c001110 con 0x7fd35c140070 2024-05-09T04:34:38.277 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.275+0000 7fd35be00640 1 -- 172.21.15.145:0/3969344985 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd340003520 con 0x7fd35c140070 2024-05-09T04:34:38.277 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.275+0000 7fd361c00640 1 --2- 172.21.15.145:0/3969344985 >> v2:172.21.15.192:3300/0 conn(0x7fd35c1432b0 0x7fd35c168100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:38.277 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.275+0000 7fd35be00640 1 -- 172.21.15.145:0/3969344985 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3417161946 0 0) 0x7fd340003520 con 0x7fd35c140070 2024-05-09T04:34:38.277 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.275+0000 7fd35be00640 1 -- 172.21.15.145:0/3969344985 >> v2:172.21.15.192:3300/0 conn(0x7fd35c1432b0 msgr2=0x7fd35c168100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:38.277 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.275+0000 7fd35be00640 1 --2- 172.21.15.145:0/3969344985 >> v2:172.21.15.192:3300/0 conn(0x7fd35c1432b0 0x7fd35c168100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:38.277 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.275+0000 7fd35be00640 1 -- 172.21.15.145:0/3969344985 >> v1:172.21.15.192:6789/0 conn(0x7fd35c0ce500 legacy=0x7fd35c142ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:38.277 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.275+0000 7fd35be00640 1 -- 172.21.15.145:0/3969344985 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd35c15dce0 con 0x7fd35c140070 2024-05-09T04:34:38.277 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.275+0000 7fd35be00640 1 -- 172.21.15.145:0/3969344985 <== mon.0 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3474413261 0 0) 0x7fd34c0018c0 con 0x7fd35c140070 2024-05-09T04:34:38.277 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.275+0000 7fd35be00640 1 -- 172.21.15.145:0/3969344985 >> v1:172.21.15.145:6789/0 conn(0x7fd35c140070 legacy=0x7fd35c0cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:38.277 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.275+0000 7fd35be00640 1 --2- 172.21.15.145:0/3969344985 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd340003a00 0x7fd340003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:38.277 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.275+0000 7fd35be00640 1 --2- 172.21.15.145:0/3969344985 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd340004800 0x7fd340004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:38.277 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.275+0000 7fd35be00640 1 --2- 172.21.15.145:0/3969344985 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd340005210 0x7fd34000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:38.277 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.275+0000 7fd35be00640 1 -- 172.21.15.145:0/3969344985 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd35c143700 con 0x7fd340005210 2024-05-09T04:34:38.277 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.275+0000 7fd35be00640 1 -- 172.21.15.145:0/3969344985 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd34000c140 con 0x7fd340003a00 2024-05-09T04:34:38.278 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.275+0000 7fd35be00640 1 -- 172.21.15.145:0/3969344985 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd34000c340 con 0x7fd340004800 2024-05-09T04:34:38.278 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.276+0000 7fd361c00640 1 --2- 172.21.15.145:0/3969344985 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd340005210 0x7fd34000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:38.278 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.276+0000 7fd361200640 1 --2- 172.21.15.145:0/3969344985 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd340003a00 0x7fd340003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:38.278 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.276+0000 7fd353e00640 1 --2- 172.21.15.145:0/3969344985 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd340004800 0x7fd340004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:38.278 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.276+0000 7fd361c00640 1 -- 172.21.15.145:0/3969344985 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd340004800 msgr2=0x7fd340004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:38.278 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.276+0000 7fd361c00640 1 --2- 172.21.15.145:0/3969344985 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd340004800 0x7fd340004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:38.278 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.276+0000 7fd361c00640 1 -- 172.21.15.145:0/3969344985 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd340003a00 msgr2=0x7fd340003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:38.278 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.276+0000 7fd361c00640 1 --2- 172.21.15.145:0/3969344985 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd340003a00 0x7fd340003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:38.278 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.276+0000 7fd361c00640 1 -- 172.21.15.145:0/3969344985 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd340003520 con 0x7fd340005210 2024-05-09T04:34:38.279 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.277+0000 7fd361c00640 1 -- 172.21.15.145:0/3969344985 reap_dead start 2024-05-09T04:34:38.280 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.277+0000 7fd361c00640 1 --2- 172.21.15.145:0/3969344985 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd340005210 0x7fd34000b9c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd354007330 tx=0x7fd340005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=856a5edfff4e6125 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:38.280 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.277+0000 7fd35be00640 1 -- 172.21.15.145:0/3969344985 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd354009be0 con 0x7fd340005210 2024-05-09T04:34:38.280 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.278+0000 7fd35be00640 1 -- 172.21.15.145:0/3969344985 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd354003b90 con 0x7fd340005210 2024-05-09T04:34:38.280 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.278+0000 7fd35be00640 1 -- 172.21.15.145:0/3969344985 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd354004020 con 0x7fd340005210 2024-05-09T04:34:38.280 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.278+0000 7fd363c00640 1 -- 172.21.15.145:0/3969344985 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd340005210 msgr2=0x7fd34000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:38.280 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.278+0000 7fd363c00640 1 --2- 172.21.15.145:0/3969344985 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd340005210 0x7fd34000b9c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd354007330 tx=0x7fd340005830 comp rx=0 tx=0).stop 2024-05-09T04:34:38.280 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.278+0000 7fd363c00640 1 -- 172.21.15.145:0/3969344985 shutdown_connections 2024-05-09T04:34:38.280 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.278+0000 7fd363c00640 1 --2- 172.21.15.145:0/3969344985 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd340005210 0x7fd34000b9c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:38.280 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.278+0000 7fd363c00640 1 -- 172.21.15.145:0/3969344985 >> 172.21.15.145:0/3969344985 conn(0x7fd35c06c8d0 msgr2=0x7fd35c06cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:38.280 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.278+0000 7fd363c00640 1 -- 172.21.15.145:0/3969344985 shutdown_connections 2024-05-09T04:34:38.280 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.278+0000 7fd363c00640 1 -- 172.21.15.145:0/3969344985 wait complete. 2024-05-09T04:34:38.280 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.280+0000 7fd363c00640 1 Processor -- start 2024-05-09T04:34:38.280 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.280+0000 7fd363c00640 1 -- start start 2024-05-09T04:34:38.282 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.281+0000 7fd363c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd340005210 0x7fd35c1f3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:38.282 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.281+0000 7fd363c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd35c1f3a90 0x7fd35c12ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:38.282 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.281+0000 7fd363c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd35c12f320 0x7fd35c12cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:38.282 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.281+0000 7fd363c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd35c140850 con 0x7fd35c12f320 2024-05-09T04:34:38.282 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.281+0000 7fd363c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd35c15e330 con 0x7fd35c1f3a90 2024-05-09T04:34:38.282 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.281+0000 7fd363c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd35c140450 con 0x7fd340005210 2024-05-09T04:34:38.282 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.282+0000 7fd361c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd35c12f320 0x7fd35c12cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:38.282 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.282+0000 7fd361c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd35c12f320 0x7fd35c12cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:51210/0 (socket says 172.21.15.145:51210) 2024-05-09T04:34:38.282 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.282+0000 7fd361c00640 1 -- 172.21.15.145:0/2682898041 learned_addr learned my addr 172.21.15.145:0/2682898041 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:38.283 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.282+0000 7fd353e00640 1 --2- 172.21.15.145:0/2682898041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd35c1f3a90 0x7fd35c12ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:38.283 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.282+0000 7fd361200640 1 --2- 172.21.15.145:0/2682898041 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd340005210 0x7fd35c1f3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:38.283 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.282+0000 7fd361c00640 1 -- 172.21.15.145:0/2682898041 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd340005210 msgr2=0x7fd35c1f3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:38.283 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.282+0000 7fd361c00640 1 --2- 172.21.15.145:0/2682898041 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd340005210 0x7fd35c1f3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:38.283 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.282+0000 7fd361c00640 1 -- 172.21.15.145:0/2682898041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd35c1f3a90 msgr2=0x7fd35c12ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:38.283 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.282+0000 7fd361c00640 1 --2- 172.21.15.145:0/2682898041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd35c1f3a90 0x7fd35c12ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:38.283 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.282+0000 7fd361c00640 1 -- 172.21.15.145:0/2682898041 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd35c12f7a0 con 0x7fd35c12f320 2024-05-09T04:34:38.283 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.282+0000 7fd361c00640 1 --2- 172.21.15.145:0/2682898041 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd35c12f320 0x7fd35c12cfa0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd35402b070 tx=0x7fd354004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:38.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.283+0000 7fd359600640 1 -- 172.21.15.145:0/2682898041 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd354046070 con 0x7fd35c12f320 2024-05-09T04:34:38.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.283+0000 7fd359600640 1 -- 172.21.15.145:0/2682898041 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd354003990 con 0x7fd35c12f320 2024-05-09T04:34:38.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.283+0000 7fd359600640 1 -- 172.21.15.145:0/2682898041 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd354041410 con 0x7fd35c12f320 2024-05-09T04:34:38.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.283+0000 7fd363c00640 1 -- 172.21.15.145:0/2682898041 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd35c1f68e0 con 0x7fd35c12f320 2024-05-09T04:34:38.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.283+0000 7fd363c00640 1 -- 172.21.15.145:0/2682898041 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd35c12e020 con 0x7fd35c12f320 2024-05-09T04:34:38.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.284+0000 7fd363c00640 1 -- 172.21.15.145:0/2682898041 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd35c06a310 con 0x7fd35c12f320 2024-05-09T04:34:38.285 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.285+0000 7fd359600640 1 -- 172.21.15.145:0/2682898041 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd354049070 con 0x7fd35c12f320 2024-05-09T04:34:38.286 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.286+0000 7fd359600640 1 --2- 172.21.15.145:0/2682898041 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fd338076300 0x7fd3380787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:38.286 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.286+0000 7fd359600640 1 -- 172.21.15.145:0/2682898041 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7fd3540b7680 con 0x7fd35c12f320 2024-05-09T04:34:38.286 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.286+0000 7fd361200640 1 --2- 172.21.15.145:0/2682898041 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fd338076300 0x7fd3380787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:38.287 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.287+0000 7fd361200640 1 --2- 172.21.15.145:0/2682898041 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fd338076300 0x7fd3380787c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd34c001050 tx=0x7fd34c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:38.290 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.290+0000 7fd359600640 1 -- 172.21.15.145:0/2682898041 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd354032030 con 0x7fd35c12f320 2024-05-09T04:34:38.575 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.575+0000 7fd363c00640 1 -- 172.21.15.145:0/2682898041 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7fd35c0b40d0 con 0x7fd35c12f320 2024-05-09T04:34:38.580 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.580+0000 7fd359600640 1 -- 172.21.15.145:0/2682898041 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7fd354081290 con 0x7fd35c12f320 2024-05-09T04:34:38.583 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.583+0000 7fd363c00640 1 -- 172.21.15.145:0/2682898041 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fd338076300 msgr2=0x7fd3380787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:38.583 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.583+0000 7fd363c00640 1 --2- 172.21.15.145:0/2682898041 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fd338076300 0x7fd3380787c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd34c001050 tx=0x7fd34c009040 comp rx=0 tx=0).stop 2024-05-09T04:34:38.583 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.583+0000 7fd363c00640 1 -- 172.21.15.145:0/2682898041 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd35c12f320 msgr2=0x7fd35c12cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:38.584 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.583+0000 7fd363c00640 1 --2- 172.21.15.145:0/2682898041 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd35c12f320 0x7fd35c12cfa0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd35402b070 tx=0x7fd354004ab0 comp rx=0 tx=0).stop 2024-05-09T04:34:38.584 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.583+0000 7fd363c00640 1 -- 172.21.15.145:0/2682898041 shutdown_connections 2024-05-09T04:34:38.584 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.583+0000 7fd363c00640 1 --2- 172.21.15.145:0/2682898041 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fd338076300 0x7fd3380787c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:38.584 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.583+0000 7fd363c00640 1 --2- 172.21.15.145:0/2682898041 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd35c12f320 0x7fd35c12cfa0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:38.584 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.583+0000 7fd363c00640 1 --2- 172.21.15.145:0/2682898041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd35c1f3a90 0x7fd35c12ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:38.584 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.583+0000 7fd363c00640 1 --2- 172.21.15.145:0/2682898041 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd340005210 0x7fd35c1f3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:38.584 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.583+0000 7fd363c00640 1 -- 172.21.15.145:0/2682898041 >> 172.21.15.145:0/2682898041 conn(0x7fd35c06c8d0 msgr2=0x7fd35c1603c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:38.584 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.583+0000 7fd363c00640 1 -- 172.21.15.145:0/2682898041 shutdown_connections 2024-05-09T04:34:38.584 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.583+0000 7fd363c00640 1 -- 172.21.15.145:0/2682898041 wait complete. 2024-05-09T04:34:38.602 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-05-09T04:34:38.803 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.802+0000 7f0ba9a00640 1 Processor -- start 2024-05-09T04:34:38.803 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.803+0000 7f0ba9a00640 1 -- start start 2024-05-09T04:34:38.804 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.804+0000 7f0ba9a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f0ba406da50 0x7f0ba4167e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:38.804 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.804+0000 7f0ba9a00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ba40ce550 con 0x7f0ba4146330 2024-05-09T04:34:38.805 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.805+0000 7f0ba9a00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ba4160a70 con 0x7f0ba4160690 2024-05-09T04:34:38.805 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.805+0000 7f0ba9a00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f0ba4146b80 con 0x7f0ba406da50 2024-05-09T04:34:38.805 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.805+0000 7f0b9be00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f0ba4146330 0x7f0ba40cde40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:59138/0 (socket says 172.21.15.145:59138) 2024-05-09T04:34:38.805 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.805+0000 7f0b9be00640 1 -- 172.21.15.145:0/964678330 learned_addr learned my addr 172.21.15.145:0/964678330 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:34:38.805 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.805+0000 7f0ba3400640 1 --2- 172.21.15.145:0/964678330 >> v2:172.21.15.145:3300/0 conn(0x7f0ba406da50 0x7f0ba4167e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:38.805 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.805+0000 7f0ba2000640 1 -- 172.21.15.145:0/964678330 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3828144339 0 0) 0x7f0ba40ce550 con 0x7f0ba4146330 2024-05-09T04:34:38.805 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.805+0000 7f0ba2000640 1 -- 172.21.15.145:0/964678330 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0b88003520 con 0x7f0ba4146330 2024-05-09T04:34:38.806 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.805+0000 7f0ba3400640 1 -- 172.21.15.145:0/964678330 >> v1:172.21.15.192:6789/0 conn(0x7f0ba4160690 legacy=0x7f0ba40cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:38.806 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.806+0000 7f0ba3400640 1 -- 172.21.15.145:0/964678330 >> v1:172.21.15.145:6789/0 conn(0x7f0ba4146330 legacy=0x7f0ba40cde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:38.806 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.806+0000 7f0ba3400640 1 -- 172.21.15.145:0/964678330 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ba40d0ab0 con 0x7f0ba406da50 2024-05-09T04:34:38.806 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.807+0000 7f0ba3400640 1 --2- 172.21.15.145:0/964678330 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ba406da50 0x7f0ba4167e60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0ba4070ae0 tx=0x7f0b9c02a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afa6801df9b32409 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:38.806 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.807+0000 7f0ba2000640 1 -- 172.21.15.145:0/964678330 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c007740 con 0x7f0ba406da50 2024-05-09T04:34:38.807 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.807+0000 7f0ba2000640 1 -- 172.21.15.145:0/964678330 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b9c02ae50 con 0x7f0ba406da50 2024-05-09T04:34:38.807 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.807+0000 7f0ba2000640 1 -- 172.21.15.145:0/964678330 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c007ae0 con 0x7f0ba406da50 2024-05-09T04:34:38.807 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.807+0000 7f0ba9a00640 1 -- 172.21.15.145:0/964678330 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ba406da50 msgr2=0x7f0ba4167e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:38.808 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.807+0000 7f0ba9a00640 1 --2- 172.21.15.145:0/964678330 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ba406da50 0x7f0ba4167e60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0ba4070ae0 tx=0x7f0b9c02a630 comp rx=0 tx=0).stop 2024-05-09T04:34:38.808 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.808+0000 7f0ba9a00640 1 -- 172.21.15.145:0/964678330 shutdown_connections 2024-05-09T04:34:38.808 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.808+0000 7f0ba9a00640 1 --2- 172.21.15.145:0/964678330 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ba406da50 0x7f0ba4167e60 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:38.808 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.808+0000 7f0ba9a00640 1 -- 172.21.15.145:0/964678330 >> 172.21.15.145:0/964678330 conn(0x7f0ba40d1590 msgr2=0x7f0ba40d19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:38.808 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.808+0000 7f0ba9a00640 1 -- 172.21.15.145:0/964678330 shutdown_connections 2024-05-09T04:34:38.808 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.808+0000 7f0ba9a00640 1 -- 172.21.15.145:0/964678330 wait complete. 2024-05-09T04:34:38.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.809+0000 7f0ba9a00640 1 Processor -- start 2024-05-09T04:34:38.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.809+0000 7f0ba9a00640 1 -- start start 2024-05-09T04:34:38.811 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.810+0000 7f0ba9a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0ba406da50 0x7f0ba41597a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:38.811 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.810+0000 7f0ba9a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ba4146330 0x7f0ba40b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:38.812 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.810+0000 7f0ba9a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ba4160690 0x7f0ba415dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:38.812 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.810+0000 7f0ba9a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba40d1350 con 0x7f0ba4146330 2024-05-09T04:34:38.812 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.810+0000 7f0ba9a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba40d1070 con 0x7f0ba4160690 2024-05-09T04:34:38.812 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.810+0000 7f0ba9a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0ba406c520 con 0x7f0ba406da50 2024-05-09T04:34:38.812 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.811+0000 7f0b9be00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ba4146330 0x7f0ba40b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:38.812 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.811+0000 7f0b9be00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ba4146330 0x7f0ba40b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:51222/0 (socket says 172.21.15.145:51222) 2024-05-09T04:34:38.812 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.811+0000 7f0b9be00640 1 -- 172.21.15.145:0/2861989448 learned_addr learned my addr 172.21.15.145:0/2861989448 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:38.812 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.811+0000 7f0b9be00640 1 -- 172.21.15.145:0/2861989448 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0ba406da50 msgr2=0x7f0ba41597a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:38.812 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.811+0000 7f0b9be00640 1 --2- 172.21.15.145:0/2861989448 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0ba406da50 0x7f0ba41597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:38.812 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.811+0000 7f0b9be00640 1 -- 172.21.15.145:0/2861989448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ba4160690 msgr2=0x7f0ba415dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:38.812 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.811+0000 7f0b9be00640 1 --2- 172.21.15.145:0/2861989448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ba4160690 0x7f0ba415dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:38.812 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.811+0000 7f0b9be00640 1 -- 172.21.15.145:0/2861989448 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ba40b5fb0 con 0x7f0ba4146330 2024-05-09T04:34:38.812 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.811+0000 7f0b9be00640 1 --2- 172.21.15.145:0/2861989448 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ba4146330 0x7f0ba40b5050 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c00c500 tx=0x7f0b8c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:38.812 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.812+0000 7f0b9aa00640 1 -- 172.21.15.145:0/2861989448 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c017070 con 0x7f0ba4146330 2024-05-09T04:34:38.813 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.812+0000 7f0b9aa00640 1 -- 172.21.15.145:0/2861989448 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b8c005ce0 con 0x7f0ba4146330 2024-05-09T04:34:38.813 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.812+0000 7f0b9aa00640 1 -- 172.21.15.145:0/2861989448 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c007270 con 0x7f0ba4146330 2024-05-09T04:34:38.813 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.812+0000 7f0ba9a00640 1 -- 172.21.15.145:0/2861989448 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ba40b59f0 con 0x7f0ba4146330 2024-05-09T04:34:38.813 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.812+0000 7f0ba9a00640 1 -- 172.21.15.145:0/2861989448 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ba415a810 con 0x7f0ba4146330 2024-05-09T04:34:38.813 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.813+0000 7f0ba9a00640 1 -- 172.21.15.145:0/2861989448 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ba40694e0 con 0x7f0ba4146330 2024-05-09T04:34:38.813 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.814+0000 7f0b9aa00640 1 -- 172.21.15.145:0/2861989448 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0b8c01a070 con 0x7f0ba4146330 2024-05-09T04:34:38.815 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.815+0000 7f0b9aa00640 1 --2- 172.21.15.145:0/2861989448 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f0b80076300 0x7f0b800787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:38.815 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.815+0000 7f0b9aa00640 1 -- 172.21.15.145:0/2861989448 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f0b8c096510 con 0x7f0ba4146330 2024-05-09T04:34:38.815 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.815+0000 7f0ba2a00640 1 --2- 172.21.15.145:0/2861989448 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f0b80076300 0x7f0b800787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:38.815 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.816+0000 7f0ba2a00640 1 --2- 172.21.15.145:0/2861989448 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f0b80076300 0x7f0b800787c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0b94003cc0 tx=0x7f0b940079b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:38.819 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:38.819+0000 7f0b9aa00640 1 -- 172.21.15.145:0/2861989448 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0b8c05fb60 con 0x7f0ba4146330 2024-05-09T04:34:39.096 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.096+0000 7f0ba9a00640 1 -- 172.21.15.145:0/2861989448 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f0ba4073290 con 0x7f0ba4146330 2024-05-09T04:34:39.101 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.101+0000 7f0b9aa00640 1 -- 172.21.15.145:0/2861989448 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f0b8c063e60 con 0x7f0ba4146330 2024-05-09T04:34:39.104 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.104+0000 7f0ba9a00640 1 -- 172.21.15.145:0/2861989448 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f0b80076300 msgr2=0x7f0b800787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:39.104 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.104+0000 7f0ba9a00640 1 --2- 172.21.15.145:0/2861989448 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f0b80076300 0x7f0b800787c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0b94003cc0 tx=0x7f0b940079b0 comp rx=0 tx=0).stop 2024-05-09T04:34:39.104 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.104+0000 7f0ba9a00640 1 -- 172.21.15.145:0/2861989448 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ba4146330 msgr2=0x7f0ba40b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:39.104 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.104+0000 7f0ba9a00640 1 --2- 172.21.15.145:0/2861989448 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ba4146330 0x7f0ba40b5050 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c00c500 tx=0x7f0b8c00c9c0 comp rx=0 tx=0).stop 2024-05-09T04:34:39.104 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.104+0000 7f0ba9a00640 1 -- 172.21.15.145:0/2861989448 shutdown_connections 2024-05-09T04:34:39.104 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.104+0000 7f0ba9a00640 1 --2- 172.21.15.145:0/2861989448 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f0b80076300 0x7f0b800787c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:39.104 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.104+0000 7f0ba9a00640 1 --2- 172.21.15.145:0/2861989448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ba4160690 0x7f0ba415dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:39.104 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.104+0000 7f0ba9a00640 1 --2- 172.21.15.145:0/2861989448 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ba4146330 0x7f0ba40b5050 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:39.104 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.104+0000 7f0ba9a00640 1 --2- 172.21.15.145:0/2861989448 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0ba406da50 0x7f0ba41597a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:39.105 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.104+0000 7f0ba9a00640 1 -- 172.21.15.145:0/2861989448 >> 172.21.15.145:0/2861989448 conn(0x7f0ba40d1590 msgr2=0x7f0ba40b2890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:39.105 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.104+0000 7f0ba9a00640 1 -- 172.21.15.145:0/2861989448 shutdown_connections 2024-05-09T04:34:39.105 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.104+0000 7f0ba9a00640 1 -- 172.21.15.145:0/2861989448 wait complete. 2024-05-09T04:34:39.123 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-05-09T04:34:39.321 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.321+0000 7fe598600640 1 Processor -- start 2024-05-09T04:34:39.321 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.322+0000 7fe598600640 1 -- start start 2024-05-09T04:34:39.323 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.323+0000 7fe598600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe590140070 0x7fe5900cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:39.323 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.323+0000 7fe598600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe5901432b0 0x7fe590160d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:39.323 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.323+0000 7fe598600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe59015d780 con 0x7fe5900ce330 2024-05-09T04:34:39.323 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.323+0000 7fe598600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fe5900ce790 con 0x7fe590140070 2024-05-09T04:34:39.323 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.323+0000 7fe598600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe59015e330 con 0x7fe5901432b0 2024-05-09T04:34:39.323 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.323+0000 7fe595200640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7fe5900ce330 0x7fe590142ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:59144/0 (socket says 172.21.15.145:59144) 2024-05-09T04:34:39.323 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.323+0000 7fe595200640 1 -- 172.21.15.145:0/2457107238 learned_addr learned my addr 172.21.15.145:0/2457107238 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:34:39.325 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.324+0000 7fe595c00640 1 --2- 172.21.15.145:0/2457107238 >> v2:172.21.15.192:3300/0 conn(0x7fe590140070 0x7fe5900cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:39.325 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.324+0000 7fe58fe00640 1 -- 172.21.15.145:0/2457107238 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 446942135 0 0) 0x7fe57c001110 con 0x7fe5900ce330 2024-05-09T04:34:39.325 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.324+0000 7fe596600640 1 --2- 172.21.15.145:0/2457107238 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe5901432b0 0x7fe590160d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:39.325 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.324+0000 7fe58fe00640 1 -- 172.21.15.145:0/2457107238 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe578003520 con 0x7fe5900ce330 2024-05-09T04:34:39.326 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.324+0000 7fe58fe00640 1 -- 172.21.15.145:0/2457107238 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 759859768 0 0) 0x7fe578003520 con 0x7fe5900ce330 2024-05-09T04:34:39.326 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.324+0000 7fe58fe00640 1 -- 172.21.15.145:0/2457107238 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe5901432b0 msgr2=0x7fe590160d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:39.326 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.324+0000 7fe58fe00640 1 --2- 172.21.15.145:0/2457107238 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe5901432b0 0x7fe590160d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:39.326 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.324+0000 7fe58fe00640 1 -- 172.21.15.145:0/2457107238 >> v2:172.21.15.192:3300/0 conn(0x7fe590140070 msgr2=0x7fe5900cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:39.326 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.324+0000 7fe58fe00640 1 --2- 172.21.15.145:0/2457107238 >> v2:172.21.15.192:3300/0 conn(0x7fe590140070 0x7fe5900cddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:39.326 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.324+0000 7fe58fe00640 1 -- 172.21.15.145:0/2457107238 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe59015da00 con 0x7fe5900ce330 2024-05-09T04:34:39.326 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.324+0000 7fe58fe00640 1 -- 172.21.15.145:0/2457107238 <== mon.0 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3474413261 0 0) 0x7fe57c0018e0 con 0x7fe5900ce330 2024-05-09T04:34:39.326 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.324+0000 7fe595c00640 1 --2- 172.21.15.145:0/2457107238 >> v2:172.21.15.192:3300/0 conn(0x7fe590140070 0x7fe5900cddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:34:39.326 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.324+0000 7fe58fe00640 1 -- 172.21.15.145:0/2457107238 >> v1:172.21.15.145:6789/0 conn(0x7fe5900ce330 legacy=0x7fe590142ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:39.326 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.324+0000 7fe58fe00640 1 --2- 172.21.15.145:0/2457107238 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe578003a00 0x7fe578003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:39.326 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.324+0000 7fe58fe00640 1 --2- 172.21.15.145:0/2457107238 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe5901432b0 0x7fe578004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:39.326 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.324+0000 7fe58fe00640 1 --2- 172.21.15.145:0/2457107238 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe578004d40 0x7fe5780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:39.326 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.324+0000 7fe58fe00640 1 -- 172.21.15.145:0/2457107238 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe59015e330 con 0x7fe578003a00 2024-05-09T04:34:39.327 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.324+0000 7fe58fe00640 1 -- 172.21.15.145:0/2457107238 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe57800ba70 con 0x7fe578004d40 2024-05-09T04:34:39.327 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.324+0000 7fe58fe00640 1 -- 172.21.15.145:0/2457107238 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe57800bbf0 con 0x7fe5901432b0 2024-05-09T04:34:39.327 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.325+0000 7fe595c00640 1 --2- 172.21.15.145:0/2457107238 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe578003a00 0x7fe578003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:39.327 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.325+0000 7fe595c00640 1 -- 172.21.15.145:0/2457107238 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe5901432b0 msgr2=0x7fe578004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:39.327 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.325+0000 7fe595c00640 1 --2- 172.21.15.145:0/2457107238 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe5901432b0 0x7fe578004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:39.327 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.325+0000 7fe595c00640 1 -- 172.21.15.145:0/2457107238 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe578004d40 msgr2=0x7fe5780051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:39.327 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.325+0000 7fe596600640 1 --2- 172.21.15.145:0/2457107238 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe578004d40 0x7fe5780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:39.327 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.325+0000 7fe595c00640 1 --2- 172.21.15.145:0/2457107238 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe578004d40 0x7fe5780051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:39.327 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.325+0000 7fe595c00640 1 -- 172.21.15.145:0/2457107238 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe578003520 con 0x7fe578003a00 2024-05-09T04:34:39.328 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.326+0000 7fe595c00640 1 --2- 172.21.15.145:0/2457107238 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe578003a00 0x7fe578003e20 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe5840098a0 tx=0x7fe578005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fbeac29ff3806d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:39.328 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.326+0000 7fe58fe00640 1 -- 172.21.15.145:0/2457107238 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe58400f040 con 0x7fe578003a00 2024-05-09T04:34:39.328 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.326+0000 7fe58fe00640 1 -- 172.21.15.145:0/2457107238 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe58402ad20 con 0x7fe578003a00 2024-05-09T04:34:39.328 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.326+0000 7fe58fe00640 1 -- 172.21.15.145:0/2457107238 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe584004250 con 0x7fe578003a00 2024-05-09T04:34:39.328 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.327+0000 7fe598600640 1 -- 172.21.15.145:0/2457107238 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe578003a00 msgr2=0x7fe578003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:39.328 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.327+0000 7fe598600640 1 --2- 172.21.15.145:0/2457107238 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe578003a00 0x7fe578003e20 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe5840098a0 tx=0x7fe578005830 comp rx=0 tx=0).stop 2024-05-09T04:34:39.329 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.327+0000 7fe596600640 1 -- 172.21.15.145:0/2457107238 reap_dead start 2024-05-09T04:34:39.329 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.327+0000 7fe598600640 1 -- 172.21.15.145:0/2457107238 shutdown_connections 2024-05-09T04:34:39.329 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.327+0000 7fe598600640 1 -- 172.21.15.145:0/2457107238 >> 172.21.15.145:0/2457107238 conn(0x7fe59006c8d0 msgr2=0x7fe59006cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:39.329 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.327+0000 7fe598600640 1 -- 172.21.15.145:0/2457107238 shutdown_connections 2024-05-09T04:34:39.329 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.327+0000 7fe598600640 1 -- 172.21.15.145:0/2457107238 wait complete. 2024-05-09T04:34:39.331 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.329+0000 7fe598600640 1 Processor -- start 2024-05-09T04:34:39.331 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.329+0000 7fe598600640 1 -- start start 2024-05-09T04:34:39.331 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.330+0000 7fe598600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe578003a00 0x7fe59013f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:39.331 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.330+0000 7fe598600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe5901f5250 0x7fe5900cd050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:39.331 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.330+0000 7fe598600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5900c65a0 0x7fe5901f8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:39.331 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.330+0000 7fe598600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe590140850 con 0x7fe5901f5250 2024-05-09T04:34:39.331 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.330+0000 7fe598600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe59015dfc0 con 0x7fe5900c65a0 2024-05-09T04:34:39.332 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.330+0000 7fe598600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe590140450 con 0x7fe578003a00 2024-05-09T04:34:39.332 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.330+0000 7fe595200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe5901f5250 0x7fe5900cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:39.332 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.330+0000 7fe595200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe5901f5250 0x7fe5900cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:51238/0 (socket says 172.21.15.145:51238) 2024-05-09T04:34:39.332 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.330+0000 7fe595200640 1 -- 172.21.15.145:0/1986593426 learned_addr learned my addr 172.21.15.145:0/1986593426 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:39.332 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.331+0000 7fe595200640 1 -- 172.21.15.145:0/1986593426 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe578003a00 msgr2=0x7fe59013f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:39.332 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.331+0000 7fe595200640 1 --2- 172.21.15.145:0/1986593426 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe578003a00 0x7fe59013f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:39.332 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.331+0000 7fe595200640 1 -- 172.21.15.145:0/1986593426 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5900c65a0 msgr2=0x7fe5901f8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:39.332 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.331+0000 7fe595200640 1 --2- 172.21.15.145:0/1986593426 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5900c65a0 0x7fe5901f8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:39.332 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.331+0000 7fe595200640 1 -- 172.21.15.145:0/1986593426 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5900c6e10 con 0x7fe5901f5250 2024-05-09T04:34:39.332 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.331+0000 7fe595200640 1 --2- 172.21.15.145:0/1986593426 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe5901f5250 0x7fe5900cd050 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe57c00c760 tx=0x7fe57c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:39.332 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.331+0000 7fe58d600640 1 -- 172.21.15.145:0/1986593426 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57c0074e0 con 0x7fe5901f5250 2024-05-09T04:34:39.332 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.331+0000 7fe58d600640 1 -- 172.21.15.145:0/1986593426 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe57c00ce10 con 0x7fe5901f5250 2024-05-09T04:34:39.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.331+0000 7fe58d600640 1 -- 172.21.15.145:0/1986593426 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57c0049e0 con 0x7fe5901f5250 2024-05-09T04:34:39.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.331+0000 7fe598600640 1 -- 172.21.15.145:0/1986593426 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5901f56d0 con 0x7fe5901f5250 2024-05-09T04:34:39.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.331+0000 7fe598600640 1 -- 172.21.15.145:0/1986593426 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5901f5c80 con 0x7fe5901f5250 2024-05-09T04:34:39.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.333+0000 7fe598600640 1 -- 172.21.15.145:0/1986593426 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe59006a310 con 0x7fe5901f5250 2024-05-09T04:34:39.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.334+0000 7fe58d600640 1 -- 172.21.15.145:0/1986593426 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe57c011070 con 0x7fe5901f5250 2024-05-09T04:34:39.334 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.335+0000 7fe58d600640 1 --2- 172.21.15.145:0/1986593426 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fe570076300 0x7fe5700787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:39.334 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.335+0000 7fe58d600640 1 -- 172.21.15.145:0/1986593426 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7fe57c096e30 con 0x7fe5901f5250 2024-05-09T04:34:39.335 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.335+0000 7fe595c00640 1 --2- 172.21.15.145:0/1986593426 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fe570076300 0x7fe5700787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:39.335 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.336+0000 7fe595c00640 1 --2- 172.21.15.145:0/1986593426 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fe570076300 0x7fe5700787c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe584033000 tx=0x7fe58402c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:39.339 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.339+0000 7fe58d600640 1 -- 172.21.15.145:0/1986593426 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe59015d780 con 0x7fe5901f5250 2024-05-09T04:34:39.621 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.621+0000 7fe598600640 1 -- 172.21.15.145:0/1986593426 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7fe5900b40d0 con 0x7fe5901f5250 2024-05-09T04:34:39.626 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.626+0000 7fe58d600640 1 -- 172.21.15.145:0/1986593426 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) ==== 97+0+0 (secure 0 0 0) 0x7fe57c060ad0 con 0x7fe5901f5250 2024-05-09T04:34:39.631 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.630+0000 7fe598600640 1 -- 172.21.15.145:0/1986593426 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fe570076300 msgr2=0x7fe5700787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:39.631 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.630+0000 7fe598600640 1 --2- 172.21.15.145:0/1986593426 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fe570076300 0x7fe5700787c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe584033000 tx=0x7fe58402c000 comp rx=0 tx=0).stop 2024-05-09T04:34:39.631 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.630+0000 7fe598600640 1 -- 172.21.15.145:0/1986593426 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe5901f5250 msgr2=0x7fe5900cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:39.632 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.630+0000 7fe598600640 1 --2- 172.21.15.145:0/1986593426 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe5901f5250 0x7fe5900cd050 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe57c00c760 tx=0x7fe57c00cc20 comp rx=0 tx=0).stop 2024-05-09T04:34:39.632 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.631+0000 7fe598600640 1 -- 172.21.15.145:0/1986593426 shutdown_connections 2024-05-09T04:34:39.632 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.631+0000 7fe598600640 1 --2- 172.21.15.145:0/1986593426 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fe570076300 0x7fe5700787c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:39.632 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.631+0000 7fe598600640 1 --2- 172.21.15.145:0/1986593426 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5900c65a0 0x7fe5901f8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:39.632 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.631+0000 7fe598600640 1 --2- 172.21.15.145:0/1986593426 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe5901f5250 0x7fe5900cd050 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:39.632 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.631+0000 7fe598600640 1 --2- 172.21.15.145:0/1986593426 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe578003a00 0x7fe59013f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:39.632 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.631+0000 7fe598600640 1 -- 172.21.15.145:0/1986593426 >> 172.21.15.145:0/1986593426 conn(0x7fe59006c8d0 msgr2=0x7fe59015e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:39.632 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.631+0000 7fe598600640 1 -- 172.21.15.145:0/1986593426 shutdown_connections 2024-05-09T04:34:39.632 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.631+0000 7fe598600640 1 -- 172.21.15.145:0/1986593426 wait complete. 2024-05-09T04:34:39.648 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-05-09T04:34:39.839 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.839+0000 7f36d8a00640 1 Processor -- start 2024-05-09T04:34:39.839 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.839+0000 7f36d8a00640 1 -- start start 2024-05-09T04:34:39.840 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.841+0000 7f36d8a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36d4071520 0x7f36d415fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:39.840 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.841+0000 7f36d8a00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36d4163d00 con 0x7f36d40b4de0 2024-05-09T04:34:39.840 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.841+0000 7f36d8a00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36d4162f40 con 0x7f36d406c480 2024-05-09T04:34:39.841 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.841+0000 7f36d8a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f36d40b5270 con 0x7f36d4071520 2024-05-09T04:34:39.841 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.841+0000 7f36cbe00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f36d40b4de0 0x7f36d415f640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:59156/0 (socket says 172.21.15.145:59156) 2024-05-09T04:34:39.841 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.841+0000 7f36cbe00640 1 -- 172.21.15.145:0/2928602825 learned_addr learned my addr 172.21.15.145:0/2928602825 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:34:39.842 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.841+0000 7f36d3200640 1 --2- 172.21.15.145:0/2928602825 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36d4071520 0x7f36d415fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:39.843 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.841+0000 7f36d1e00640 1 -- 172.21.15.145:0/2928602825 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3079315183 0 0) 0x7f36d4163d00 con 0x7f36d40b4de0 2024-05-09T04:34:39.843 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.841+0000 7f36d1e00640 1 -- 172.21.15.145:0/2928602825 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f36b8003520 con 0x7f36d40b4de0 2024-05-09T04:34:39.843 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.842+0000 7f36d1e00640 1 -- 172.21.15.145:0/2928602825 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3473916119 0 0) 0x7f36b8003520 con 0x7f36d40b4de0 2024-05-09T04:34:39.843 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.842+0000 7f36d1e00640 1 -- 172.21.15.145:0/2928602825 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36d4071520 msgr2=0x7f36d415fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:39.843 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.842+0000 7f36d1e00640 1 --2- 172.21.15.145:0/2928602825 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36d4071520 0x7f36d415fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:39.843 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.842+0000 7f36d1e00640 1 -- 172.21.15.145:0/2928602825 >> v1:172.21.15.192:6789/0 conn(0x7f36d406c480 legacy=0x7f36d406c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:39.843 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.842+0000 7f36d1e00640 1 -- 172.21.15.145:0/2928602825 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36d4163220 con 0x7f36d40b4de0 2024-05-09T04:34:39.843 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.842+0000 7f36d1e00640 1 -- 172.21.15.145:0/2928602825 <== mon.0 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3474413261 0 0) 0x7f36bc0016f0 con 0x7f36d40b4de0 2024-05-09T04:34:39.843 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.842+0000 7f36d1e00640 1 -- 172.21.15.145:0/2928602825 >> v1:172.21.15.145:6789/0 conn(0x7f36d40b4de0 legacy=0x7f36d415f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:39.843 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.842+0000 7f36d1e00640 1 --2- 172.21.15.145:0/2928602825 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36b8003a80 0x7f36b8003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:39.843 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.842+0000 7f36d1e00640 1 --2- 172.21.15.145:0/2928602825 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36b8004800 0x7f36b8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:39.843 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.842+0000 7f36d1e00640 1 --2- 172.21.15.145:0/2928602825 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36d4071520 0x7f36b8005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:39.844 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.842+0000 7f36d1e00640 1 -- 172.21.15.145:0/2928602825 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f36d40b5270 con 0x7f36b8003a80 2024-05-09T04:34:39.844 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.842+0000 7f36d1e00640 1 -- 172.21.15.145:0/2928602825 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f36b800bb80 con 0x7f36b8004800 2024-05-09T04:34:39.844 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.842+0000 7f36d1e00640 1 -- 172.21.15.145:0/2928602825 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f36b800bd80 con 0x7f36d4071520 2024-05-09T04:34:39.844 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.842+0000 7f36d2800640 1 --2- 172.21.15.145:0/2928602825 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36b8003a80 0x7f36b8003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:39.844 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.842+0000 7f36d2800640 1 -- 172.21.15.145:0/2928602825 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36d4071520 msgr2=0x7f36b8005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:39.844 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.842+0000 7f36d2800640 1 --2- 172.21.15.145:0/2928602825 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36d4071520 0x7f36b8005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:39.844 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.842+0000 7f36d2800640 1 -- 172.21.15.145:0/2928602825 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36b8004800 msgr2=0x7f36b8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:39.844 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.842+0000 7f36d2800640 1 --2- 172.21.15.145:0/2928602825 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36b8004800 0x7f36b8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:39.844 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.842+0000 7f36d2800640 1 -- 172.21.15.145:0/2928602825 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36b8003520 con 0x7f36b8003a80 2024-05-09T04:34:39.846 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.843+0000 7f36d2800640 1 --2- 172.21.15.145:0/2928602825 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36b8003a80 0x7f36b8003ea0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f36c4001730 tx=0x7f36b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=536e51de11c9a059 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:39.846 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.844+0000 7f36d1e00640 1 -- 172.21.15.145:0/2928602825 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c4001350 con 0x7f36b8003a80 2024-05-09T04:34:39.846 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.844+0000 7f36d1e00640 1 -- 172.21.15.145:0/2928602825 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36c40052d0 con 0x7f36b8003a80 2024-05-09T04:34:39.846 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.844+0000 7f36d1e00640 1 -- 172.21.15.145:0/2928602825 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c402aac0 con 0x7f36b8003a80 2024-05-09T04:34:39.846 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.844+0000 7f36d8a00640 1 -- 172.21.15.145:0/2928602825 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36b8003a80 msgr2=0x7f36b8003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:39.846 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.844+0000 7f36d8a00640 1 --2- 172.21.15.145:0/2928602825 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36b8003a80 0x7f36b8003ea0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f36c4001730 tx=0x7f36b8005830 comp rx=0 tx=0).stop 2024-05-09T04:34:39.846 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.844+0000 7f36d3200640 1 -- 172.21.15.145:0/2928602825 reap_dead start 2024-05-09T04:34:39.846 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.844+0000 7f36d8a00640 1 -- 172.21.15.145:0/2928602825 shutdown_connections 2024-05-09T04:34:39.846 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.844+0000 7f36d8a00640 1 -- 172.21.15.145:0/2928602825 >> 172.21.15.145:0/2928602825 conn(0x7f36d40ce830 msgr2=0x7f36d40cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:39.846 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.844+0000 7f36d8a00640 1 -- 172.21.15.145:0/2928602825 shutdown_connections 2024-05-09T04:34:39.846 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.844+0000 7f36d8a00640 1 -- 172.21.15.145:0/2928602825 wait complete. 2024-05-09T04:34:39.846 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.846+0000 7f36d8a00640 1 Processor -- start 2024-05-09T04:34:39.847 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.846+0000 7f36d8a00640 1 -- start start 2024-05-09T04:34:39.848 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.847+0000 7f36d8a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36b8003a80 0x7f36d40c6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:39.848 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.847+0000 7f36d8a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36d41f49a0 0x7f36d41f4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:39.848 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.847+0000 7f36d8a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36d40b15e0 0x7f36d40b1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:39.849 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.847+0000 7f36d8a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f36d406ff90 con 0x7f36d41f49a0 2024-05-09T04:34:39.849 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.847+0000 7f36d8a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f36d41637e0 con 0x7f36b8003a80 2024-05-09T04:34:39.849 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.847+0000 7f36d8a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f36d40cf560 con 0x7f36d40b15e0 2024-05-09T04:34:39.849 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.847+0000 7f36cbe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36d41f49a0 0x7f36d41f4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:39.849 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.847+0000 7f36cbe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36d41f49a0 0x7f36d41f4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:51260/0 (socket says 172.21.15.145:51260) 2024-05-09T04:34:39.849 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.847+0000 7f36cbe00640 1 -- 172.21.15.145:0/1356732017 learned_addr learned my addr 172.21.15.145:0/1356732017 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:39.849 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.848+0000 7f36d2800640 1 --2- 172.21.15.145:0/1356732017 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36b8003a80 0x7f36d40c6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:39.849 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.848+0000 7f36cbe00640 1 -- 172.21.15.145:0/1356732017 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36d40b15e0 msgr2=0x7f36d40b1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:39.849 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.848+0000 7f36d3200640 1 --2- 172.21.15.145:0/1356732017 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36d40b15e0 0x7f36d40b1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:39.849 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.848+0000 7f36cbe00640 1 --2- 172.21.15.145:0/1356732017 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36d40b15e0 0x7f36d40b1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:39.849 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.848+0000 7f36cbe00640 1 -- 172.21.15.145:0/1356732017 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36b8003a80 msgr2=0x7f36d40c6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:39.849 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.848+0000 7f36cbe00640 1 --2- 172.21.15.145:0/1356732017 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36b8003a80 0x7f36d40c6fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:39.849 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.848+0000 7f36cbe00640 1 -- 172.21.15.145:0/1356732017 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36d41f5360 con 0x7f36d41f49a0 2024-05-09T04:34:39.849 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.848+0000 7f36cbe00640 1 --2- 172.21.15.145:0/1356732017 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36d41f49a0 0x7f36d41f4e20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f36bc00c800 tx=0x7f36bc00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:39.849 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.848+0000 7f36caa00640 1 -- 172.21.15.145:0/1356732017 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36bc0074e0 con 0x7f36d41f49a0 2024-05-09T04:34:39.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.848+0000 7f36caa00640 1 -- 172.21.15.145:0/1356732017 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36bc0047e0 con 0x7f36d41f49a0 2024-05-09T04:34:39.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.848+0000 7f36caa00640 1 -- 172.21.15.145:0/1356732017 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36bc004bd0 con 0x7f36d41f49a0 2024-05-09T04:34:39.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.848+0000 7f36d8a00640 1 -- 172.21.15.145:0/1356732017 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36d40b2310 con 0x7f36d41f49a0 2024-05-09T04:34:39.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.849+0000 7f36d8a00640 1 -- 172.21.15.145:0/1356732017 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36d40b28d0 con 0x7f36d41f49a0 2024-05-09T04:34:39.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.850+0000 7f36d8a00640 1 -- 172.21.15.145:0/1356732017 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36d4076780 con 0x7f36d41f49a0 2024-05-09T04:34:39.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.851+0000 7f36caa00640 1 -- 172.21.15.145:0/1356732017 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f36bc011070 con 0x7f36d41f49a0 2024-05-09T04:34:39.851 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.852+0000 7f36caa00640 1 --2- 172.21.15.145:0/1356732017 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f36b00762d0 0x7f36b0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:39.851 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.852+0000 7f36caa00640 1 -- 172.21.15.145:0/1356732017 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f36bc097070 con 0x7f36d41f49a0 2024-05-09T04:34:39.852 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.852+0000 7f36d2800640 1 --2- 172.21.15.145:0/1356732017 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f36b00762d0 0x7f36b0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:39.852 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.853+0000 7f36d2800640 1 --2- 172.21.15.145:0/1356732017 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f36b00762d0 0x7f36b0078790 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f36c4032000 tx=0x7f36c402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:39.856 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:39.856+0000 7f36caa00640 1 -- 172.21.15.145:0/1356732017 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f36bc060c80 con 0x7f36d41f49a0 2024-05-09T04:34:40.115 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.116+0000 7f36d8a00640 1 -- 172.21.15.145:0/1356732017 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f36d406cfb0 con 0x7f36d41f49a0 2024-05-09T04:34:40.121 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.121+0000 7f36caa00640 1 -- 172.21.15.145:0/1356732017 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7f36bc060580 con 0x7f36d41f49a0 2024-05-09T04:34:40.124 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.123+0000 7f36d8a00640 1 -- 172.21.15.145:0/1356732017 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f36b00762d0 msgr2=0x7f36b0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:40.124 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.123+0000 7f36d8a00640 1 --2- 172.21.15.145:0/1356732017 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f36b00762d0 0x7f36b0078790 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f36c4032000 tx=0x7f36c402e000 comp rx=0 tx=0).stop 2024-05-09T04:34:40.124 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.124+0000 7f36d8a00640 1 -- 172.21.15.145:0/1356732017 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36d41f49a0 msgr2=0x7f36d41f4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:40.124 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.124+0000 7f36d8a00640 1 --2- 172.21.15.145:0/1356732017 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36d41f49a0 0x7f36d41f4e20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f36bc00c800 tx=0x7f36bc00ccc0 comp rx=0 tx=0).stop 2024-05-09T04:34:40.124 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.124+0000 7f36d8a00640 1 -- 172.21.15.145:0/1356732017 shutdown_connections 2024-05-09T04:34:40.124 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.124+0000 7f36d8a00640 1 --2- 172.21.15.145:0/1356732017 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f36b00762d0 0x7f36b0078790 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:40.124 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.124+0000 7f36d8a00640 1 --2- 172.21.15.145:0/1356732017 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36d40b15e0 0x7f36d40b1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:40.124 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.124+0000 7f36d8a00640 1 --2- 172.21.15.145:0/1356732017 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36d41f49a0 0x7f36d41f4e20 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:40.124 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.124+0000 7f36d8a00640 1 --2- 172.21.15.145:0/1356732017 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36b8003a80 0x7f36d40c6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:40.124 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.124+0000 7f36d8a00640 1 -- 172.21.15.145:0/1356732017 >> 172.21.15.145:0/1356732017 conn(0x7f36d40ce830 msgr2=0x7f36d40d05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:40.124 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.124+0000 7f36d8a00640 1 -- 172.21.15.145:0/1356732017 shutdown_connections 2024-05-09T04:34:40.124 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.124+0000 7f36d8a00640 1 -- 172.21.15.145:0/1356732017 wait complete. 2024-05-09T04:34:40.142 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-09T04:34:40.339 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.339+0000 7f20e3000640 1 Processor -- start 2024-05-09T04:34:40.339 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.339+0000 7f20e3000640 1 -- start start 2024-05-09T04:34:40.340 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.341+0000 7f20e3000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f20dc15bac0 0x7f20dc0b4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:40.341 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.341+0000 7f20e3000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f20dc0738c0 0x7f20dc0cf940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:40.341 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.341+0000 7f20e3000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20dc0b5af0 con 0x7f20dc06f0f0 2024-05-09T04:34:40.341 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.341+0000 7f20e3000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f20dc0cf4b0 con 0x7f20dc15bac0 2024-05-09T04:34:40.341 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.341+0000 7f20e3000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f20dc0b4ec0 con 0x7f20dc0738c0 2024-05-09T04:34:40.342 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.341+0000 7f20dbe00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f20dc06f0f0 0x7f20dc06f4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:59158/0 (socket says 172.21.15.145:59158) 2024-05-09T04:34:40.342 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.341+0000 7f20dbe00640 1 -- 172.21.15.145:0/992790295 learned_addr learned my addr 172.21.15.145:0/992790295 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:34:40.342 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.341+0000 7f20db400640 1 --2- 172.21.15.145:0/992790295 >> v2:172.21.15.145:3300/0 conn(0x7f20dc15bac0 0x7f20dc0b4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:40.342 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.342+0000 7f20e1000640 1 --2- 172.21.15.145:0/992790295 >> v2:172.21.15.192:3300/0 conn(0x7f20dc0738c0 0x7f20dc0cf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:40.342 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.342+0000 7f20db400640 1 -- 172.21.15.145:0/992790295 >> v2:172.21.15.192:3300/0 conn(0x7f20dc0738c0 msgr2=0x7f20dc0cf940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:40.342 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.342+0000 7f20db400640 1 --2- 172.21.15.145:0/992790295 >> v2:172.21.15.192:3300/0 conn(0x7f20dc0738c0 0x7f20dc0cf940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:40.342 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.342+0000 7f20db400640 1 -- 172.21.15.145:0/992790295 >> v1:172.21.15.145:6789/0 conn(0x7f20dc06f0f0 legacy=0x7f20dc06f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:40.342 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.342+0000 7f20db400640 1 -- 172.21.15.145:0/992790295 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20dc0b5dd0 con 0x7f20dc15bac0 2024-05-09T04:34:40.344 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.343+0000 7f20db400640 1 --2- 172.21.15.145:0/992790295 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f20dc15bac0 0x7f20dc0b4980 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f20dc0b0b90 tx=0x7f20c802b2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8568954c94a3a200 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:40.344 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.343+0000 7f20daa00640 1 -- 172.21.15.145:0/992790295 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c8038070 con 0x7f20dc15bac0 2024-05-09T04:34:40.344 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.343+0000 7f20daa00640 1 -- 172.21.15.145:0/992790295 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20c8033440 con 0x7f20dc15bac0 2024-05-09T04:34:40.344 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.343+0000 7f20daa00640 1 -- 172.21.15.145:0/992790295 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c8033810 con 0x7f20dc15bac0 2024-05-09T04:34:40.344 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.344+0000 7f20e3000640 1 -- 172.21.15.145:0/992790295 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f20dc15bac0 msgr2=0x7f20dc0b4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:40.344 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.344+0000 7f20e3000640 1 --2- 172.21.15.145:0/992790295 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f20dc15bac0 0x7f20dc0b4980 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f20dc0b0b90 tx=0x7f20c802b2b0 comp rx=0 tx=0).stop 2024-05-09T04:34:40.344 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.344+0000 7f20e3000640 1 -- 172.21.15.145:0/992790295 shutdown_connections 2024-05-09T04:34:40.345 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.344+0000 7f20e3000640 1 --2- 172.21.15.145:0/992790295 >> v2:172.21.15.192:3300/0 conn(0x7f20dc0738c0 0x7f20dc0cf940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:40.345 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.344+0000 7f20e3000640 1 --2- 172.21.15.145:0/992790295 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f20dc15bac0 0x7f20dc0b4980 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:40.345 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.344+0000 7f20e3000640 1 -- 172.21.15.145:0/992790295 >> 172.21.15.145:0/992790295 conn(0x7f20dc06c870 msgr2=0x7f20dc06cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:40.345 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.344+0000 7f20e3000640 1 -- 172.21.15.145:0/992790295 shutdown_connections 2024-05-09T04:34:40.345 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.344+0000 7f20e3000640 1 -- 172.21.15.145:0/992790295 wait complete. 2024-05-09T04:34:40.345 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.346+0000 7f20e3000640 1 Processor -- start 2024-05-09T04:34:40.346 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.346+0000 7f20e3000640 1 -- start start 2024-05-09T04:34:40.347 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.347+0000 7f20e3000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f20dc06f0f0 0x7f20dc153bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:40.347 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.347+0000 7f20e3000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20dc0738c0 0x7f20dc14e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:40.347 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.347+0000 7f20e3000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f20dc15bac0 0x7f20dc14db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:40.347 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.347+0000 7f20e3000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f20dc0cf140 con 0x7f20dc15bac0 2024-05-09T04:34:40.348 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.347+0000 7f20e3000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f20dc0b6390 con 0x7f20dc0738c0 2024-05-09T04:34:40.348 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.347+0000 7f20e3000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f20dc06e860 con 0x7f20dc06f0f0 2024-05-09T04:34:40.348 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.347+0000 7f20e1000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f20dc15bac0 0x7f20dc14db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:40.348 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.347+0000 7f20e1000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f20dc15bac0 0x7f20dc14db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:51280/0 (socket says 172.21.15.145:51280) 2024-05-09T04:34:40.348 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.347+0000 7f20e1000640 1 -- 172.21.15.145:0/3717783914 learned_addr learned my addr 172.21.15.145:0/3717783914 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:40.348 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.347+0000 7f20e1000640 1 -- 172.21.15.145:0/3717783914 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f20dc06f0f0 msgr2=0x7f20dc153bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:40.348 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.347+0000 7f20e1000640 1 --2- 172.21.15.145:0/3717783914 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f20dc06f0f0 0x7f20dc153bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:40.348 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.347+0000 7f20db400640 1 --2- 172.21.15.145:0/3717783914 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20dc0738c0 0x7f20dc14e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:40.348 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.347+0000 7f20e1000640 1 -- 172.21.15.145:0/3717783914 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20dc0738c0 msgr2=0x7f20dc14e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:40.348 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.347+0000 7f20e1000640 1 --2- 172.21.15.145:0/3717783914 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20dc0738c0 0x7f20dc14e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:40.348 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.348+0000 7f20e1000640 1 -- 172.21.15.145:0/3717783914 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20dc151ae0 con 0x7f20dc15bac0 2024-05-09T04:34:40.348 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.348+0000 7f20e1000640 1 --2- 172.21.15.145:0/3717783914 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f20dc15bac0 0x7f20dc14db70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f20d4007de0 tx=0x7f20d400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:40.349 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.348+0000 7f20d3e00640 1 -- 172.21.15.145:0/3717783914 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d4014620 con 0x7f20dc15bac0 2024-05-09T04:34:40.349 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.348+0000 7f20d3e00640 1 -- 172.21.15.145:0/3717783914 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20d4019070 con 0x7f20dc15bac0 2024-05-09T04:34:40.349 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.348+0000 7f20d3e00640 1 -- 172.21.15.145:0/3717783914 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d4014a20 con 0x7f20dc15bac0 2024-05-09T04:34:40.349 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.348+0000 7f20e3000640 1 -- 172.21.15.145:0/3717783914 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20dc1530e0 con 0x7f20dc15bac0 2024-05-09T04:34:40.349 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.348+0000 7f20e3000640 1 -- 172.21.15.145:0/3717783914 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20dc1536a0 con 0x7f20dc15bac0 2024-05-09T04:34:40.350 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.349+0000 7f20e3000640 1 -- 172.21.15.145:0/3717783914 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20dc06a310 con 0x7f20dc15bac0 2024-05-09T04:34:40.350 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.350+0000 7f20d3e00640 1 -- 172.21.15.145:0/3717783914 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f20d4014bc0 con 0x7f20dc15bac0 2024-05-09T04:34:40.351 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.351+0000 7f20d3e00640 1 --2- 172.21.15.145:0/3717783914 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f20bc0762d0 0x7f20bc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:40.351 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.351+0000 7f20d3e00640 1 -- 172.21.15.145:0/3717783914 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f20d4097cd0 con 0x7f20dc15bac0 2024-05-09T04:34:40.352 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.352+0000 7f20dbe00640 1 --2- 172.21.15.145:0/3717783914 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f20bc0762d0 0x7f20bc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:40.352 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.352+0000 7f20dbe00640 1 --2- 172.21.15.145:0/3717783914 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f20bc0762d0 0x7f20bc078790 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f20cc0015e0 tx=0x7f20cc007a40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:40.356 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.356+0000 7f20d3e00640 1 -- 172.21.15.145:0/3717783914 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f20d4061970 con 0x7f20dc15bac0 2024-05-09T04:34:40.550 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.550+0000 7f20e3000640 1 -- 172.21.15.145:0/3717783914 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f20dc0b29b0 con 0x7f20dc15bac0 2024-05-09T04:34:40.551 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.551+0000 7f20d3e00640 1 -- 172.21.15.145:0/3717783914 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v28) ==== 84+0+65 (secure 0 0 0) 0x7f20d4061270 con 0x7f20dc15bac0 2024-05-09T04:34:40.551 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:34:40.551 INFO:teuthology.orchestra.run.smithi145.stdout:[ 2024-05-09T04:34:40.551 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-05-09T04:34:40.551 INFO:teuthology.orchestra.run.smithi145.stdout: "poolnum": 1, 2024-05-09T04:34:40.551 INFO:teuthology.orchestra.run.smithi145.stdout: "poolname": ".mgr" 2024-05-09T04:34:40.552 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-05-09T04:34:40.552 INFO:teuthology.orchestra.run.smithi145.stdout:] 2024-05-09T04:34:40.554 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.553+0000 7f20e3000640 1 -- 172.21.15.145:0/3717783914 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f20bc0762d0 msgr2=0x7f20bc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:40.554 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.553+0000 7f20e3000640 1 --2- 172.21.15.145:0/3717783914 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f20bc0762d0 0x7f20bc078790 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f20cc0015e0 tx=0x7f20cc007a40 comp rx=0 tx=0).stop 2024-05-09T04:34:40.554 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.553+0000 7f20e3000640 1 -- 172.21.15.145:0/3717783914 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f20dc15bac0 msgr2=0x7f20dc14db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:40.554 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.553+0000 7f20e3000640 1 --2- 172.21.15.145:0/3717783914 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f20dc15bac0 0x7f20dc14db70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f20d4007de0 tx=0x7f20d400e4b0 comp rx=0 tx=0).stop 2024-05-09T04:34:40.554 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.554+0000 7f20e3000640 1 -- 172.21.15.145:0/3717783914 shutdown_connections 2024-05-09T04:34:40.554 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.554+0000 7f20e3000640 1 --2- 172.21.15.145:0/3717783914 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f20bc0762d0 0x7f20bc078790 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:40.554 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.554+0000 7f20e3000640 1 --2- 172.21.15.145:0/3717783914 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f20dc15bac0 0x7f20dc14db70 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:40.555 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.554+0000 7f20e3000640 1 --2- 172.21.15.145:0/3717783914 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20dc0738c0 0x7f20dc14e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:40.555 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.554+0000 7f20e3000640 1 --2- 172.21.15.145:0/3717783914 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f20dc06f0f0 0x7f20dc153bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:40.555 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.554+0000 7f20e3000640 1 -- 172.21.15.145:0/3717783914 >> 172.21.15.145:0/3717783914 conn(0x7f20dc06c870 msgr2=0x7f20dc15a330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:40.555 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.554+0000 7f20e3000640 1 -- 172.21.15.145:0/3717783914 shutdown_connections 2024-05-09T04:34:40.555 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.554+0000 7f20e3000640 1 -- 172.21.15.145:0/3717783914 wait complete. 2024-05-09T04:34:40.572 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-09T04:34:40.573 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-09T04:34:40.766 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.766+0000 7fcc8b200640 1 Processor -- start 2024-05-09T04:34:40.766 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.766+0000 7fcc8b200640 1 -- start start 2024-05-09T04:34:40.769 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.768+0000 7fcc8b200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fcc8c146970 0x7fcc8c0cea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:40.769 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.768+0000 7fcc8b200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc8c06e060 0x7fcc8c15d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:40.769 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.768+0000 7fcc8b200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fcc8c0cd550 0x7fcc8c0cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:40.769 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.768+0000 7fcc8b200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fcc8c072ee0 con 0x7fcc8c0cd550 2024-05-09T04:34:40.769 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.768+0000 7fcc8b200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fcc8c15dc10 con 0x7fcc8c146970 2024-05-09T04:34:40.769 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.768+0000 7fcc8b200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fcc8c15def0 con 0x7fcc8c06e060 2024-05-09T04:34:40.769 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.768+0000 7fcc8a800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fcc8c0cd550 0x7fcc8c0cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:40.769 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.768+0000 7fcc8a800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fcc8c0cd550 0x7fcc8c0cd9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:51296/0 (socket says 172.21.15.145:51296) 2024-05-09T04:34:40.769 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.768+0000 7fcc8a800640 1 -- 172.21.15.145:0/3600880002 learned_addr learned my addr 172.21.15.145:0/3600880002 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:40.770 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.769+0000 7fcc89e00640 1 --2- 172.21.15.145:0/3600880002 >> v2:172.21.15.192:3300/0 conn(0x7fcc8c146970 0x7fcc8c0cea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:40.770 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.769+0000 7fcc8a800640 1 -- 172.21.15.145:0/3600880002 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc8c06e060 msgr2=0x7fcc8c15d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:40.770 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.769+0000 7fcc83e00640 1 --2- 172.21.15.145:0/3600880002 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc8c06e060 0x7fcc8c15d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:40.770 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.769+0000 7fcc8a800640 1 --2- 172.21.15.145:0/3600880002 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc8c06e060 0x7fcc8c15d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:40.770 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.769+0000 7fcc8a800640 1 -- 172.21.15.145:0/3600880002 >> v2:172.21.15.192:3300/0 conn(0x7fcc8c146970 msgr2=0x7fcc8c0cea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:40.770 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.769+0000 7fcc8a800640 1 --2- 172.21.15.145:0/3600880002 >> v2:172.21.15.192:3300/0 conn(0x7fcc8c146970 0x7fcc8c0cea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:40.770 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.769+0000 7fcc8a800640 1 -- 172.21.15.145:0/3600880002 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc8c15e1d0 con 0x7fcc8c0cd550 2024-05-09T04:34:40.770 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.769+0000 7fcc89e00640 1 --2- 172.21.15.145:0/3600880002 >> v2:172.21.15.192:3300/0 conn(0x7fcc8c146970 0x7fcc8c0cea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:34:40.770 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.769+0000 7fcc83e00640 1 --2- 172.21.15.145:0/3600880002 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc8c06e060 0x7fcc8c15d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:34:40.772 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.770+0000 7fcc8a800640 1 --2- 172.21.15.145:0/3600880002 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc8c0cd550 0x7fcc8c0cd9e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c06d4d0 tx=0x7fcc8402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23ec91f4f11337f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:40.772 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.770+0000 7fcc89400640 1 -- 172.21.15.145:0/3600880002 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc84036070 con 0x7fcc8c0cd550 2024-05-09T04:34:40.772 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.770+0000 7fcc89400640 1 -- 172.21.15.145:0/3600880002 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc84031600 con 0x7fcc8c0cd550 2024-05-09T04:34:40.772 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.770+0000 7fcc89400640 1 -- 172.21.15.145:0/3600880002 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc840319d0 con 0x7fcc8c0cd550 2024-05-09T04:34:40.772 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.771+0000 7fcc8b200640 1 -- 172.21.15.145:0/3600880002 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc8c0cd550 msgr2=0x7fcc8c0cd9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:40.773 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.771+0000 7fcc8b200640 1 --2- 172.21.15.145:0/3600880002 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc8c0cd550 0x7fcc8c0cd9e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c06d4d0 tx=0x7fcc8402a6a0 comp rx=0 tx=0).stop 2024-05-09T04:34:40.773 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.771+0000 7fcc8b200640 1 -- 172.21.15.145:0/3600880002 shutdown_connections 2024-05-09T04:34:40.773 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.771+0000 7fcc8b200640 1 --2- 172.21.15.145:0/3600880002 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc8c0cd550 0x7fcc8c0cd9e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:40.773 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.771+0000 7fcc8b200640 1 --2- 172.21.15.145:0/3600880002 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc8c06e060 0x7fcc8c15d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:40.773 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.771+0000 7fcc8b200640 1 --2- 172.21.15.145:0/3600880002 >> v2:172.21.15.192:3300/0 conn(0x7fcc8c146970 0x7fcc8c0cea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:40.773 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.771+0000 7fcc8b200640 1 -- 172.21.15.145:0/3600880002 >> 172.21.15.145:0/3600880002 conn(0x7fcc8c0b2850 msgr2=0x7fcc8c0b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:40.773 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.771+0000 7fcc8b200640 1 -- 172.21.15.145:0/3600880002 shutdown_connections 2024-05-09T04:34:40.773 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.771+0000 7fcc8b200640 1 -- 172.21.15.145:0/3600880002 wait complete. 2024-05-09T04:34:40.773 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.772+0000 7fcc8b200640 1 Processor -- start 2024-05-09T04:34:40.773 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.773+0000 7fcc8b200640 1 -- start start 2024-05-09T04:34:40.778 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.773+0000 7fcc8b200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc8c06e060 0x7fcc8c0c5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:40.778 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.774+0000 7fcc8b200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc8c0cd550 0x7fcc8c0af040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:40.779 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.774+0000 7fcc8b200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc8c146970 0x7fcc8c0af580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:40.779 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.774+0000 7fcc8b200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fcc8c0ce7c0 con 0x7fcc8c06e060 2024-05-09T04:34:40.779 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.774+0000 7fcc8b200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fcc8c0ce640 con 0x7fcc8c0cd550 2024-05-09T04:34:40.779 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.774+0000 7fcc8b200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fcc8c073370 con 0x7fcc8c146970 2024-05-09T04:34:40.779 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.774+0000 7fcc89e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc8c06e060 0x7fcc8c0c5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:40.779 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.774+0000 7fcc89e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc8c06e060 0x7fcc8c0c5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:51306/0 (socket says 172.21.15.145:51306) 2024-05-09T04:34:40.780 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.774+0000 7fcc89e00640 1 -- 172.21.15.145:0/1053012676 learned_addr learned my addr 172.21.15.145:0/1053012676 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:40.780 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.774+0000 7fcc89e00640 1 -- 172.21.15.145:0/1053012676 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc8c146970 msgr2=0x7fcc8c0af580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:40.780 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.774+0000 7fcc83e00640 1 --2- 172.21.15.145:0/1053012676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc8c0cd550 0x7fcc8c0af040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:40.780 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.774+0000 7fcc89e00640 1 --2- 172.21.15.145:0/1053012676 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc8c146970 0x7fcc8c0af580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:40.780 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.774+0000 7fcc89e00640 1 -- 172.21.15.145:0/1053012676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc8c0cd550 msgr2=0x7fcc8c0af040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:40.781 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.774+0000 7fcc89e00640 1 --2- 172.21.15.145:0/1053012676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc8c0cd550 0x7fcc8c0af040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:40.781 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.774+0000 7fcc89e00640 1 -- 172.21.15.145:0/1053012676 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc8c135470 con 0x7fcc8c06e060 2024-05-09T04:34:40.781 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.775+0000 7fcc89e00640 1 --2- 172.21.15.145:0/1053012676 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc8c06e060 0x7fcc8c0c5ac0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c00c7f0 tx=0x7fcc7c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:40.781 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.775+0000 7fcc82000640 1 -- 172.21.15.145:0/1053012676 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c015070 con 0x7fcc8c06e060 2024-05-09T04:34:40.781 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.775+0000 7fcc82000640 1 -- 172.21.15.145:0/1053012676 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc7c007820 con 0x7fcc8c06e060 2024-05-09T04:34:40.782 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.775+0000 7fcc8b200640 1 -- 172.21.15.145:0/1053012676 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc8c135750 con 0x7fcc8c06e060 2024-05-09T04:34:40.782 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.775+0000 7fcc82000640 1 -- 172.21.15.145:0/1053012676 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c007bd0 con 0x7fcc8c06e060 2024-05-09T04:34:40.783 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.775+0000 7fcc8b200640 1 -- 172.21.15.145:0/1053012676 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc8c135d10 con 0x7fcc8c06e060 2024-05-09T04:34:40.783 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.776+0000 7fcc8b200640 1 -- 172.21.15.145:0/1053012676 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc8c076930 con 0x7fcc8c06e060 2024-05-09T04:34:40.783 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.777+0000 7fcc82000640 1 -- 172.21.15.145:0/1053012676 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fcc7c018070 con 0x7fcc8c06e060 2024-05-09T04:34:40.783 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.778+0000 7fcc82000640 1 --2- 172.21.15.145:0/1053012676 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fcc680762d0 0x7fcc68078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:40.784 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.778+0000 7fcc82000640 1 -- 172.21.15.145:0/1053012676 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7fcc7c0974e0 con 0x7fcc8c06e060 2024-05-09T04:34:40.784 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.778+0000 7fcc83e00640 1 --2- 172.21.15.145:0/1053012676 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fcc680762d0 0x7fcc68078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:40.785 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.779+0000 7fcc83e00640 1 --2- 172.21.15.145:0/1053012676 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fcc680762d0 0x7fcc68078790 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c0b1c90 tx=0x7fcc74008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:40.786 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.783+0000 7fcc82000640 1 -- 172.21.15.145:0/1053012676 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fcc7c061060 con 0x7fcc8c06e060 2024-05-09T04:34:40.976 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:40.976+0000 7fcc8b200640 1 -- 172.21.15.145:0/1053012676 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fcc8c0b19e0 con 0x7fcc8c06e060 2024-05-09T04:34:41.634 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.633+0000 7fcc82000640 1 -- 172.21.15.145:0/1053012676 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v29) ==== 138+0+0 (secure 0 0 0) 0x7fcc7c060960 con 0x7fcc8c06e060 2024-05-09T04:34:41.634 INFO:teuthology.orchestra.run.smithi145.stderr:pool 'cephfs_metadata' created 2024-05-09T04:34:41.638 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.638+0000 7fcc8b200640 1 -- 172.21.15.145:0/1053012676 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fcc680762d0 msgr2=0x7fcc68078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:41.638 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.638+0000 7fcc8b200640 1 --2- 172.21.15.145:0/1053012676 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fcc680762d0 0x7fcc68078790 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c0b1c90 tx=0x7fcc74008040 comp rx=0 tx=0).stop 2024-05-09T04:34:41.638 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.638+0000 7fcc8b200640 1 -- 172.21.15.145:0/1053012676 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc8c06e060 msgr2=0x7fcc8c0c5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:41.638 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.638+0000 7fcc8b200640 1 --2- 172.21.15.145:0/1053012676 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc8c06e060 0x7fcc8c0c5ac0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c00c7f0 tx=0x7fcc7c00ccb0 comp rx=0 tx=0).stop 2024-05-09T04:34:41.640 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.640+0000 7fcc8b200640 1 -- 172.21.15.145:0/1053012676 shutdown_connections 2024-05-09T04:34:41.640 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.640+0000 7fcc8b200640 1 --2- 172.21.15.145:0/1053012676 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fcc680762d0 0x7fcc68078790 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:41.640 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.640+0000 7fcc8b200640 1 --2- 172.21.15.145:0/1053012676 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc8c146970 0x7fcc8c0af580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:41.640 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.640+0000 7fcc8b200640 1 --2- 172.21.15.145:0/1053012676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc8c0cd550 0x7fcc8c0af040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:41.641 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.640+0000 7fcc8b200640 1 --2- 172.21.15.145:0/1053012676 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc8c06e060 0x7fcc8c0c5ac0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:41.641 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.640+0000 7fcc8b200640 1 -- 172.21.15.145:0/1053012676 >> 172.21.15.145:0/1053012676 conn(0x7fcc8c0b2850 msgr2=0x7fcc8c073730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:41.641 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.641+0000 7fcc8b200640 1 -- 172.21.15.145:0/1053012676 shutdown_connections 2024-05-09T04:34:41.642 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.642+0000 7fcc8b200640 1 -- 172.21.15.145:0/1053012676 wait complete. 2024-05-09T04:34:41.670 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-09T04:34:41.869 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.869+0000 7f09a5200640 1 Processor -- start 2024-05-09T04:34:41.869 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.869+0000 7f09a5200640 1 -- start start 2024-05-09T04:34:41.871 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.871+0000 7f09a5200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f09a00b3020 0x7f09a0073f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:41.871 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.871+0000 7f09a5200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f09a00b3400 0x7f09a015d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:41.871 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.871+0000 7f09a5200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09a015e2f0 con 0x7f09a015dad0 2024-05-09T04:34:41.871 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.871+0000 7f09a5200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f09a0073a90 con 0x7f09a00b3020 2024-05-09T04:34:41.871 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.871+0000 7f09a5200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f09a0073440 con 0x7f09a00b3400 2024-05-09T04:34:41.871 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.871+0000 7f099de00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f09a00b3020 0x7f09a0073f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:41.871 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.871+0000 7f099de00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f09a00b3020 0x7f09a0073f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:59190/0 (socket says 172.21.15.145:59190) 2024-05-09T04:34:41.871 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.871+0000 7f099de00640 1 -- 172.21.15.145:0/2184557276 learned_addr learned my addr 172.21.15.145:0/2184557276 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:41.872 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.871+0000 7f099de00640 1 -- 172.21.15.145:0/2184557276 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f09a00b3400 msgr2=0x7f09a015d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:41.872 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.871+0000 7f099de00640 1 --2- 172.21.15.145:0/2184557276 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f09a00b3400 0x7f09a015d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:41.872 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.871+0000 7f099de00640 1 -- 172.21.15.145:0/2184557276 >> v1:172.21.15.192:6789/0 conn(0x7f09a015dad0 legacy=0x7f09a015e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:41.872 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.871+0000 7f099de00640 1 -- 172.21.15.145:0/2184557276 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09a0073720 con 0x7f09a00b3020 2024-05-09T04:34:41.872 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.872+0000 7f099de00640 1 --2- 172.21.15.145:0/2184557276 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f09a00b3020 0x7f09a0073f20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0990009870 tx=0x7f099002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=577aaf2411176cbc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:41.873 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.873+0000 7f099ca00640 1 -- 172.21.15.145:0/2184557276 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0990037070 con 0x7f09a00b3020 2024-05-09T04:34:41.873 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.873+0000 7f099ca00640 1 -- 172.21.15.145:0/2184557276 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0990032440 con 0x7f09a00b3020 2024-05-09T04:34:41.873 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.873+0000 7f099ca00640 1 -- 172.21.15.145:0/2184557276 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0990032810 con 0x7f09a00b3020 2024-05-09T04:34:41.873 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.873+0000 7f09a5200640 1 -- 172.21.15.145:0/2184557276 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f09a00b3020 msgr2=0x7f09a0073f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:41.874 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.873+0000 7f09a5200640 1 --2- 172.21.15.145:0/2184557276 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f09a00b3020 0x7f09a0073f20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0990009870 tx=0x7f099002a1c0 comp rx=0 tx=0).stop 2024-05-09T04:34:41.874 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.874+0000 7f09a5200640 1 -- 172.21.15.145:0/2184557276 shutdown_connections 2024-05-09T04:34:41.874 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.874+0000 7f09a5200640 1 --2- 172.21.15.145:0/2184557276 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f09a00b3400 0x7f09a015d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:41.874 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.874+0000 7f09a5200640 1 --2- 172.21.15.145:0/2184557276 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f09a00b3020 0x7f09a0073f20 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:41.874 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.874+0000 7f09a5200640 1 -- 172.21.15.145:0/2184557276 >> 172.21.15.145:0/2184557276 conn(0x7f09a00b6600 msgr2=0x7f09a00b6a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:41.874 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.874+0000 7f09a5200640 1 -- 172.21.15.145:0/2184557276 shutdown_connections 2024-05-09T04:34:41.874 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.874+0000 7f09a5200640 1 -- 172.21.15.145:0/2184557276 wait complete. 2024-05-09T04:34:41.875 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.876+0000 7f09a5200640 1 Processor -- start 2024-05-09T04:34:41.875 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.876+0000 7f09a5200640 1 -- start start 2024-05-09T04:34:41.876 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.877+0000 7f09a5200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f09a00b3020 0x7f09a01428e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:41.877 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.877+0000 7f09a5200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09a00b3400 0x7f09a01524b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:41.877 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.877+0000 7f09a5200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f09a015dad0 0x7f09a00b4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:41.877 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.877+0000 7f09a5200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f09a015df40 con 0x7f09a015dad0 2024-05-09T04:34:41.877 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.877+0000 7f09a5200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f09a00cda60 con 0x7f09a00b3400 2024-05-09T04:34:41.877 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.877+0000 7f09a5200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f09a00b4c40 con 0x7f09a00b3020 2024-05-09T04:34:41.877 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.877+0000 7f099e800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f09a015dad0 0x7f09a00b4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:41.877 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.877+0000 7f099e800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f09a015dad0 0x7f09a00b4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:59204/0 (socket says 172.21.15.145:59204) 2024-05-09T04:34:41.877 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.877+0000 7f099e800640 1 -- 172.21.15.145:0/4024971663 learned_addr learned my addr 172.21.15.145:0/4024971663 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:41.878 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.877+0000 7f099de00640 1 --2- 172.21.15.145:0/4024971663 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f09a00b3020 0x7f09a01428e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:41.879 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.877+0000 7f099d400640 1 --2- 172.21.15.145:0/4024971663 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09a00b3400 0x7f09a01524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:41.879 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.877+0000 7f099e800640 1 -- 172.21.15.145:0/4024971663 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f09a00b3020 msgr2=0x7f09a01428e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:41.879 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.877+0000 7f099e800640 1 --2- 172.21.15.145:0/4024971663 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f09a00b3020 0x7f09a01428e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:41.879 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.877+0000 7f099e800640 1 -- 172.21.15.145:0/4024971663 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09a00b3400 msgr2=0x7f09a01524b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:41.879 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.877+0000 7f099e800640 1 --2- 172.21.15.145:0/4024971663 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09a00b3400 0x7f09a01524b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:41.879 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.878+0000 7f099e800640 1 -- 172.21.15.145:0/4024971663 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09a0152b40 con 0x7f09a015dad0 2024-05-09T04:34:41.879 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.878+0000 7f099de00640 1 --2- 172.21.15.145:0/4024971663 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f09a00b3020 0x7f09a01428e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:34:41.879 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.878+0000 7f099e800640 1 --2- 172.21.15.145:0/4024971663 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f09a015dad0 0x7f09a00b4700 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f099800b490 tx=0x7f099800b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:41.879 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.878+0000 7f0996000640 1 -- 172.21.15.145:0/4024971663 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0998015020 con 0x7f09a015dad0 2024-05-09T04:34:41.879 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.878+0000 7f0996000640 1 -- 172.21.15.145:0/4024971663 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0998011850 con 0x7f09a015dad0 2024-05-09T04:34:41.879 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.878+0000 7f0996000640 1 -- 172.21.15.145:0/4024971663 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0998011c40 con 0x7f09a015dad0 2024-05-09T04:34:41.879 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.878+0000 7f09a5200640 1 -- 172.21.15.145:0/4024971663 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09a0152e20 con 0x7f09a015dad0 2024-05-09T04:34:41.879 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.878+0000 7f09a5200640 1 -- 172.21.15.145:0/4024971663 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09a00b52c0 con 0x7f09a015dad0 2024-05-09T04:34:41.880 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.879+0000 7f09a5200640 1 -- 172.21.15.145:0/4024971663 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09a00b54c0 con 0x7f09a015dad0 2024-05-09T04:34:41.880 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.881+0000 7f0996000640 1 -- 172.21.15.145:0/4024971663 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0998018020 con 0x7f09a015dad0 2024-05-09T04:34:41.882 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.882+0000 7f0996000640 1 --2- 172.21.15.145:0/4024971663 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f09800762d0 0x7f0980078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:41.882 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.882+0000 7f0996000640 1 -- 172.21.15.145:0/4024971663 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5227+0+0 (secure 0 0 0) 0x7f09980972f0 con 0x7f09a015dad0 2024-05-09T04:34:41.882 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.882+0000 7f099de00640 1 --2- 172.21.15.145:0/4024971663 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f09800762d0 0x7f0980078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:41.882 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.883+0000 7f099de00640 1 --2- 172.21.15.145:0/4024971663 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f09800762d0 0x7f0980078790 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0990034000 tx=0x7f099002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:41.886 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:41.887+0000 7f0996000640 1 -- 172.21.15.145:0/4024971663 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f09a00b54c0 con 0x7f09a015dad0 2024-05-09T04:34:42.094 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.095+0000 7f09a5200640 1 -- 172.21.15.145:0/4024971663 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f09a006a310 con 0x7f09a015dad0 2024-05-09T04:34:42.649 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.649+0000 7f0996000640 1 -- 172.21.15.145:0/4024971663 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v30) ==== 170+0+0 (secure 0 0 0) 0x7f099801f020 con 0x7f09a015dad0 2024-05-09T04:34:42.650 INFO:teuthology.orchestra.run.smithi145.stderr:pool 'cephfs_data' created 2024-05-09T04:34:42.654 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.654+0000 7f09a5200640 1 -- 172.21.15.145:0/4024971663 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f09800762d0 msgr2=0x7f0980078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:42.654 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.654+0000 7f09a5200640 1 --2- 172.21.15.145:0/4024971663 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f09800762d0 0x7f0980078790 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0990034000 tx=0x7f099002e000 comp rx=0 tx=0).stop 2024-05-09T04:34:42.654 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.654+0000 7f09a5200640 1 -- 172.21.15.145:0/4024971663 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f09a015dad0 msgr2=0x7f09a00b4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:42.654 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.654+0000 7f09a5200640 1 --2- 172.21.15.145:0/4024971663 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f09a015dad0 0x7f09a00b4700 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f099800b490 tx=0x7f099800b950 comp rx=0 tx=0).stop 2024-05-09T04:34:42.654 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.654+0000 7f09a5200640 1 -- 172.21.15.145:0/4024971663 shutdown_connections 2024-05-09T04:34:42.654 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.654+0000 7f09a5200640 1 --2- 172.21.15.145:0/4024971663 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f09800762d0 0x7f0980078790 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:42.654 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.654+0000 7f09a5200640 1 --2- 172.21.15.145:0/4024971663 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f09a015dad0 0x7f09a00b4700 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:42.654 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.654+0000 7f09a5200640 1 --2- 172.21.15.145:0/4024971663 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09a00b3400 0x7f09a01524b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:42.655 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.654+0000 7f09a5200640 1 --2- 172.21.15.145:0/4024971663 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f09a00b3020 0x7f09a01428e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:42.655 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.654+0000 7f09a5200640 1 -- 172.21.15.145:0/4024971663 >> 172.21.15.145:0/4024971663 conn(0x7f09a00b6600 msgr2=0x7f09a015f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:42.655 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.654+0000 7f09a5200640 1 -- 172.21.15.145:0/4024971663 shutdown_connections 2024-05-09T04:34:42.655 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.654+0000 7f09a5200640 1 -- 172.21.15.145:0/4024971663 wait complete. 2024-05-09T04:34:42.691 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-09T04:34:42.868 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.868+0000 7fe17e200640 1 Processor -- start 2024-05-09T04:34:42.868 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.868+0000 7fe17e200640 1 -- start start 2024-05-09T04:34:42.870 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.870+0000 7fe17e200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe1780d37d0 0x7fe1780d3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:42.870 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.870+0000 7fe17e200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fe1780b2590 0x7fe1780b2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:42.870 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.870+0000 7fe17e200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1780d0940 con 0x7fe17815f590 2024-05-09T04:34:42.870 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.870+0000 7fe17e200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fe1780b1e20 con 0x7fe1780b2590 2024-05-09T04:34:42.870 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.870+0000 7fe17e200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fe1780d0c20 con 0x7fe1780d37d0 2024-05-09T04:34:42.871 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.870+0000 7fe177e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fe1780b2590 0x7fe1780b2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:42.871 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.870+0000 7fe177e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fe1780b2590 0x7fe1780b2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:59208/0 (socket says 172.21.15.145:59208) 2024-05-09T04:34:42.871 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.870+0000 7fe177e00640 1 -- 172.21.15.145:0/318464665 learned_addr learned my addr 172.21.15.145:0/318464665 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:42.871 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.871+0000 7fe176a00640 1 --2- 172.21.15.145:0/318464665 >> v2:172.21.15.192:3300/0 conn(0x7fe1780d37d0 0x7fe1780d3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:42.871 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.871+0000 7fe177e00640 1 -- 172.21.15.145:0/318464665 >> v2:172.21.15.192:3300/0 conn(0x7fe1780d37d0 msgr2=0x7fe1780d3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:42.871 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.871+0000 7fe177e00640 1 --2- 172.21.15.145:0/318464665 >> v2:172.21.15.192:3300/0 conn(0x7fe1780d37d0 0x7fe1780d3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:42.871 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.871+0000 7fe177e00640 1 -- 172.21.15.145:0/318464665 >> v1:172.21.15.192:6789/0 conn(0x7fe17815f590 legacy=0x7fe1780d5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:42.871 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.871+0000 7fe177e00640 1 -- 172.21.15.145:0/318464665 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1780d0f00 con 0x7fe1780b2590 2024-05-09T04:34:42.872 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.872+0000 7fe177e00640 1 --2- 172.21.15.145:0/318464665 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1780b2590 0x7fe1780b2a00 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe17808f850 tx=0x7fe17002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5129aa35c862b48 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:42.872 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.872+0000 7fe176000640 1 -- 172.21.15.145:0/318464665 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe170036070 con 0x7fe1780b2590 2024-05-09T04:34:42.872 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.872+0000 7fe176000640 1 -- 172.21.15.145:0/318464665 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe170031600 con 0x7fe1780b2590 2024-05-09T04:34:42.872 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.872+0000 7fe176000640 1 -- 172.21.15.145:0/318464665 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1700319d0 con 0x7fe1780b2590 2024-05-09T04:34:42.873 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.873+0000 7fe17e200640 1 -- 172.21.15.145:0/318464665 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1780b2590 msgr2=0x7fe1780b2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:42.873 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.873+0000 7fe17e200640 1 --2- 172.21.15.145:0/318464665 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1780b2590 0x7fe1780b2a00 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe17808f850 tx=0x7fe17002a610 comp rx=0 tx=0).stop 2024-05-09T04:34:42.873 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.873+0000 7fe17e200640 1 -- 172.21.15.145:0/318464665 shutdown_connections 2024-05-09T04:34:42.873 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.873+0000 7fe17e200640 1 --2- 172.21.15.145:0/318464665 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1780b2590 0x7fe1780b2a00 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:42.873 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.873+0000 7fe17e200640 1 --2- 172.21.15.145:0/318464665 >> v2:172.21.15.192:3300/0 conn(0x7fe1780d37d0 0x7fe1780d3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:42.874 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.873+0000 7fe17e200640 1 -- 172.21.15.145:0/318464665 >> 172.21.15.145:0/318464665 conn(0x7fe17809e530 msgr2=0x7fe1780d51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:42.874 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.873+0000 7fe17e200640 1 -- 172.21.15.145:0/318464665 shutdown_connections 2024-05-09T04:34:42.874 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.873+0000 7fe17e200640 1 -- 172.21.15.145:0/318464665 wait complete. 2024-05-09T04:34:42.874 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.875+0000 7fe17e200640 1 Processor -- start 2024-05-09T04:34:42.874 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.875+0000 7fe17e200640 1 -- start start 2024-05-09T04:34:42.876 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.876+0000 7fe17e200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1780b2590 0x7fe1781599f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:42.876 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.876+0000 7fe17e200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1780d37d0 0x7fe17815dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:42.876 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.876+0000 7fe17e200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe17815f590 0x7fe17815e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:42.876 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.876+0000 7fe17e200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe1780d6010 con 0x7fe1780b2590 2024-05-09T04:34:42.876 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.876+0000 7fe17e200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe17815f970 con 0x7fe1780d37d0 2024-05-09T04:34:42.877 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.876+0000 7fe17e200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe1780d3180 con 0x7fe17815f590 2024-05-09T04:34:42.877 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.876+0000 7fe177400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1780b2590 0x7fe1781599f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:42.877 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.876+0000 7fe177400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1780b2590 0x7fe1781599f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:59214/0 (socket says 172.21.15.145:59214) 2024-05-09T04:34:42.877 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.876+0000 7fe177400640 1 -- 172.21.15.145:0/631033083 learned_addr learned my addr 172.21.15.145:0/631033083 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:42.877 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.876+0000 7fe177400640 1 -- 172.21.15.145:0/631033083 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe17815f590 msgr2=0x7fe17815e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:42.877 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.876+0000 7fe177400640 1 --2- 172.21.15.145:0/631033083 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe17815f590 0x7fe17815e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:42.877 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.877+0000 7fe177400640 1 -- 172.21.15.145:0/631033083 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1780d37d0 msgr2=0x7fe17815dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:42.877 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.877+0000 7fe177400640 1 --2- 172.21.15.145:0/631033083 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1780d37d0 0x7fe17815dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:42.877 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.877+0000 7fe177400640 1 -- 172.21.15.145:0/631033083 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1780cc2c0 con 0x7fe1780b2590 2024-05-09T04:34:42.877 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.877+0000 7fe177400640 1 --2- 172.21.15.145:0/631033083 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1780b2590 0x7fe1781599f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe16800b420 tx=0x7fe16800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:42.878 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.877+0000 7fe16f400640 1 -- 172.21.15.145:0/631033083 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe168011830 con 0x7fe1780b2590 2024-05-09T04:34:42.878 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.877+0000 7fe16f400640 1 -- 172.21.15.145:0/631033083 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe168016070 con 0x7fe1780b2590 2024-05-09T04:34:42.878 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.877+0000 7fe16f400640 1 -- 172.21.15.145:0/631033083 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe168011c10 con 0x7fe1780b2590 2024-05-09T04:34:42.878 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.877+0000 7fe17e200640 1 -- 172.21.15.145:0/631033083 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1780cc5a0 con 0x7fe1780b2590 2024-05-09T04:34:42.878 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.877+0000 7fe17e200640 1 -- 172.21.15.145:0/631033083 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1780ccb60 con 0x7fe1780b2590 2024-05-09T04:34:42.878 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.878+0000 7fe17e200640 1 -- 172.21.15.145:0/631033083 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe17806a310 con 0x7fe1780b2590 2024-05-09T04:34:42.879 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.880+0000 7fe16f400640 1 -- 172.21.15.145:0/631033083 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe16801b460 con 0x7fe1780b2590 2024-05-09T04:34:42.880 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.881+0000 7fe16f400640 1 --2- 172.21.15.145:0/631033083 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fe1580762d0 0x7fe158078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:42.881 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.881+0000 7fe16f400640 1 -- 172.21.15.145:0/631033083 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5609+0+0 (secure 0 0 0) 0x7fe168097150 con 0x7fe1780b2590 2024-05-09T04:34:42.881 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.881+0000 7fe176a00640 1 --2- 172.21.15.145:0/631033083 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fe1580762d0 0x7fe158078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:42.881 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.882+0000 7fe176a00640 1 --2- 172.21.15.145:0/631033083 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fe1580762d0 0x7fe158078790 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe1780cf470 tx=0x7fe164008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:42.886 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:42.886+0000 7fe16f400640 1 -- 172.21.15.145:0/631033083 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe168020020 con 0x7fe1780b2590 2024-05-09T04:34:43.131 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.132+0000 7fe17e200640 1 -- 172.21.15.145:0/631033083 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fe1780b3eb0 con 0x7fe1780b2590 2024-05-09T04:34:43.671 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.670+0000 7fe16f400640 1 -- 172.21.15.145:0/631033083 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) ==== 172+0+0 (secure 0 0 0) 0x7fe168060a70 con 0x7fe1780b2590 2024-05-09T04:34:43.673 INFO:teuthology.orchestra.run.smithi145.stderr:new fs with metadata pool 4 and data pool 5 2024-05-09T04:34:43.680 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.680+0000 7fe17e200640 1 -- 172.21.15.145:0/631033083 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fe1580762d0 msgr2=0x7fe158078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:43.680 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.680+0000 7fe17e200640 1 --2- 172.21.15.145:0/631033083 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fe1580762d0 0x7fe158078790 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe1780cf470 tx=0x7fe164008040 comp rx=0 tx=0).stop 2024-05-09T04:34:43.680 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.680+0000 7fe17e200640 1 -- 172.21.15.145:0/631033083 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1780b2590 msgr2=0x7fe1781599f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:43.680 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.680+0000 7fe17e200640 1 --2- 172.21.15.145:0/631033083 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1780b2590 0x7fe1781599f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe16800b420 tx=0x7fe16800b8e0 comp rx=0 tx=0).stop 2024-05-09T04:34:43.683 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.683+0000 7fe17e200640 1 -- 172.21.15.145:0/631033083 shutdown_connections 2024-05-09T04:34:43.683 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.683+0000 7fe17e200640 1 --2- 172.21.15.145:0/631033083 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fe1580762d0 0x7fe158078790 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:43.683 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.683+0000 7fe17e200640 1 --2- 172.21.15.145:0/631033083 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe17815f590 0x7fe17815e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:43.683 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.683+0000 7fe17e200640 1 --2- 172.21.15.145:0/631033083 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1780d37d0 0x7fe17815dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:43.683 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.683+0000 7fe17e200640 1 --2- 172.21.15.145:0/631033083 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe1780b2590 0x7fe1781599f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:43.683 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.683+0000 7fe17e200640 1 -- 172.21.15.145:0/631033083 >> 172.21.15.145:0/631033083 conn(0x7fe17809e530 msgr2=0x7fe1780b4690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:43.684 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.684+0000 7fe17e200640 1 -- 172.21.15.145:0/631033083 shutdown_connections 2024-05-09T04:34:43.685 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.686+0000 7fe17e200640 1 -- 172.21.15.145:0/631033083 wait complete. 2024-05-09T04:34:43.719 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T04:34:43.887 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.887+0000 7f305d000640 1 Processor -- start 2024-05-09T04:34:43.887 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.887+0000 7f305d000640 1 -- start start 2024-05-09T04:34:43.888 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.888+0000 7f305d000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f305806e860 0x7f305806ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:43.888 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.888+0000 7f305d000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f305815d780 0x7f3058160c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:43.888 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.888+0000 7f305d000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f305806f180 con 0x7f30580ce620 2024-05-09T04:34:43.888 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.888+0000 7f305d000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f30581615c0 con 0x7f305815d780 2024-05-09T04:34:43.888 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.888+0000 7f305d000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f30581618a0 con 0x7f305806e860 2024-05-09T04:34:43.889 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.889+0000 7f3056800640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f30580ce620 0x7f30580ceab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:48996/0 (socket says 172.21.15.145:48996) 2024-05-09T04:34:43.889 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.889+0000 7f3056800640 1 -- 172.21.15.145:0/646222083 learned_addr learned my addr 172.21.15.145:0/646222083 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:34:43.891 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.889+0000 7f3055400640 1 --2- 172.21.15.145:0/646222083 >> v2:172.21.15.192:3300/0 conn(0x7f305815d780 0x7f3058160c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:43.891 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.889+0000 7f3054a00640 1 -- 172.21.15.145:0/646222083 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2293982744 0 0) 0x7f305806f180 con 0x7f30580ce620 2024-05-09T04:34:43.891 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.889+0000 7f3054a00640 1 -- 172.21.15.145:0/646222083 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f303c003520 con 0x7f30580ce620 2024-05-09T04:34:43.891 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.889+0000 7f3055e00640 1 --2- 172.21.15.145:0/646222083 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f305806e860 0x7f305806ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:43.891 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.889+0000 7f3054a00640 1 -- 172.21.15.145:0/646222083 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 78740891 0 0) 0x7f303c003520 con 0x7f30580ce620 2024-05-09T04:34:43.891 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.890+0000 7f3054a00640 1 -- 172.21.15.145:0/646222083 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f305806e860 msgr2=0x7f305806ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:43.891 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.890+0000 7f3054a00640 1 --2- 172.21.15.145:0/646222083 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f305806e860 0x7f305806ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:43.891 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.890+0000 7f3054a00640 1 -- 172.21.15.145:0/646222083 >> v2:172.21.15.192:3300/0 conn(0x7f305815d780 msgr2=0x7f3058160c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:43.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.890+0000 7f3054a00640 1 --2- 172.21.15.145:0/646222083 >> v2:172.21.15.192:3300/0 conn(0x7f305815d780 0x7f3058160c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:43.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.890+0000 7f3054a00640 1 -- 172.21.15.145:0/646222083 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30580b4110 con 0x7f30580ce620 2024-05-09T04:34:43.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.890+0000 7f3054a00640 1 -- 172.21.15.145:0/646222083 <== mon.0 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3474413261 0 0) 0x7f30480016f0 con 0x7f30580ce620 2024-05-09T04:34:43.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.890+0000 7f3054a00640 1 -- 172.21.15.145:0/646222083 >> v1:172.21.15.145:6789/0 conn(0x7f30580ce620 legacy=0x7f30580ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:43.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.890+0000 7f3054a00640 1 --2- 172.21.15.145:0/646222083 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f303c003a00 0x7f303c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:43.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.890+0000 7f3054a00640 1 --2- 172.21.15.145:0/646222083 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f303c004800 0x7f303c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:43.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.890+0000 7f3054a00640 1 --2- 172.21.15.145:0/646222083 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f305806e860 0x7f303c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:43.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.890+0000 7f3054a00640 1 -- 172.21.15.145:0/646222083 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f30581615c0 con 0x7f303c004800 2024-05-09T04:34:43.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.890+0000 7f3054a00640 1 -- 172.21.15.145:0/646222083 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f30581618a0 con 0x7f303c003a00 2024-05-09T04:34:43.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.890+0000 7f3054a00640 1 -- 172.21.15.145:0/646222083 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f303c00ff30 con 0x7f305806e860 2024-05-09T04:34:43.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.890+0000 7f3055400640 1 --2- 172.21.15.145:0/646222083 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f303c004800 0x7f303c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:43.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.890+0000 7f3055e00640 1 --2- 172.21.15.145:0/646222083 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f303c003a00 0x7f303c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:43.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.890+0000 7f3055400640 1 -- 172.21.15.145:0/646222083 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f305806e860 msgr2=0x7f303c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:43.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.890+0000 7f3056800640 1 --2- 172.21.15.145:0/646222083 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f305806e860 0x7f303c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:43.893 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.891+0000 7f3055400640 1 --2- 172.21.15.145:0/646222083 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f305806e860 0x7f303c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:43.893 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.891+0000 7f3055400640 1 -- 172.21.15.145:0/646222083 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f303c003a00 msgr2=0x7f303c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:43.893 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.891+0000 7f3055400640 1 --2- 172.21.15.145:0/646222083 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f303c003a00 0x7f303c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:43.893 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.891+0000 7f3055400640 1 -- 172.21.15.145:0/646222083 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f303c003520 con 0x7f303c004800 2024-05-09T04:34:43.893 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.891+0000 7f3055e00640 1 --2- 172.21.15.145:0/646222083 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f303c003a00 0x7f303c003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:34:43.893 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.891+0000 7f3056800640 1 --2- 172.21.15.145:0/646222083 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f305806e860 0x7f303c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:34:43.893 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.892+0000 7f3055400640 1 --2- 172.21.15.145:0/646222083 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f303c004800 0x7f303c004c80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f305815dba0 tx=0x7f303c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae113afba22c7ef0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:43.894 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.892+0000 7f3054a00640 1 -- 172.21.15.145:0/646222083 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3044030070 con 0x7f303c004800 2024-05-09T04:34:43.894 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.892+0000 7f3054a00640 1 -- 172.21.15.145:0/646222083 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f304402ada0 con 0x7f303c004800 2024-05-09T04:34:43.894 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.892+0000 7f3054a00640 1 -- 172.21.15.145:0/646222083 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304400b600 con 0x7f303c004800 2024-05-09T04:34:43.895 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.892+0000 7f305d000640 1 -- 172.21.15.145:0/646222083 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f303c004800 msgr2=0x7f303c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:43.895 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.892+0000 7f305d000640 1 --2- 172.21.15.145:0/646222083 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f303c004800 0x7f303c004c80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f305815dba0 tx=0x7f303c005830 comp rx=0 tx=0).stop 2024-05-09T04:34:43.895 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.892+0000 7f3056800640 1 -- 172.21.15.145:0/646222083 reap_dead start 2024-05-09T04:34:43.895 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.893+0000 7f305d000640 1 -- 172.21.15.145:0/646222083 shutdown_connections 2024-05-09T04:34:43.895 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.893+0000 7f305d000640 1 -- 172.21.15.145:0/646222083 >> 172.21.15.145:0/646222083 conn(0x7f305806c870 msgr2=0x7f305806cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:43.895 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.893+0000 7f305d000640 1 -- 172.21.15.145:0/646222083 shutdown_connections 2024-05-09T04:34:43.895 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.893+0000 7f305d000640 1 -- 172.21.15.145:0/646222083 wait complete. 2024-05-09T04:34:43.895 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.894+0000 7f305d000640 1 Processor -- start 2024-05-09T04:34:43.895 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.894+0000 7f305d000640 1 -- start start 2024-05-09T04:34:43.897 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.895+0000 7f305d000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f303c004800 0x7f305815caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:43.897 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.895+0000 7f305d000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f30581f12a0 0x7f3058140da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:43.897 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.895+0000 7f305d000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f305813d0c0 0x7f305813d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:43.897 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.895+0000 7f305d000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3058146d80 con 0x7f30581f12a0 2024-05-09T04:34:43.897 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.895+0000 7f305d000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3058146c00 con 0x7f305813d0c0 2024-05-09T04:34:43.898 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.895+0000 7f305d000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f3058161190 con 0x7f303c004800 2024-05-09T04:34:43.898 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.896+0000 7f3055400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f30581f12a0 0x7f3058140da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:43.898 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.896+0000 7f3055400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f30581f12a0 0x7f3058140da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:59230/0 (socket says 172.21.15.145:59230) 2024-05-09T04:34:43.898 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.896+0000 7f3055400640 1 -- 172.21.15.145:0/3176461568 learned_addr learned my addr 172.21.15.145:0/3176461568 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:43.898 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.896+0000 7f3055e00640 1 --2- 172.21.15.145:0/3176461568 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f303c004800 0x7f305815caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:43.898 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.896+0000 7f3055400640 1 -- 172.21.15.145:0/3176461568 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f303c004800 msgr2=0x7f305815caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:43.898 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.896+0000 7f3056800640 1 --2- 172.21.15.145:0/3176461568 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f305813d0c0 0x7f305813d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:43.898 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.896+0000 7f3055400640 1 --2- 172.21.15.145:0/3176461568 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f303c004800 0x7f305815caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:43.898 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.896+0000 7f3055400640 1 -- 172.21.15.145:0/3176461568 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f305813d0c0 msgr2=0x7f305813d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:43.898 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.896+0000 7f3055400640 1 --2- 172.21.15.145:0/3176461568 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f305813d0c0 0x7f305813d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:43.898 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.896+0000 7f3055400640 1 -- 172.21.15.145:0/3176461568 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3058138810 con 0x7f30581f12a0 2024-05-09T04:34:43.898 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.896+0000 7f3056800640 1 --2- 172.21.15.145:0/3176461568 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f305813d0c0 0x7f305813d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:34:43.898 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.896+0000 7f3055400640 1 --2- 172.21.15.145:0/3176461568 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f30581f12a0 0x7f3058140da0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f304402a890 tx=0x7f3044004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:43.898 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.897+0000 7f304e000640 1 -- 172.21.15.145:0/3176461568 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3044043020 con 0x7f30581f12a0 2024-05-09T04:34:43.899 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.897+0000 7f304e000640 1 -- 172.21.15.145:0/3176461568 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f304403f680 con 0x7f30581f12a0 2024-05-09T04:34:43.899 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.897+0000 7f304e000640 1 -- 172.21.15.145:0/3176461568 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304400b600 con 0x7f30581f12a0 2024-05-09T04:34:43.899 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.897+0000 7f305d000640 1 -- 172.21.15.145:0/3176461568 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30581416a0 con 0x7f30581f12a0 2024-05-09T04:34:43.899 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.897+0000 7f305d000640 1 -- 172.21.15.145:0/3176461568 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3058141c60 con 0x7f30581f12a0 2024-05-09T04:34:43.899 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.898+0000 7f305d000640 1 -- 172.21.15.145:0/3176461568 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f305806a310 con 0x7f30581f12a0 2024-05-09T04:34:43.899 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.899+0000 7f304e000640 1 -- 172.21.15.145:0/3176461568 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3044046020 con 0x7f30581f12a0 2024-05-09T04:34:43.899 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.900+0000 7f304e000640 1 --2- 172.21.15.145:0/3176461568 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f30380762d0 0x7f3038078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:43.900 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.900+0000 7f304e000640 1 -- 172.21.15.145:0/3176461568 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f30440b8540 con 0x7f30581f12a0 2024-05-09T04:34:43.900 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.900+0000 7f3055e00640 1 --2- 172.21.15.145:0/3176461568 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f30380762d0 0x7f3038078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:43.900 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.901+0000 7f3055e00640 1 --2- 172.21.15.145:0/3176461568 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f30380762d0 0x7f3038078790 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f305813c720 tx=0x7f305000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:43.904 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:43.904+0000 7f304e000640 1 -- 172.21.15.145:0/3176461568 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f304404d020 con 0x7f30581f12a0 2024-05-09T04:34:44.103 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.103+0000 7f305d000640 1 -- 172.21.15.145:0/3176461568 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f305806cc80 con 0x7f30581f12a0 2024-05-09T04:34:44.106 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.107+0000 7f304e000640 1 -- 172.21.15.145:0/3176461568 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) ==== 74+0+16730 (secure 0 0 0) 0x7f3058138810 con 0x7f30581f12a0 2024-05-09T04:34:44.107 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:34:44.107 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":31,"fsid":"957ee6c1-073b-44cf-9b93-9491b3fdfc1d","created":"2024-05-09T04:33:26.544367+0000","modified":"2024-05-09T04:34:43.134054+0000","last_up_change":"2024-05-09T04:33:39.039318+0000","last_in_change":"2024-05-09T04:33:29.902361+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T04:33:39.795344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-09T04:34:40.995268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-09T04:34:42.107773+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22e550ad-847b-4952-80e6-63d859ed1a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6801","nonce":3182572912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6803","nonce":3182572912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6807","nonce":3182572912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6805","nonce":3182572912}]},"public_addr":"172.21.15.145:6801/3182572912","cluster_addr":"172.21.15.145:6803/3182572912","heartbeat_back_addr":"172.21.15.145:6807/3182572912","heartbeat_front_addr":"172.21.15.145:6805/3182572912","state":["exists","up"]},{"osd":1,"uuid":"0f6b9fca-0c41-4145-8d74-598ddb42f2a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6825","nonce":1845225555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6827","nonce":1845225555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6831","nonce":1845225555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6829","nonce":1845225555}]},"public_addr":"172.21.15.145:6825/1845225555","cluster_addr":"172.21.15.145:6827/1845225555","heartbeat_back_addr":"172.21.15.145:6831/1845225555","heartbeat_front_addr":"172.21.15.145:6829/1845225555","state":["exists","up"]},{"osd":2,"uuid":"b3b1f67e-5790-483f-8610-e05a5bce4c30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6817","nonce":2168958993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6819","nonce":2168958993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6823","nonce":2168958993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6821","nonce":2168958993}]},"public_addr":"172.21.15.145:6817/2168958993","cluster_addr":"172.21.15.145:6819/2168958993","heartbeat_back_addr":"172.21.15.145:6823/2168958993","heartbeat_front_addr":"172.21.15.145:6821/2168958993","state":["exists","up"]},{"osd":3,"uuid":"884787a5-e037-4d31-9aae-743030eba4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6809","nonce":337009826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6811","nonce":337009826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6815","nonce":337009826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6813","nonce":337009826}]},"public_addr":"172.21.15.145:6809/337009826","cluster_addr":"172.21.15.145:6811/337009826","heartbeat_back_addr":"172.21.15.145:6815/337009826","heartbeat_front_addr":"172.21.15.145:6813/337009826","state":["exists","up"]},{"osd":4,"uuid":"0759fc1e-c5fc-4a2f-86bb-e8eccd83f833","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6818","nonce":3117700995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6821","nonce":3117700995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6827","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6829","nonce":3117700995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6823","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6825","nonce":3117700995}]},"public_addr":"172.21.15.192:6818/3117700995","cluster_addr":"172.21.15.192:6821/3117700995","heartbeat_back_addr":"172.21.15.192:6829/3117700995","heartbeat_front_addr":"172.21.15.192:6825/3117700995","state":["exists","up"]},{"osd":5,"uuid":"9443cc22-9379-46e3-9173-98b592bf2c16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6801","nonce":1328373268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6803","nonce":1328373268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6807","nonce":1328373268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6805","nonce":1328373268}]},"public_addr":"172.21.15.192:6801/1328373268","cluster_addr":"172.21.15.192:6803/1328373268","heartbeat_back_addr":"172.21.15.192:6807/1328373268","heartbeat_front_addr":"172.21.15.192:6805/1328373268","state":["exists","up"]},{"osd":6,"uuid":"d05a3311-505a-45b5-85b9-aa3113e96987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6817","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6820","nonce":1868011057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6824","nonce":1868011057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6831","nonce":1868011057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6828","nonce":1868011057}]},"public_addr":"172.21.15.192:6820/1868011057","cluster_addr":"172.21.15.192:6824/1868011057","heartbeat_back_addr":"172.21.15.192:6831/1868011057","heartbeat_front_addr":"172.21.15.192:6828/1868011057","state":["exists","up"]},{"osd":7,"uuid":"c7c23dbc-2029-43ec-9f53-25f3d58019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6809","nonce":3368512157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6811","nonce":3368512157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6815","nonce":3368512157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6813","nonce":3368512157}]},"public_addr":"172.21.15.192:6809/3368512157","cluster_addr":"172.21.15.192:6811/3368512157","heartbeat_back_addr":"172.21.15.192:6815/3368512157","heartbeat_front_addr":"172.21.15.192:6813/3368512157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.091188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.032567+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:34.797468+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.228839+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.622400+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.473103+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.566092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.349395+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T04:34:44.109 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.109+0000 7f305d000640 1 -- 172.21.15.145:0/3176461568 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f30380762d0 msgr2=0x7f3038078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:44.109 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.109+0000 7f305d000640 1 --2- 172.21.15.145:0/3176461568 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f30380762d0 0x7f3038078790 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f305813c720 tx=0x7f305000b040 comp rx=0 tx=0).stop 2024-05-09T04:34:44.109 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.109+0000 7f305d000640 1 -- 172.21.15.145:0/3176461568 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f30581f12a0 msgr2=0x7f3058140da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:44.109 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.109+0000 7f305d000640 1 --2- 172.21.15.145:0/3176461568 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f30581f12a0 0x7f3058140da0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f304402a890 tx=0x7f3044004730 comp rx=0 tx=0).stop 2024-05-09T04:34:44.109 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.109+0000 7f305d000640 1 -- 172.21.15.145:0/3176461568 shutdown_connections 2024-05-09T04:34:44.109 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.109+0000 7f305d000640 1 --2- 172.21.15.145:0/3176461568 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f30380762d0 0x7f3038078790 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:44.110 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.109+0000 7f305d000640 1 --2- 172.21.15.145:0/3176461568 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f305813d0c0 0x7f305813d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:44.110 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.109+0000 7f305d000640 1 --2- 172.21.15.145:0/3176461568 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f30581f12a0 0x7f3058140da0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:44.110 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.109+0000 7f305d000640 1 --2- 172.21.15.145:0/3176461568 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f303c004800 0x7f305815caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:44.110 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.109+0000 7f305d000640 1 -- 172.21.15.145:0/3176461568 >> 172.21.15.145:0/3176461568 conn(0x7f305806c870 msgr2=0x7f30580ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:44.110 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.109+0000 7f305d000640 1 -- 172.21.15.145:0/3176461568 shutdown_connections 2024-05-09T04:34:44.110 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.109+0000 7f305d000640 1 -- 172.21.15.145:0/3176461568 wait complete. 2024-05-09T04:34:44.127 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T04:34:44.319 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.319+0000 7f54e8a00640 1 Processor -- start 2024-05-09T04:34:44.319 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.319+0000 7f54e8a00640 1 -- start start 2024-05-09T04:34:44.320 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.321+0000 7f54e8a00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f54e40d5ad0 0x7f54e40d22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:44.320 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.321+0000 7f54e8a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f54e40b4760 0x7f54e40b4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:44.321 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.321+0000 7f54e8a00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54e40d3e70 con 0x7f54e40d3180 2024-05-09T04:34:44.321 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.321+0000 7f54e8a00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f54e40d4150 con 0x7f54e40d5ad0 2024-05-09T04:34:44.321 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.321+0000 7f54e8a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f54e40d2830 con 0x7f54e40b4760 2024-05-09T04:34:44.322 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.321+0000 7f54e1e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f54e40d5ad0 0x7f54e40d22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:44.322 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.321+0000 7f54e3200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f54e40b4760 0x7f54e40b4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:44.322 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.321+0000 7f54e1e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f54e40d5ad0 0x7f54e40d22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.145:53952/0 (socket says 172.21.15.145:53952) 2024-05-09T04:34:44.322 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.321+0000 7f54e1e00640 1 -- 172.21.15.145:0/4006606538 learned_addr learned my addr 172.21.15.145:0/4006606538 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:44.322 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.322+0000 7f54e1e00640 1 -- 172.21.15.145:0/4006606538 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f54e40b4760 msgr2=0x7f54e40b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:44.322 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.322+0000 7f54e1e00640 1 --2- 172.21.15.145:0/4006606538 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f54e40b4760 0x7f54e40b4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:44.322 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.322+0000 7f54e1e00640 1 -- 172.21.15.145:0/4006606538 >> v1:172.21.15.192:6789/0 conn(0x7f54e40d3180 legacy=0x7f54e40d3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:44.322 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.322+0000 7f54e1e00640 1 -- 172.21.15.145:0/4006606538 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54e40b6450 con 0x7f54e40d5ad0 2024-05-09T04:34:44.323 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.323+0000 7f54e1e00640 1 --2- 172.21.15.145:0/4006606538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54e40d5ad0 0x7f54e40d22f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f54d0009870 tx=0x7f54d002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a233213e8ccd4b22 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:44.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.324+0000 7f54e1400640 1 -- 172.21.15.145:0/4006606538 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d0037070 con 0x7f54e40d5ad0 2024-05-09T04:34:44.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.324+0000 7f54e1400640 1 -- 172.21.15.145:0/4006606538 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54d0032440 con 0x7f54e40d5ad0 2024-05-09T04:34:44.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.324+0000 7f54e1400640 1 -- 172.21.15.145:0/4006606538 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d0032810 con 0x7f54e40d5ad0 2024-05-09T04:34:44.325 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.325+0000 7f54e8a00640 1 -- 172.21.15.145:0/4006606538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54e40d5ad0 msgr2=0x7f54e40d22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:44.325 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.325+0000 7f54e8a00640 1 --2- 172.21.15.145:0/4006606538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54e40d5ad0 0x7f54e40d22f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f54d0009870 tx=0x7f54d002a1c0 comp rx=0 tx=0).stop 2024-05-09T04:34:44.325 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.325+0000 7f54e8a00640 1 -- 172.21.15.145:0/4006606538 shutdown_connections 2024-05-09T04:34:44.325 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.325+0000 7f54e8a00640 1 --2- 172.21.15.145:0/4006606538 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f54e40b4760 0x7f54e40b4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:44.325 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.325+0000 7f54e8a00640 1 --2- 172.21.15.145:0/4006606538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54e40d5ad0 0x7f54e40d22f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:44.325 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.325+0000 7f54e8a00640 1 -- 172.21.15.145:0/4006606538 >> 172.21.15.145:0/4006606538 conn(0x7f54e40d4fb0 msgr2=0x7f54e40d53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:44.325 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.325+0000 7f54e8a00640 1 -- 172.21.15.145:0/4006606538 shutdown_connections 2024-05-09T04:34:44.325 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.325+0000 7f54e8a00640 1 -- 172.21.15.145:0/4006606538 wait complete. 2024-05-09T04:34:44.326 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.327+0000 7f54e8a00640 1 Processor -- start 2024-05-09T04:34:44.326 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.327+0000 7f54e8a00640 1 -- start start 2024-05-09T04:34:44.329 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.328+0000 7f54e8a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f54e40b4760 0x7f54e414f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:44.329 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.328+0000 7f54e8a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f54e40d3180 0x7f54e415c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:44.329 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.328+0000 7f54e8a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54e40d5ad0 0x7f54e415cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:44.329 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.328+0000 7f54e8a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f54e40d1e60 con 0x7f54e40b4760 2024-05-09T04:34:44.329 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.328+0000 7f54e8a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f54e40b6200 con 0x7f54e40d5ad0 2024-05-09T04:34:44.329 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.328+0000 7f54e8a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f54e40b3fa0 con 0x7f54e40d3180 2024-05-09T04:34:44.329 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.328+0000 7f54e2800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f54e40b4760 0x7f54e414f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:44.329 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.328+0000 7f54e2800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f54e40b4760 0x7f54e414f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:59236/0 (socket says 172.21.15.145:59236) 2024-05-09T04:34:44.329 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.328+0000 7f54e2800640 1 -- 172.21.15.145:0/1642780719 learned_addr learned my addr 172.21.15.145:0/1642780719 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:44.329 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.328+0000 7f54e3200640 1 --2- 172.21.15.145:0/1642780719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54e40d5ad0 0x7f54e415cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:44.329 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.328+0000 7f54e1e00640 1 --2- 172.21.15.145:0/1642780719 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f54e40d3180 0x7f54e415c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:44.329 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.328+0000 7f54e2800640 1 -- 172.21.15.145:0/1642780719 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f54e40d3180 msgr2=0x7f54e415c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:44.330 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.328+0000 7f54e2800640 1 --2- 172.21.15.145:0/1642780719 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f54e40d3180 0x7f54e415c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:44.330 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.328+0000 7f54e2800640 1 -- 172.21.15.145:0/1642780719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54e40d5ad0 msgr2=0x7f54e415cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:44.330 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.328+0000 7f54e2800640 1 --2- 172.21.15.145:0/1642780719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54e40d5ad0 0x7f54e415cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:44.330 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.328+0000 7f54e2800640 1 -- 172.21.15.145:0/1642780719 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54e415d250 con 0x7f54e40b4760 2024-05-09T04:34:44.330 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.328+0000 7f54e3200640 1 --2- 172.21.15.145:0/1642780719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54e40d5ad0 0x7f54e415cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:34:44.330 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.329+0000 7f54e2800640 1 --2- 172.21.15.145:0/1642780719 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f54e40b4760 0x7f54e414f460 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f54d400c450 tx=0x7f54d400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:44.330 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.329+0000 7f54daa00640 1 -- 172.21.15.145:0/1642780719 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d40052d0 con 0x7f54e40b4760 2024-05-09T04:34:44.330 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.329+0000 7f54daa00640 1 -- 172.21.15.145:0/1642780719 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54d4005470 con 0x7f54e40b4760 2024-05-09T04:34:44.330 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.329+0000 7f54daa00640 1 -- 172.21.15.145:0/1642780719 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d4012870 con 0x7f54e40b4760 2024-05-09T04:34:44.330 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.329+0000 7f54e8a00640 1 -- 172.21.15.145:0/1642780719 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54e415d530 con 0x7f54e40b4760 2024-05-09T04:34:44.330 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.329+0000 7f54e8a00640 1 -- 172.21.15.145:0/1642780719 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54e415daf0 con 0x7f54e40b4760 2024-05-09T04:34:44.331 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.330+0000 7f54e8a00640 1 -- 172.21.15.145:0/1642780719 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54e406a310 con 0x7f54e40b4760 2024-05-09T04:34:44.331 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.331+0000 7f54daa00640 1 -- 172.21.15.145:0/1642780719 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f54d4012a10 con 0x7f54e40b4760 2024-05-09T04:34:44.332 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.332+0000 7f54daa00640 1 --2- 172.21.15.145:0/1642780719 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f54c40762d0 0x7f54c4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:44.332 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.332+0000 7f54daa00640 1 -- 172.21.15.145:0/1642780719 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f54d4097220 con 0x7f54e40b4760 2024-05-09T04:34:44.332 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.333+0000 7f54e1e00640 1 --2- 172.21.15.145:0/1642780719 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f54c40762d0 0x7f54c4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:44.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.333+0000 7f54e1e00640 1 --2- 172.21.15.145:0/1642780719 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f54c40762d0 0x7f54c4078790 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f54d0034000 tx=0x7f54d002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:44.336 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.337+0000 7f54daa00640 1 -- 172.21.15.145:0/1642780719 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f54d4060b70 con 0x7f54e40b4760 2024-05-09T04:34:44.534 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.535+0000 7f54e8a00640 1 -- 172.21.15.145:0/1642780719 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f54e40d1a60 con 0x7f54e40b4760 2024-05-09T04:34:44.536 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.537+0000 7f54daa00640 1 -- 172.21.15.145:0/1642780719 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) ==== 74+0+16730 (secure 0 0 0) 0x7f54d4060470 con 0x7f54e40b4760 2024-05-09T04:34:44.537 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:34:44.537 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":31,"fsid":"957ee6c1-073b-44cf-9b93-9491b3fdfc1d","created":"2024-05-09T04:33:26.544367+0000","modified":"2024-05-09T04:34:43.134054+0000","last_up_change":"2024-05-09T04:33:39.039318+0000","last_in_change":"2024-05-09T04:33:29.902361+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T04:33:39.795344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-09T04:34:40.995268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-09T04:34:42.107773+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22e550ad-847b-4952-80e6-63d859ed1a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6801","nonce":3182572912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6803","nonce":3182572912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6807","nonce":3182572912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6805","nonce":3182572912}]},"public_addr":"172.21.15.145:6801/3182572912","cluster_addr":"172.21.15.145:6803/3182572912","heartbeat_back_addr":"172.21.15.145:6807/3182572912","heartbeat_front_addr":"172.21.15.145:6805/3182572912","state":["exists","up"]},{"osd":1,"uuid":"0f6b9fca-0c41-4145-8d74-598ddb42f2a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6825","nonce":1845225555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6827","nonce":1845225555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6831","nonce":1845225555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6829","nonce":1845225555}]},"public_addr":"172.21.15.145:6825/1845225555","cluster_addr":"172.21.15.145:6827/1845225555","heartbeat_back_addr":"172.21.15.145:6831/1845225555","heartbeat_front_addr":"172.21.15.145:6829/1845225555","state":["exists","up"]},{"osd":2,"uuid":"b3b1f67e-5790-483f-8610-e05a5bce4c30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6817","nonce":2168958993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6819","nonce":2168958993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6823","nonce":2168958993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6821","nonce":2168958993}]},"public_addr":"172.21.15.145:6817/2168958993","cluster_addr":"172.21.15.145:6819/2168958993","heartbeat_back_addr":"172.21.15.145:6823/2168958993","heartbeat_front_addr":"172.21.15.145:6821/2168958993","state":["exists","up"]},{"osd":3,"uuid":"884787a5-e037-4d31-9aae-743030eba4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6809","nonce":337009826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6811","nonce":337009826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6815","nonce":337009826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6813","nonce":337009826}]},"public_addr":"172.21.15.145:6809/337009826","cluster_addr":"172.21.15.145:6811/337009826","heartbeat_back_addr":"172.21.15.145:6815/337009826","heartbeat_front_addr":"172.21.15.145:6813/337009826","state":["exists","up"]},{"osd":4,"uuid":"0759fc1e-c5fc-4a2f-86bb-e8eccd83f833","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6818","nonce":3117700995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6821","nonce":3117700995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6827","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6829","nonce":3117700995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6823","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6825","nonce":3117700995}]},"public_addr":"172.21.15.192:6818/3117700995","cluster_addr":"172.21.15.192:6821/3117700995","heartbeat_back_addr":"172.21.15.192:6829/3117700995","heartbeat_front_addr":"172.21.15.192:6825/3117700995","state":["exists","up"]},{"osd":5,"uuid":"9443cc22-9379-46e3-9173-98b592bf2c16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6801","nonce":1328373268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6803","nonce":1328373268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6807","nonce":1328373268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6805","nonce":1328373268}]},"public_addr":"172.21.15.192:6801/1328373268","cluster_addr":"172.21.15.192:6803/1328373268","heartbeat_back_addr":"172.21.15.192:6807/1328373268","heartbeat_front_addr":"172.21.15.192:6805/1328373268","state":["exists","up"]},{"osd":6,"uuid":"d05a3311-505a-45b5-85b9-aa3113e96987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6817","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6820","nonce":1868011057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6824","nonce":1868011057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6831","nonce":1868011057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6828","nonce":1868011057}]},"public_addr":"172.21.15.192:6820/1868011057","cluster_addr":"172.21.15.192:6824/1868011057","heartbeat_back_addr":"172.21.15.192:6831/1868011057","heartbeat_front_addr":"172.21.15.192:6828/1868011057","state":["exists","up"]},{"osd":7,"uuid":"c7c23dbc-2029-43ec-9f53-25f3d58019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6809","nonce":3368512157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6811","nonce":3368512157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6815","nonce":3368512157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6813","nonce":3368512157}]},"public_addr":"172.21.15.192:6809/3368512157","cluster_addr":"172.21.15.192:6811/3368512157","heartbeat_back_addr":"172.21.15.192:6815/3368512157","heartbeat_front_addr":"172.21.15.192:6813/3368512157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.091188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.032567+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:34.797468+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.228839+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.622400+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.473103+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.566092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.349395+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T04:34:44.539 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.539+0000 7f54e8a00640 1 -- 172.21.15.145:0/1642780719 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f54c40762d0 msgr2=0x7f54c4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:44.540 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.539+0000 7f54e8a00640 1 --2- 172.21.15.145:0/1642780719 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f54c40762d0 0x7f54c4078790 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f54d0034000 tx=0x7f54d002e000 comp rx=0 tx=0).stop 2024-05-09T04:34:44.540 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.539+0000 7f54e8a00640 1 -- 172.21.15.145:0/1642780719 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f54e40b4760 msgr2=0x7f54e414f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:44.540 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.539+0000 7f54e8a00640 1 --2- 172.21.15.145:0/1642780719 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f54e40b4760 0x7f54e414f460 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f54d400c450 tx=0x7f54d400c910 comp rx=0 tx=0).stop 2024-05-09T04:34:44.540 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.540+0000 7f54e8a00640 1 -- 172.21.15.145:0/1642780719 shutdown_connections 2024-05-09T04:34:44.540 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.540+0000 7f54e8a00640 1 --2- 172.21.15.145:0/1642780719 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f54c40762d0 0x7f54c4078790 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:44.540 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.540+0000 7f54e8a00640 1 --2- 172.21.15.145:0/1642780719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54e40d5ad0 0x7f54e415cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:44.540 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.540+0000 7f54e8a00640 1 --2- 172.21.15.145:0/1642780719 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f54e40d3180 0x7f54e415c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:44.540 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.540+0000 7f54e8a00640 1 --2- 172.21.15.145:0/1642780719 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f54e40b4760 0x7f54e414f460 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:44.540 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.540+0000 7f54e8a00640 1 -- 172.21.15.145:0/1642780719 >> 172.21.15.145:0/1642780719 conn(0x7f54e40d4fb0 msgr2=0x7f54e4162720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:44.540 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.540+0000 7f54e8a00640 1 -- 172.21.15.145:0/1642780719 shutdown_connections 2024-05-09T04:34:44.540 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.540+0000 7f54e8a00640 1 -- 172.21.15.145:0/1642780719 wait complete. 2024-05-09T04:34:44.558 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-09T04:34:44.761 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.761+0000 7fa5de400640 1 Processor -- start 2024-05-09T04:34:44.762 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.762+0000 7fa5de400640 1 -- start start 2024-05-09T04:34:44.764 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.763+0000 7fa5de400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa5d815aac0 0x7fa5d80b02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:44.764 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.763+0000 7fa5de400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fa5d806f360 0x7fa5d806f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:44.764 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.763+0000 7fa5de400640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5d80ade70 con 0x7fa5d806ffe0 2024-05-09T04:34:44.764 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.763+0000 7fa5de400640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fa5d80ae3f0 con 0x7fa5d806f360 2024-05-09T04:34:44.764 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.763+0000 7fa5de400640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fa5d8160bf0 con 0x7fa5d815aac0 2024-05-09T04:34:44.765 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.763+0000 7fa5dda00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fa5d806f360 0x7fa5d806f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:44.765 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.763+0000 7fa5d7e00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7fa5d806ffe0 0x7fa5d81613d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:49008/0 (socket says 172.21.15.145:49008) 2024-05-09T04:34:44.765 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.764+0000 7fa5d7e00640 1 -- 172.21.15.145:0/2093487486 learned_addr learned my addr 172.21.15.145:0/2093487486 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:34:44.765 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.764+0000 7fa5dd000640 1 --2- 172.21.15.145:0/2093487486 >> v2:172.21.15.192:3300/0 conn(0x7fa5d815aac0 0x7fa5d80b02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:44.765 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.764+0000 7fa5dda00640 1 -- 172.21.15.145:0/2093487486 >> v2:172.21.15.192:3300/0 conn(0x7fa5d815aac0 msgr2=0x7fa5d80b02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:44.765 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.764+0000 7fa5dda00640 1 --2- 172.21.15.145:0/2093487486 >> v2:172.21.15.192:3300/0 conn(0x7fa5d815aac0 0x7fa5d80b02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:44.765 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.764+0000 7fa5dda00640 1 -- 172.21.15.145:0/2093487486 >> v1:172.21.15.145:6789/0 conn(0x7fa5d806ffe0 legacy=0x7fa5d81613d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:44.765 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.764+0000 7fa5dda00640 1 -- 172.21.15.145:0/2093487486 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5d8070460 con 0x7fa5d806f360 2024-05-09T04:34:44.765 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.764+0000 7fa5d7400640 1 -- 172.21.15.145:0/2093487486 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2475381249 0 0) 0x7fa5d80ade70 con 0x7fa5d806ffe0 2024-05-09T04:34:44.765 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.764+0000 7fa5dd000640 1 --2- 172.21.15.145:0/2093487486 >> v2:172.21.15.192:3300/0 conn(0x7fa5d815aac0 0x7fa5d80b02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:34:44.766 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.765+0000 7fa5dda00640 1 --2- 172.21.15.145:0/2093487486 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa5d806f360 0x7fa5d806f810 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa5d806d9a0 tx=0x7fa5d002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bccde215ee5c58a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:44.766 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.765+0000 7fa5d7400640 1 -- 172.21.15.145:0/2093487486 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d0007740 con 0x7fa5d806f360 2024-05-09T04:34:44.767 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.765+0000 7fa5d7400640 1 -- 172.21.15.145:0/2093487486 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5d00078e0 con 0x7fa5d806f360 2024-05-09T04:34:44.767 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.765+0000 7fa5d7400640 1 -- 172.21.15.145:0/2093487486 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d0007cb0 con 0x7fa5d806f360 2024-05-09T04:34:44.767 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.766+0000 7fa5de400640 1 -- 172.21.15.145:0/2093487486 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa5d806f360 msgr2=0x7fa5d806f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:44.767 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.766+0000 7fa5de400640 1 --2- 172.21.15.145:0/2093487486 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa5d806f360 0x7fa5d806f810 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa5d806d9a0 tx=0x7fa5d002a5f0 comp rx=0 tx=0).stop 2024-05-09T04:34:44.767 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.766+0000 7fa5de400640 1 -- 172.21.15.145:0/2093487486 shutdown_connections 2024-05-09T04:34:44.767 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.766+0000 7fa5de400640 1 --2- 172.21.15.145:0/2093487486 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa5d806f360 0x7fa5d806f810 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:44.767 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.766+0000 7fa5de400640 1 --2- 172.21.15.145:0/2093487486 >> v2:172.21.15.192:3300/0 conn(0x7fa5d815aac0 0x7fa5d80b02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:44.767 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.766+0000 7fa5de400640 1 -- 172.21.15.145:0/2093487486 >> 172.21.15.145:0/2093487486 conn(0x7fa5d80b28f0 msgr2=0x7fa5d80b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:44.767 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.766+0000 7fa5de400640 1 -- 172.21.15.145:0/2093487486 shutdown_connections 2024-05-09T04:34:44.767 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.766+0000 7fa5de400640 1 -- 172.21.15.145:0/2093487486 wait complete. 2024-05-09T04:34:44.768 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.768+0000 7fa5de400640 1 Processor -- start 2024-05-09T04:34:44.768 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.768+0000 7fa5de400640 1 -- start start 2024-05-09T04:34:44.769 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.769+0000 7fa5de400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5d806f360 0x7fa5d814f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:44.769 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.769+0000 7fa5de400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa5d806ffe0 0x7fa5d80b5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:44.769 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.769+0000 7fa5de400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa5d815aac0 0x7fa5d815bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:44.769 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.769+0000 7fa5de400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d815d580 con 0x7fa5d815aac0 2024-05-09T04:34:44.769 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.769+0000 7fa5de400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d815d400 con 0x7fa5d806f360 2024-05-09T04:34:44.769 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.769+0000 7fa5de400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa5d815d700 con 0x7fa5d806ffe0 2024-05-09T04:34:44.770 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.769+0000 7fa5dda00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa5d815aac0 0x7fa5d815bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:44.770 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.769+0000 7fa5dda00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa5d815aac0 0x7fa5d815bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:59256/0 (socket says 172.21.15.145:59256) 2024-05-09T04:34:44.770 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.769+0000 7fa5dda00640 1 -- 172.21.15.145:0/2369714160 learned_addr learned my addr 172.21.15.145:0/2369714160 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:44.770 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.769+0000 7fa5dd000640 1 --2- 172.21.15.145:0/2369714160 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5d806f360 0x7fa5d814f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:44.770 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.769+0000 7fa5dda00640 1 -- 172.21.15.145:0/2369714160 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa5d806ffe0 msgr2=0x7fa5d80b5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:44.770 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.769+0000 7fa5dda00640 1 --2- 172.21.15.145:0/2369714160 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa5d806ffe0 0x7fa5d80b5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:44.770 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.769+0000 7fa5dda00640 1 -- 172.21.15.145:0/2369714160 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5d806f360 msgr2=0x7fa5d814f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:44.770 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.769+0000 7fa5dda00640 1 --2- 172.21.15.145:0/2369714160 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5d806f360 0x7fa5d814f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:44.770 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.769+0000 7fa5dda00640 1 -- 172.21.15.145:0/2369714160 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5d80b3d90 con 0x7fa5d815aac0 2024-05-09T04:34:44.770 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.770+0000 7fa5dda00640 1 --2- 172.21.15.145:0/2369714160 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa5d815aac0 0x7fa5d815bef0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa5d002abe0 tx=0x7fa5d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:44.771 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.770+0000 7fa5d4c00640 1 -- 172.21.15.145:0/2369714160 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d0040070 con 0x7fa5d815aac0 2024-05-09T04:34:44.771 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.770+0000 7fa5d4c00640 1 -- 172.21.15.145:0/2369714160 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5d0031c90 con 0x7fa5d815aac0 2024-05-09T04:34:44.771 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.770+0000 7fa5de400640 1 -- 172.21.15.145:0/2369714160 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5d80b4070 con 0x7fa5d815aac0 2024-05-09T04:34:44.771 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.770+0000 7fa5d4c00640 1 -- 172.21.15.145:0/2369714160 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d0045680 con 0x7fa5d815aac0 2024-05-09T04:34:44.771 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.770+0000 7fa5de400640 1 -- 172.21.15.145:0/2369714160 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5d80b4630 con 0x7fa5d815aac0 2024-05-09T04:34:44.771 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.771+0000 7fa5de400640 1 -- 172.21.15.145:0/2369714160 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5d815e350 con 0x7fa5d815aac0 2024-05-09T04:34:44.772 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.772+0000 7fa5d4c00640 1 -- 172.21.15.145:0/2369714160 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa5d0039040 con 0x7fa5d815aac0 2024-05-09T04:34:44.773 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.773+0000 7fa5d4c00640 1 --2- 172.21.15.145:0/2369714160 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fa5b80762d0 0x7fa5b8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:44.773 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.773+0000 7fa5d4c00640 1 -- 172.21.15.145:0/2369714160 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fa5d00b8ff0 con 0x7fa5d815aac0 2024-05-09T04:34:44.773 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.774+0000 7fa5dd000640 1 --2- 172.21.15.145:0/2369714160 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fa5b80762d0 0x7fa5b8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:44.774 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.774+0000 7fa5dd000640 1 --2- 172.21.15.145:0/2369714160 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fa5b80762d0 0x7fa5b8078790 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc00b840 tx=0x7fa5cc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:44.777 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:44.778+0000 7fa5d4c00640 1 -- 172.21.15.145:0/2369714160 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa5d815e350 con 0x7fa5d815aac0 2024-05-09T04:34:45.026 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.026+0000 7fa5de400640 1 -- 172.21.15.145:0/2369714160 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fa5d806a310 con 0x7fa5d815aac0 2024-05-09T04:34:45.692 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.692+0000 7fa5d4c00640 1 -- 172.21.15.145:0/2369714160 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7fa5d00829b0 con 0x7fa5d815aac0 2024-05-09T04:34:45.695 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.696+0000 7fa5de400640 1 -- 172.21.15.145:0/2369714160 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fa5b80762d0 msgr2=0x7fa5b8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:45.696 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.696+0000 7fa5de400640 1 --2- 172.21.15.145:0/2369714160 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fa5b80762d0 0x7fa5b8078790 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc00b840 tx=0x7fa5cc008040 comp rx=0 tx=0).stop 2024-05-09T04:34:45.696 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.696+0000 7fa5de400640 1 -- 172.21.15.145:0/2369714160 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa5d815aac0 msgr2=0x7fa5d815bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:45.696 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.696+0000 7fa5de400640 1 --2- 172.21.15.145:0/2369714160 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa5d815aac0 0x7fa5d815bef0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa5d002abe0 tx=0x7fa5d0004a40 comp rx=0 tx=0).stop 2024-05-09T04:34:45.697 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.696+0000 7fa5de400640 1 -- 172.21.15.145:0/2369714160 shutdown_connections 2024-05-09T04:34:45.697 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.696+0000 7fa5de400640 1 --2- 172.21.15.145:0/2369714160 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fa5b80762d0 0x7fa5b8078790 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:45.698 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.696+0000 7fa5de400640 1 --2- 172.21.15.145:0/2369714160 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa5d815aac0 0x7fa5d815bef0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:45.698 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.696+0000 7fa5de400640 1 --2- 172.21.15.145:0/2369714160 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa5d806ffe0 0x7fa5d80b5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:45.698 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.696+0000 7fa5de400640 1 --2- 172.21.15.145:0/2369714160 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa5d806f360 0x7fa5d814f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:45.698 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.696+0000 7fa5de400640 1 -- 172.21.15.145:0/2369714160 >> 172.21.15.145:0/2369714160 conn(0x7fa5d80b28f0 msgr2=0x7fa5d806eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:45.698 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.697+0000 7fa5de400640 1 -- 172.21.15.145:0/2369714160 shutdown_connections 2024-05-09T04:34:45.699 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.697+0000 7fa5de400640 1 -- 172.21.15.145:0/2369714160 wait complete. 2024-05-09T04:34:45.718 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-09T04:34:45.719 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-09T04:34:45.917 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.917+0000 7fe440a00640 1 Processor -- start 2024-05-09T04:34:45.917 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.918+0000 7fe440a00640 1 -- start start 2024-05-09T04:34:45.919 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.919+0000 7fe440a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fe43c15f610 0x7fe43c0d2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:45.919 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.919+0000 7fe440a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe43c0d58a0 0x7fe43c0d0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:45.919 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.919+0000 7fe440a00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe43c15f9f0 con 0x7fe43c0b5d70 2024-05-09T04:34:45.919 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.919+0000 7fe440a00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fe43c0d40c0 con 0x7fe43c15f610 2024-05-09T04:34:45.919 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.919+0000 7fe440a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe43c0b61d0 con 0x7fe43c0d58a0 2024-05-09T04:34:45.919 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.919+0000 7fe439e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fe43c15f610 0x7fe43c0d2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:45.919 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.919+0000 7fe439400640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7fe43c0b5d70 0x7fe43c0d5190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:49016/0 (socket says 172.21.15.145:49016) 2024-05-09T04:34:45.920 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.919+0000 7fe439400640 1 -- 172.21.15.145:0/3606203927 learned_addr learned my addr 172.21.15.145:0/3606203927 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:34:45.921 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.920+0000 7fe43a800640 1 --2- 172.21.15.145:0/3606203927 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe43c0d58a0 0x7fe43c0d0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:45.921 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.920+0000 7fe438a00640 1 -- 172.21.15.145:0/3606203927 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4079789769 0 0) 0x7fe43c15f9f0 con 0x7fe43c0b5d70 2024-05-09T04:34:45.921 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.920+0000 7fe439e00640 1 -- 172.21.15.145:0/3606203927 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe43c0d58a0 msgr2=0x7fe43c0d0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:45.921 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.920+0000 7fe439e00640 1 --2- 172.21.15.145:0/3606203927 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe43c0d58a0 0x7fe43c0d0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:45.921 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.920+0000 7fe439e00640 1 -- 172.21.15.145:0/3606203927 >> v1:172.21.15.145:6789/0 conn(0x7fe43c0b5d70 legacy=0x7fe43c0d5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:45.921 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.920+0000 7fe439e00640 1 -- 172.21.15.145:0/3606203927 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe43c0d34b0 con 0x7fe43c15f610 2024-05-09T04:34:45.921 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.921+0000 7fe439e00640 1 --2- 172.21.15.145:0/3606203927 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe43c15f610 0x7fe43c0d2dd0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe42c0097a0 tx=0x7fe42c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1348941cb6c5fcd1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:45.922 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.921+0000 7fe438a00640 1 -- 172.21.15.145:0/3606203927 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42c037070 con 0x7fe43c15f610 2024-05-09T04:34:45.922 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.921+0000 7fe438a00640 1 -- 172.21.15.145:0/3606203927 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe42c032440 con 0x7fe43c15f610 2024-05-09T04:34:45.922 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.921+0000 7fe438a00640 1 -- 172.21.15.145:0/3606203927 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42c032810 con 0x7fe43c15f610 2024-05-09T04:34:45.923 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.922+0000 7fe440a00640 1 -- 172.21.15.145:0/3606203927 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe43c15f610 msgr2=0x7fe43c0d2dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:45.923 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.922+0000 7fe440a00640 1 --2- 172.21.15.145:0/3606203927 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe43c15f610 0x7fe43c0d2dd0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe42c0097a0 tx=0x7fe42c02a210 comp rx=0 tx=0).stop 2024-05-09T04:34:45.923 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.922+0000 7fe440a00640 1 -- 172.21.15.145:0/3606203927 shutdown_connections 2024-05-09T04:34:45.923 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.922+0000 7fe440a00640 1 --2- 172.21.15.145:0/3606203927 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe43c0d58a0 0x7fe43c0d0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:45.923 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.922+0000 7fe440a00640 1 --2- 172.21.15.145:0/3606203927 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe43c15f610 0x7fe43c0d2dd0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:45.923 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.922+0000 7fe440a00640 1 -- 172.21.15.145:0/3606203927 >> 172.21.15.145:0/3606203927 conn(0x7fe43c08ec20 msgr2=0x7fe43c09e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:45.923 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.922+0000 7fe440a00640 1 -- 172.21.15.145:0/3606203927 shutdown_connections 2024-05-09T04:34:45.923 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.922+0000 7fe440a00640 1 -- 172.21.15.145:0/3606203927 wait complete. 2024-05-09T04:34:45.924 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.924+0000 7fe440a00640 1 Processor -- start 2024-05-09T04:34:45.924 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.924+0000 7fe440a00640 1 -- start start 2024-05-09T04:34:45.926 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.925+0000 7fe440a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe43c0b5d70 0x7fe43c0c9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:45.926 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.925+0000 7fe440a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe43c0d58a0 0x7fe43c15fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:45.927 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.925+0000 7fe440a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe43c15f610 0x7fe43c0cb110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:45.927 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.925+0000 7fe440a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe43c0d3d50 con 0x7fe43c0b5d70 2024-05-09T04:34:45.927 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.925+0000 7fe440a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe43c0d3a70 con 0x7fe43c15f610 2024-05-09T04:34:45.927 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.925+0000 7fe440a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe43c0d0590 con 0x7fe43c0d58a0 2024-05-09T04:34:45.927 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.925+0000 7fe439e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe43c0b5d70 0x7fe43c0c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:45.927 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.926+0000 7fe439e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe43c0b5d70 0x7fe43c0c9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:59278/0 (socket says 172.21.15.145:59278) 2024-05-09T04:34:45.927 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.926+0000 7fe439e00640 1 -- 172.21.15.145:0/251226618 learned_addr learned my addr 172.21.15.145:0/251226618 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:45.927 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.926+0000 7fe439400640 1 --2- 172.21.15.145:0/251226618 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe43c0d58a0 0x7fe43c15fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:45.927 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.926+0000 7fe439e00640 1 -- 172.21.15.145:0/251226618 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe43c0d58a0 msgr2=0x7fe43c15fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:45.927 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.926+0000 7fe439e00640 1 --2- 172.21.15.145:0/251226618 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe43c0d58a0 0x7fe43c15fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:45.927 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.926+0000 7fe439e00640 1 -- 172.21.15.145:0/251226618 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe43c15f610 msgr2=0x7fe43c0cb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:45.927 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.926+0000 7fe439e00640 1 --2- 172.21.15.145:0/251226618 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe43c15f610 0x7fe43c0cb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:45.927 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.926+0000 7fe439e00640 1 -- 172.21.15.145:0/251226618 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe43c0c9c80 con 0x7fe43c0b5d70 2024-05-09T04:34:45.928 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.926+0000 7fe439e00640 1 --2- 172.21.15.145:0/251226618 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe43c0b5d70 0x7fe43c0c9740 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe42c0098c0 tx=0x7fe42c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:45.928 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.927+0000 7fe432000640 1 -- 172.21.15.145:0/251226618 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42c037040 con 0x7fe43c0b5d70 2024-05-09T04:34:45.928 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.927+0000 7fe432000640 1 -- 172.21.15.145:0/251226618 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe42c030770 con 0x7fe43c0b5d70 2024-05-09T04:34:45.928 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.927+0000 7fe432000640 1 -- 172.21.15.145:0/251226618 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42c030b40 con 0x7fe43c0b5d70 2024-05-09T04:34:45.928 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.927+0000 7fe440a00640 1 -- 172.21.15.145:0/251226618 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe43c0b40e0 con 0x7fe43c0b5d70 2024-05-09T04:34:45.928 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.927+0000 7fe440a00640 1 -- 172.21.15.145:0/251226618 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe43c15c420 con 0x7fe43c0b5d70 2024-05-09T04:34:45.928 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.928+0000 7fe440a00640 1 -- 172.21.15.145:0/251226618 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe43c06a310 con 0x7fe43c0b5d70 2024-05-09T04:34:45.929 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.929+0000 7fe432000640 1 -- 172.21.15.145:0/251226618 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe42c044020 con 0x7fe43c0b5d70 2024-05-09T04:34:45.930 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.930+0000 7fe432000640 1 --2- 172.21.15.145:0/251226618 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fe4180762d0 0x7fe418078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:45.930 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.930+0000 7fe432000640 1 -- 172.21.15.145:0/251226618 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fe42c0b7b80 con 0x7fe43c0b5d70 2024-05-09T04:34:45.930 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.930+0000 7fe439400640 1 --2- 172.21.15.145:0/251226618 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fe4180762d0 0x7fe418078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:45.931 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.931+0000 7fe439400640 1 --2- 172.21.15.145:0/251226618 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fe4180762d0 0x7fe418078790 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe43c0d4700 tx=0x7fe424007b10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:45.935 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:45.935+0000 7fe432000640 1 -- 172.21.15.145:0/251226618 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe42c032e20 con 0x7fe43c0b5d70 2024-05-09T04:34:46.179 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.179+0000 7fe440a00640 1 -- 172.21.15.145:0/251226618 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fe43c0cf6c0 con 0x7fe43c0b5d70 2024-05-09T04:34:46.703 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.703+0000 7fe432000640 1 -- 172.21.15.145:0/251226618 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) ==== 122+0+0 (secure 0 0 0) 0x7fe42c081440 con 0x7fe43c0b5d70 2024-05-09T04:34:46.706 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.706+0000 7fe440a00640 1 -- 172.21.15.145:0/251226618 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fe4180762d0 msgr2=0x7fe418078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:46.707 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.706+0000 7fe440a00640 1 --2- 172.21.15.145:0/251226618 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fe4180762d0 0x7fe418078790 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe43c0d4700 tx=0x7fe424007b10 comp rx=0 tx=0).stop 2024-05-09T04:34:46.707 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.706+0000 7fe440a00640 1 -- 172.21.15.145:0/251226618 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe43c0b5d70 msgr2=0x7fe43c0c9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:46.707 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.706+0000 7fe440a00640 1 --2- 172.21.15.145:0/251226618 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe43c0b5d70 0x7fe43c0c9740 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe42c0098c0 tx=0x7fe42c0046c0 comp rx=0 tx=0).stop 2024-05-09T04:34:46.707 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.707+0000 7fe440a00640 1 -- 172.21.15.145:0/251226618 shutdown_connections 2024-05-09T04:34:46.707 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.707+0000 7fe440a00640 1 --2- 172.21.15.145:0/251226618 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fe4180762d0 0x7fe418078790 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:46.707 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.707+0000 7fe440a00640 1 --2- 172.21.15.145:0/251226618 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe43c15f610 0x7fe43c0cb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:46.707 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.707+0000 7fe440a00640 1 --2- 172.21.15.145:0/251226618 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe43c0d58a0 0x7fe43c15fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:46.707 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.707+0000 7fe440a00640 1 --2- 172.21.15.145:0/251226618 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe43c0b5d70 0x7fe43c0c9740 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:46.707 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.707+0000 7fe440a00640 1 -- 172.21.15.145:0/251226618 >> 172.21.15.145:0/251226618 conn(0x7fe43c08ec20 msgr2=0x7fe43c06d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:46.707 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.707+0000 7fe440a00640 1 -- 172.21.15.145:0/251226618 shutdown_connections 2024-05-09T04:34:46.708 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.707+0000 7fe440a00640 1 -- 172.21.15.145:0/251226618 wait complete. 2024-05-09T04:34:46.727 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-09T04:34:46.727 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T04:34:46.925 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.925+0000 7f6f11e00640 1 Processor -- start 2024-05-09T04:34:46.925 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.925+0000 7f6f11e00640 1 -- start start 2024-05-09T04:34:46.927 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.926+0000 7f6f11e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f6f0c15f340 0x7f6f0c06eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:46.927 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.927+0000 7f6f11e00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f0c06ddc0 con 0x7f6f0c163b10 2024-05-09T04:34:46.927 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.927+0000 7f6f11e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f0c0b4f50 con 0x7f6f0c163610 2024-05-09T04:34:46.927 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.927+0000 7f6f11e00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f6f0c06dba0 con 0x7f6f0c15f340 2024-05-09T04:34:46.927 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.927+0000 7f6f0aa00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f6f0c15f340 0x7f6f0c06eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:46.927 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.927+0000 7f6f03e00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f6f0c163b10 0x7f6f0c162f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:49032/0 (socket says 172.21.15.145:49032) 2024-05-09T04:34:46.927 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.927+0000 7f6f03e00640 1 -- 172.21.15.145:0/1184891804 learned_addr learned my addr 172.21.15.145:0/1184891804 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:34:46.927 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.927+0000 7f6f0aa00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f6f0c15f340 0x7f6f0c06eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:59280/0 (socket says 172.21.15.145:59280) 2024-05-09T04:34:46.928 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.927+0000 7f6f0aa00640 1 -- 172.21.15.145:0/1184891804 >> v1:172.21.15.192:6789/0 conn(0x7f6f0c163610 legacy=0x7f6f0c06d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:46.928 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.927+0000 7f6f0aa00640 1 -- 172.21.15.145:0/1184891804 >> v1:172.21.15.145:6789/0 conn(0x7f6f0c163b10 legacy=0x7f6f0c162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:46.928 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.927+0000 7f6f0aa00640 1 -- 172.21.15.145:0/1184891804 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f0c0b5230 con 0x7f6f0c15f340 2024-05-09T04:34:46.929 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.928+0000 7f6f0aa00640 1 --2- 172.21.15.145:0/1184891804 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6f0c15f340 0x7f6f0c06eae0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6efc0097a0 tx=0x7f6efc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcb7cf601c43222a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:46.929 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.929+0000 7f6f0a000640 1 -- 172.21.15.145:0/1184891804 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6efc037070 con 0x7f6f0c15f340 2024-05-09T04:34:46.929 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.929+0000 7f6f0a000640 1 -- 172.21.15.145:0/1184891804 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6efc032440 con 0x7f6f0c15f340 2024-05-09T04:34:46.929 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.929+0000 7f6f0a000640 1 -- 172.21.15.145:0/1184891804 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6efc032810 con 0x7f6f0c15f340 2024-05-09T04:34:46.929 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.929+0000 7f6f11e00640 1 -- 172.21.15.145:0/1184891804 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6f0c15f340 msgr2=0x7f6f0c06eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:46.929 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.929+0000 7f6f11e00640 1 --2- 172.21.15.145:0/1184891804 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6f0c15f340 0x7f6f0c06eae0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6efc0097a0 tx=0x7f6efc02a180 comp rx=0 tx=0).stop 2024-05-09T04:34:46.930 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.929+0000 7f6f11e00640 1 -- 172.21.15.145:0/1184891804 shutdown_connections 2024-05-09T04:34:46.930 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.929+0000 7f6f11e00640 1 --2- 172.21.15.145:0/1184891804 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6f0c15f340 0x7f6f0c06eae0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:46.930 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.929+0000 7f6f11e00640 1 -- 172.21.15.145:0/1184891804 >> 172.21.15.145:0/1184891804 conn(0x7f6f0c0d1790 msgr2=0x7f6f0c0d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:46.930 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.929+0000 7f6f11e00640 1 -- 172.21.15.145:0/1184891804 shutdown_connections 2024-05-09T04:34:46.930 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.930+0000 7f6f11e00640 1 -- 172.21.15.145:0/1184891804 wait complete. 2024-05-09T04:34:46.931 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.931+0000 7f6f11e00640 1 Processor -- start 2024-05-09T04:34:46.931 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.931+0000 7f6f11e00640 1 -- start start 2024-05-09T04:34:46.933 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.932+0000 7f6f11e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6f0c15f340 0x7f6f0c14d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:46.934 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.932+0000 7f6f11e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f0c163610 0x7f6f0c0d0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:46.934 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.932+0000 7f6f11e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f0c163b10 0x7f6f0c0af290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:46.934 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.932+0000 7f6f11e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0c15f720 con 0x7f6f0c15f340 2024-05-09T04:34:46.934 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.932+0000 7f6f11e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6f0c0b57f0 con 0x7f6f0c163b10 2024-05-09T04:34:46.934 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.932+0000 7f6f11e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f6f0c0b5d10 con 0x7f6f0c163610 2024-05-09T04:34:46.934 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.932+0000 7f6f0aa00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6f0c15f340 0x7f6f0c14d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:46.934 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.933+0000 7f6f0aa00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6f0c15f340 0x7f6f0c14d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:59290/0 (socket says 172.21.15.145:59290) 2024-05-09T04:34:46.934 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.933+0000 7f6f0aa00640 1 -- 172.21.15.145:0/1608894003 learned_addr learned my addr 172.21.15.145:0/1608894003 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:46.934 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.933+0000 7f6f0aa00640 1 -- 172.21.15.145:0/1608894003 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f0c163610 msgr2=0x7f6f0c0d0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:46.934 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.933+0000 7f6f0aa00640 1 --2- 172.21.15.145:0/1608894003 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f0c163610 0x7f6f0c0d0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:46.935 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.933+0000 7f6f0aa00640 1 -- 172.21.15.145:0/1608894003 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f0c163b10 msgr2=0x7f6f0c0af290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:46.935 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.933+0000 7f6f0b400640 1 --2- 172.21.15.145:0/1608894003 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f0c163b10 0x7f6f0c0af290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:46.935 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.933+0000 7f6f0aa00640 1 --2- 172.21.15.145:0/1608894003 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f0c163b10 0x7f6f0c0af290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:46.935 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.933+0000 7f6f0aa00640 1 -- 172.21.15.145:0/1608894003 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f0c161cc0 con 0x7f6f0c15f340 2024-05-09T04:34:46.935 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.933+0000 7f6f0aa00640 1 --2- 172.21.15.145:0/1608894003 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6f0c15f340 0x7f6f0c14d090 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6efc0098c0 tx=0x7f6efc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:46.935 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.933+0000 7f6f03400640 1 -- 172.21.15.145:0/1608894003 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6efc041070 con 0x7f6f0c15f340 2024-05-09T04:34:46.935 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.934+0000 7f6f03400640 1 -- 172.21.15.145:0/1608894003 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6efc03c930 con 0x7f6f0c15f340 2024-05-09T04:34:46.935 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.934+0000 7f6f03400640 1 -- 172.21.15.145:0/1608894003 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6efc037040 con 0x7f6f0c15f340 2024-05-09T04:34:46.935 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.934+0000 7f6f11e00640 1 -- 172.21.15.145:0/1608894003 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f0c161fa0 con 0x7f6f0c15f340 2024-05-09T04:34:46.935 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.934+0000 7f6f11e00640 1 -- 172.21.15.145:0/1608894003 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f0c162560 con 0x7f6f0c15f340 2024-05-09T04:34:46.936 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.935+0000 7f6f11e00640 1 -- 172.21.15.145:0/1608894003 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f0c06a310 con 0x7f6f0c15f340 2024-05-09T04:34:46.936 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.936+0000 7f6f03400640 1 -- 172.21.15.145:0/1608894003 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6efc044070 con 0x7f6f0c15f340 2024-05-09T04:34:46.937 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.937+0000 7f6f03400640 1 --2- 172.21.15.145:0/1608894003 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f6ee80762d0 0x7f6ee8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:46.937 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.937+0000 7f6f03400640 1 -- 172.21.15.145:0/1608894003 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f6efc0b8290 con 0x7f6f0c15f340 2024-05-09T04:34:46.937 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.937+0000 7f6f03e00640 1 --2- 172.21.15.145:0/1608894003 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f6ee80762d0 0x7f6ee8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:46.937 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.938+0000 7f6f03e00640 1 --2- 172.21.15.145:0/1608894003 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f6ee80762d0 0x7f6ee8078790 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c1eacf0 tx=0x7f6ef4007a40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:46.941 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:46.942+0000 7f6f03400640 1 -- 172.21.15.145:0/1608894003 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6efc081b50 con 0x7f6f0c15f340 2024-05-09T04:34:47.189 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.189+0000 7f6f11e00640 1 -- 172.21.15.145:0/1608894003 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6f0c0b4c90 con 0x7f6f0c15f340 2024-05-09T04:34:47.190 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.190+0000 7f6f03400640 1 -- 172.21.15.145:0/1608894003 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5187 (secure 0 0 0) 0x7f6efc081450 con 0x7f6f0c15f340 2024-05-09T04:34:47.190 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:34:47.190 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4499,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6834/3227728059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3227728059},{"type":"v1","addr":"172.21.15.192:6834","nonce":3227728059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4502,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1603013787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6833","nonce":1603013787},{"type":"v1","addr":"172.21.15.192:6835","nonce":1603013787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4672,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/68778418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":68778418},{"type":"v1","addr":"172.21.15.145:6837","nonce":68778418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T04:34:43.133970+0000","modified":"2024-05-09T04:34:46.181657+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4761},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4761":{"gid":4761,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.145:6835/4260798843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4260798843},{"type":"v1","addr":"172.21.15.145:6835","nonce":4260798843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4761,"qdb_cluster":[4761]},"id":2}]} 2024-05-09T04:34:47.191 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 17 2024-05-09T04:34:47.193 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.193+0000 7f6f11e00640 1 -- 172.21.15.145:0/1608894003 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f6ee80762d0 msgr2=0x7f6ee8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:47.193 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.193+0000 7f6f11e00640 1 --2- 172.21.15.145:0/1608894003 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f6ee80762d0 0x7f6ee8078790 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c1eacf0 tx=0x7f6ef4007a40 comp rx=0 tx=0).stop 2024-05-09T04:34:47.193 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.193+0000 7f6f11e00640 1 -- 172.21.15.145:0/1608894003 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6f0c15f340 msgr2=0x7f6f0c14d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:47.193 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.193+0000 7f6f11e00640 1 --2- 172.21.15.145:0/1608894003 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6f0c15f340 0x7f6f0c14d090 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6efc0098c0 tx=0x7f6efc0046c0 comp rx=0 tx=0).stop 2024-05-09T04:34:47.196 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.193+0000 7f6f11e00640 1 -- 172.21.15.145:0/1608894003 shutdown_connections 2024-05-09T04:34:47.196 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.193+0000 7f6f11e00640 1 --2- 172.21.15.145:0/1608894003 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f6ee80762d0 0x7f6ee8078790 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:47.196 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.193+0000 7f6f11e00640 1 --2- 172.21.15.145:0/1608894003 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f0c163b10 0x7f6f0c0af290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:47.196 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.193+0000 7f6f11e00640 1 --2- 172.21.15.145:0/1608894003 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f0c163610 0x7f6f0c0d0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:47.196 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.193+0000 7f6f11e00640 1 --2- 172.21.15.145:0/1608894003 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6f0c15f340 0x7f6f0c14d090 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:47.196 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.193+0000 7f6f11e00640 1 -- 172.21.15.145:0/1608894003 >> 172.21.15.145:0/1608894003 conn(0x7f6f0c0d1790 msgr2=0x7f6f0c06d180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:47.196 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.193+0000 7f6f11e00640 1 -- 172.21.15.145:0/1608894003 shutdown_connections 2024-05-09T04:34:47.196 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.193+0000 7f6f11e00640 1 -- 172.21.15.145:0/1608894003 wait complete. 2024-05-09T04:34:47.211 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T04:34:47.409 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.409+0000 7fc35e800640 1 Processor -- start 2024-05-09T04:34:47.409 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.409+0000 7fc35e800640 1 -- start start 2024-05-09T04:34:47.411 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.410+0000 7fc35e800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fc35806f0f0 0x7fc3580b4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:47.411 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.410+0000 7fc35e800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc35815bac0 0x7fc35806f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:47.411 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.411+0000 7fc35e800640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc35806fa10 con 0x7fc3580738c0 2024-05-09T04:34:47.411 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.411+0000 7fc35e800640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fc3580cf4b0 con 0x7fc35806f0f0 2024-05-09T04:34:47.411 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.411+0000 7fc35e800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fc3580b4ec0 con 0x7fc35815bac0 2024-05-09T04:34:47.411 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.411+0000 7fc357400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fc35806f0f0 0x7fc3580b4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:47.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.411+0000 7fc357400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fc35806f0f0 0x7fc3580b4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:59304/0 (socket says 172.21.15.145:59304) 2024-05-09T04:34:47.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.411+0000 7fc357400640 1 -- 172.21.15.145:0/2522465613 learned_addr learned my addr 172.21.15.145:0/2522465613 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:47.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.411+0000 7fc34fe00640 1 --2- 172.21.15.145:0/2522465613 >> v2:172.21.15.192:3300/0 conn(0x7fc35815bac0 0x7fc35806f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:47.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.411+0000 7fc357400640 1 -- 172.21.15.145:0/2522465613 >> v2:172.21.15.192:3300/0 conn(0x7fc35815bac0 msgr2=0x7fc35806f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:47.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.411+0000 7fc357400640 1 --2- 172.21.15.145:0/2522465613 >> v2:172.21.15.192:3300/0 conn(0x7fc35815bac0 0x7fc35806f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:47.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.411+0000 7fc357400640 1 -- 172.21.15.145:0/2522465613 >> v1:172.21.15.192:6789/0 conn(0x7fc3580738c0 legacy=0x7fc358072220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:47.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.411+0000 7fc357400640 1 -- 172.21.15.145:0/2522465613 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3580b5af0 con 0x7fc35806f0f0 2024-05-09T04:34:47.413 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.412+0000 7fc357400640 1 --2- 172.21.15.145:0/2522465613 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc35806f0f0 0x7fc3580b4980 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc358074110 tx=0x7fc34802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82d96aaa9470668c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:47.413 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.413+0000 7fc356a00640 1 -- 172.21.15.145:0/2522465613 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc348037070 con 0x7fc35806f0f0 2024-05-09T04:34:47.413 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.413+0000 7fc356a00640 1 -- 172.21.15.145:0/2522465613 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc348032440 con 0x7fc35806f0f0 2024-05-09T04:34:47.413 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.413+0000 7fc356a00640 1 -- 172.21.15.145:0/2522465613 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc348032810 con 0x7fc35806f0f0 2024-05-09T04:34:47.413 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.413+0000 7fc35e800640 1 -- 172.21.15.145:0/2522465613 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc35806f0f0 msgr2=0x7fc3580b4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:47.414 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.413+0000 7fc35e800640 1 --2- 172.21.15.145:0/2522465613 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc35806f0f0 0x7fc3580b4980 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc358074110 tx=0x7fc34802a1e0 comp rx=0 tx=0).stop 2024-05-09T04:34:47.414 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.413+0000 7fc35e800640 1 -- 172.21.15.145:0/2522465613 shutdown_connections 2024-05-09T04:34:47.414 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.413+0000 7fc35e800640 1 --2- 172.21.15.145:0/2522465613 >> v2:172.21.15.192:3300/0 conn(0x7fc35815bac0 0x7fc35806f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:47.414 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.413+0000 7fc35e800640 1 --2- 172.21.15.145:0/2522465613 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc35806f0f0 0x7fc3580b4980 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:47.414 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.413+0000 7fc35e800640 1 -- 172.21.15.145:0/2522465613 >> 172.21.15.145:0/2522465613 conn(0x7fc35806c870 msgr2=0x7fc35806cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:47.414 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.414+0000 7fc35e800640 1 -- 172.21.15.145:0/2522465613 shutdown_connections 2024-05-09T04:34:47.414 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.414+0000 7fc35e800640 1 -- 172.21.15.145:0/2522465613 wait complete. 2024-05-09T04:34:47.415 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.415+0000 7fc35e800640 1 Processor -- start 2024-05-09T04:34:47.415 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.415+0000 7fc35e800640 1 -- start start 2024-05-09T04:34:47.417 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.416+0000 7fc35e800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc35806f0f0 0x7fc3581529c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:47.417 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.416+0000 7fc35e800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc3580738c0 0x7fc358142520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:47.417 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.416+0000 7fc35e800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc35815bac0 0x7fc3581402a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:47.417 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.416+0000 7fc35e800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc3580b6390 con 0x7fc35806f0f0 2024-05-09T04:34:47.417 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.416+0000 7fc35e800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc3580b60b0 con 0x7fc35815bac0 2024-05-09T04:34:47.417 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.416+0000 7fc35e800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc3580cf140 con 0x7fc3580738c0 2024-05-09T04:34:47.417 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.417+0000 7fc357400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc35806f0f0 0x7fc3581529c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:47.417 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.417+0000 7fc357400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc35806f0f0 0x7fc3581529c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:59310/0 (socket says 172.21.15.145:59310) 2024-05-09T04:34:47.417 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.417+0000 7fc357400640 1 -- 172.21.15.145:0/1558098925 learned_addr learned my addr 172.21.15.145:0/1558098925 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:47.417 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.417+0000 7fc357400640 1 -- 172.21.15.145:0/1558098925 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc3580738c0 msgr2=0x7fc358142520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:47.418 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.417+0000 7fc357400640 1 --2- 172.21.15.145:0/1558098925 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc3580738c0 0x7fc358142520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:47.418 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.417+0000 7fc357400640 1 -- 172.21.15.145:0/1558098925 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc35815bac0 msgr2=0x7fc3581402a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:47.418 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.417+0000 7fc357e00640 1 --2- 172.21.15.145:0/1558098925 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc35815bac0 0x7fc3581402a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:47.418 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.417+0000 7fc357400640 1 --2- 172.21.15.145:0/1558098925 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc35815bac0 0x7fc3581402a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:47.418 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.417+0000 7fc357400640 1 -- 172.21.15.145:0/1558098925 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3580b3a90 con 0x7fc35806f0f0 2024-05-09T04:34:47.418 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.417+0000 7fc357400640 1 --2- 172.21.15.145:0/1558098925 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc35806f0f0 0x7fc3581529c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc348000c00 tx=0x7fc3480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:47.419 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.418+0000 7fc34f400640 1 -- 172.21.15.145:0/1558098925 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc348041020 con 0x7fc35806f0f0 2024-05-09T04:34:47.419 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.418+0000 7fc34f400640 1 -- 172.21.15.145:0/1558098925 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc348030970 con 0x7fc35806f0f0 2024-05-09T04:34:47.419 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.418+0000 7fc34f400640 1 -- 172.21.15.145:0/1558098925 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc348037040 con 0x7fc35806f0f0 2024-05-09T04:34:47.419 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.418+0000 7fc35e800640 1 -- 172.21.15.145:0/1558098925 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3580b3d70 con 0x7fc35806f0f0 2024-05-09T04:34:47.419 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.418+0000 7fc35e800640 1 -- 172.21.15.145:0/1558098925 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc358142000 con 0x7fc35806f0f0 2024-05-09T04:34:47.419 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.419+0000 7fc35e800640 1 -- 172.21.15.145:0/1558098925 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc358141b30 con 0x7fc35806f0f0 2024-05-09T04:34:47.419 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.420+0000 7fc34f400640 1 -- 172.21.15.145:0/1558098925 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc348044020 con 0x7fc35806f0f0 2024-05-09T04:34:47.421 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.421+0000 7fc34f400640 1 --2- 172.21.15.145:0/1558098925 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fc3340762d0 0x7fc334078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:47.421 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.422+0000 7fc34f400640 1 -- 172.21.15.145:0/1558098925 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fc3480b7ff0 con 0x7fc35806f0f0 2024-05-09T04:34:47.421 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.422+0000 7fc34fe00640 1 --2- 172.21.15.145:0/1558098925 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fc3340762d0 0x7fc334078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:47.422 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.422+0000 7fc34fe00640 1 --2- 172.21.15.145:0/1558098925 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fc3340762d0 0x7fc334078790 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc358072bb0 tx=0x7fc340008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:47.425 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.426+0000 7fc34f400640 1 -- 172.21.15.145:0/1558098925 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc34804b030 con 0x7fc35806f0f0 2024-05-09T04:34:47.483 INFO:tasks.ceph.mds.c.smithi145.stderr:2024-05-09T04:34:47.483+0000 7fd003c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-09T04:34:47.636 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.636+0000 7fc35e800640 1 -- 172.21.15.145:0/1558098925 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc35806a310 con 0x7fc35806f0f0 2024-05-09T04:34:47.639 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.639+0000 7fc34f400640 1 -- 172.21.15.145:0/1558098925 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16717 (secure 0 0 0) 0x7fc348081980 con 0x7fc35806f0f0 2024-05-09T04:34:47.640 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:34:47.640 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":32,"fsid":"957ee6c1-073b-44cf-9b93-9491b3fdfc1d","created":"2024-05-09T04:33:26.544367+0000","modified":"2024-05-09T04:34:44.671523+0000","last_up_change":"2024-05-09T04:33:39.039318+0000","last_in_change":"2024-05-09T04:33:29.902361+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T04:33:39.795344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-09T04:34:40.995268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-09T04:34:42.107773+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22e550ad-847b-4952-80e6-63d859ed1a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6801","nonce":3182572912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6803","nonce":3182572912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6807","nonce":3182572912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6805","nonce":3182572912}]},"public_addr":"172.21.15.145:6801/3182572912","cluster_addr":"172.21.15.145:6803/3182572912","heartbeat_back_addr":"172.21.15.145:6807/3182572912","heartbeat_front_addr":"172.21.15.145:6805/3182572912","state":["exists","up"]},{"osd":1,"uuid":"0f6b9fca-0c41-4145-8d74-598ddb42f2a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6825","nonce":1845225555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6827","nonce":1845225555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6831","nonce":1845225555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6829","nonce":1845225555}]},"public_addr":"172.21.15.145:6825/1845225555","cluster_addr":"172.21.15.145:6827/1845225555","heartbeat_back_addr":"172.21.15.145:6831/1845225555","heartbeat_front_addr":"172.21.15.145:6829/1845225555","state":["exists","up"]},{"osd":2,"uuid":"b3b1f67e-5790-483f-8610-e05a5bce4c30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6817","nonce":2168958993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6819","nonce":2168958993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6823","nonce":2168958993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6821","nonce":2168958993}]},"public_addr":"172.21.15.145:6817/2168958993","cluster_addr":"172.21.15.145:6819/2168958993","heartbeat_back_addr":"172.21.15.145:6823/2168958993","heartbeat_front_addr":"172.21.15.145:6821/2168958993","state":["exists","up"]},{"osd":3,"uuid":"884787a5-e037-4d31-9aae-743030eba4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6809","nonce":337009826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6811","nonce":337009826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6815","nonce":337009826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6813","nonce":337009826}]},"public_addr":"172.21.15.145:6809/337009826","cluster_addr":"172.21.15.145:6811/337009826","heartbeat_back_addr":"172.21.15.145:6815/337009826","heartbeat_front_addr":"172.21.15.145:6813/337009826","state":["exists","up"]},{"osd":4,"uuid":"0759fc1e-c5fc-4a2f-86bb-e8eccd83f833","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6818","nonce":3117700995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6821","nonce":3117700995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6827","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6829","nonce":3117700995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6823","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6825","nonce":3117700995}]},"public_addr":"172.21.15.192:6818/3117700995","cluster_addr":"172.21.15.192:6821/3117700995","heartbeat_back_addr":"172.21.15.192:6829/3117700995","heartbeat_front_addr":"172.21.15.192:6825/3117700995","state":["exists","up"]},{"osd":5,"uuid":"9443cc22-9379-46e3-9173-98b592bf2c16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6801","nonce":1328373268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6803","nonce":1328373268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6807","nonce":1328373268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6805","nonce":1328373268}]},"public_addr":"172.21.15.192:6801/1328373268","cluster_addr":"172.21.15.192:6803/1328373268","heartbeat_back_addr":"172.21.15.192:6807/1328373268","heartbeat_front_addr":"172.21.15.192:6805/1328373268","state":["exists","up"]},{"osd":6,"uuid":"d05a3311-505a-45b5-85b9-aa3113e96987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6817","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6820","nonce":1868011057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6824","nonce":1868011057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6831","nonce":1868011057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6828","nonce":1868011057}]},"public_addr":"172.21.15.192:6820/1868011057","cluster_addr":"172.21.15.192:6824/1868011057","heartbeat_back_addr":"172.21.15.192:6831/1868011057","heartbeat_front_addr":"172.21.15.192:6828/1868011057","state":["exists","up"]},{"osd":7,"uuid":"c7c23dbc-2029-43ec-9f53-25f3d58019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6809","nonce":3368512157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6811","nonce":3368512157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6815","nonce":3368512157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6813","nonce":3368512157}]},"public_addr":"172.21.15.192:6809/3368512157","cluster_addr":"172.21.15.192:6811/3368512157","heartbeat_back_addr":"172.21.15.192:6815/3368512157","heartbeat_front_addr":"172.21.15.192:6813/3368512157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.091188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.032567+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:34.797468+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.228839+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.622400+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.473103+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.566092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.349395+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T04:34:47.642 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.642+0000 7fc35e800640 1 -- 172.21.15.145:0/1558098925 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fc3340762d0 msgr2=0x7fc334078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:47.642 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.642+0000 7fc35e800640 1 --2- 172.21.15.145:0/1558098925 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fc3340762d0 0x7fc334078790 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc358072bb0 tx=0x7fc340008040 comp rx=0 tx=0).stop 2024-05-09T04:34:47.642 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.642+0000 7fc35e800640 1 -- 172.21.15.145:0/1558098925 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc35806f0f0 msgr2=0x7fc3581529c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:47.642 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.642+0000 7fc35e800640 1 --2- 172.21.15.145:0/1558098925 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc35806f0f0 0x7fc3581529c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc348000c00 tx=0x7fc3480046c0 comp rx=0 tx=0).stop 2024-05-09T04:34:47.642 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.642+0000 7fc35e800640 1 -- 172.21.15.145:0/1558098925 shutdown_connections 2024-05-09T04:34:47.642 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.642+0000 7fc35e800640 1 --2- 172.21.15.145:0/1558098925 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fc3340762d0 0x7fc334078790 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:47.642 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.642+0000 7fc35e800640 1 --2- 172.21.15.145:0/1558098925 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc35815bac0 0x7fc3581402a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:47.643 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.642+0000 7fc35e800640 1 --2- 172.21.15.145:0/1558098925 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc3580738c0 0x7fc358142520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:47.643 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.642+0000 7fc35e800640 1 --2- 172.21.15.145:0/1558098925 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc35806f0f0 0x7fc3581529c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:47.643 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.642+0000 7fc35e800640 1 -- 172.21.15.145:0/1558098925 >> 172.21.15.145:0/1558098925 conn(0x7fc35806c870 msgr2=0x7fc3580b0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:47.643 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.642+0000 7fc35e800640 1 -- 172.21.15.145:0/1558098925 shutdown_connections 2024-05-09T04:34:47.643 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.642+0000 7fc35e800640 1 -- 172.21.15.145:0/1558098925 wait complete. 2024-05-09T04:34:47.660 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-09T04:34:47.660 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T04:34:47.846 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.846+0000 7f7246e00640 1 Processor -- start 2024-05-09T04:34:47.846 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.846+0000 7f7246e00640 1 -- start start 2024-05-09T04:34:47.847 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.847+0000 7f7246e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f724006e860 0x7f724006ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:47.847 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.847+0000 7f7246e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f72400b4910 0x7f72400b4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:47.847 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.847+0000 7f7246e00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f724006f180 con 0x7f724015d780 2024-05-09T04:34:47.849 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.847+0000 7f7246e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f7240161360 con 0x7f72400b4910 2024-05-09T04:34:47.849 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.847+0000 7f7246e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f72401615c0 con 0x7f724006e860 2024-05-09T04:34:47.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.848+0000 7f7237400640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f724015d780 0x7f7240160c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:49036/0 (socket says 172.21.15.145:49036) 2024-05-09T04:34:47.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.848+0000 7f7237400640 1 -- 172.21.15.145:0/588788276 learned_addr learned my addr 172.21.15.145:0/588788276 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:34:47.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.848+0000 7f7244e00640 1 --2- 172.21.15.145:0/588788276 >> v2:172.21.15.192:3300/0 conn(0x7f72400b4910 0x7f72400b4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:47.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.848+0000 7f723f400640 1 -- 172.21.15.145:0/588788276 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1824491826 0 0) 0x7f724006f180 con 0x7f724015d780 2024-05-09T04:34:47.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.848+0000 7f723fe00640 1 --2- 172.21.15.145:0/588788276 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f724006e860 0x7f724006ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:47.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.848+0000 7f723f400640 1 -- 172.21.15.145:0/588788276 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7224003520 con 0x7f724015d780 2024-05-09T04:34:47.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.849+0000 7f723f400640 1 -- 172.21.15.145:0/588788276 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 39273010 0 0) 0x7f7224003520 con 0x7f724015d780 2024-05-09T04:34:47.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.849+0000 7f723f400640 1 -- 172.21.15.145:0/588788276 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f724006e860 msgr2=0x7f724006ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:47.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.849+0000 7f723f400640 1 --2- 172.21.15.145:0/588788276 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f724006e860 0x7f724006ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:47.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.849+0000 7f723f400640 1 -- 172.21.15.145:0/588788276 >> v2:172.21.15.192:3300/0 conn(0x7f72400b4910 msgr2=0x7f72400b4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:47.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.849+0000 7f723f400640 1 --2- 172.21.15.145:0/588788276 >> v2:172.21.15.192:3300/0 conn(0x7f72400b4910 0x7f72400b4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:47.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.849+0000 7f723f400640 1 -- 172.21.15.145:0/588788276 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72401618a0 con 0x7f724015d780 2024-05-09T04:34:47.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.849+0000 7f723f400640 1 -- 172.21.15.145:0/588788276 <== mon.0 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3474413261 0 0) 0x7f72280016f0 con 0x7f724015d780 2024-05-09T04:34:47.851 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.849+0000 7f723f400640 1 -- 172.21.15.145:0/588788276 >> v1:172.21.15.145:6789/0 conn(0x7f724015d780 legacy=0x7f7240160c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:47.851 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.849+0000 7f723f400640 1 --2- 172.21.15.145:0/588788276 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7224003a50 0x7f7224003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:47.851 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.849+0000 7f723f400640 1 --2- 172.21.15.145:0/588788276 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7224004800 0x7f7224004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:47.851 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.849+0000 7f723f400640 1 --2- 172.21.15.145:0/588788276 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f724006e860 0x7f72240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:47.851 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.849+0000 7f723f400640 1 -- 172.21.15.145:0/588788276 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7240161360 con 0x7f7224004800 2024-05-09T04:34:47.851 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.849+0000 7f723f400640 1 -- 172.21.15.145:0/588788276 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f72401615c0 con 0x7f7224003a50 2024-05-09T04:34:47.851 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.849+0000 7f723f400640 1 -- 172.21.15.145:0/588788276 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f722400bac0 con 0x7f724006e860 2024-05-09T04:34:47.851 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.849+0000 7f7237400640 1 --2- 172.21.15.145:0/588788276 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7224004800 0x7f7224004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:47.851 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.849+0000 7f723fe00640 1 --2- 172.21.15.145:0/588788276 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7224003a50 0x7f7224003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:47.851 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.849+0000 7f7237400640 1 -- 172.21.15.145:0/588788276 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f724006e860 msgr2=0x7f72240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:47.851 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.849+0000 7f7237400640 1 --2- 172.21.15.145:0/588788276 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f724006e860 0x7f72240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:47.851 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.849+0000 7f7237400640 1 -- 172.21.15.145:0/588788276 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7224003a50 msgr2=0x7f7224003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:47.851 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.849+0000 7f7237400640 1 --2- 172.21.15.145:0/588788276 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7224003a50 0x7f7224003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:47.852 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.850+0000 7f7237400640 1 -- 172.21.15.145:0/588788276 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7224003520 con 0x7f7224004800 2024-05-09T04:34:47.852 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.850+0000 7f723fe00640 1 --2- 172.21.15.145:0/588788276 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7224003a50 0x7f7224003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:34:47.853 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.850+0000 7f7237400640 1 --2- 172.21.15.145:0/588788276 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7224004800 0x7f7224004c80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f724015dba0 tx=0x7f7224005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7179bb0b17e86ed2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:47.853 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.851+0000 7f723f400640 1 -- 172.21.15.145:0/588788276 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7228007ce0 con 0x7f7224004800 2024-05-09T04:34:47.853 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.851+0000 7f723f400640 1 -- 172.21.15.145:0/588788276 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7228005af0 con 0x7f7224004800 2024-05-09T04:34:47.853 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.851+0000 7f723f400640 1 -- 172.21.15.145:0/588788276 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7228007070 con 0x7f7224004800 2024-05-09T04:34:47.853 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.851+0000 7f7246e00640 1 -- 172.21.15.145:0/588788276 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7224004800 msgr2=0x7f7224004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:47.853 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.851+0000 7f7246e00640 1 --2- 172.21.15.145:0/588788276 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7224004800 0x7f7224004c80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f724015dba0 tx=0x7f7224005830 comp rx=0 tx=0).stop 2024-05-09T04:34:47.853 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.851+0000 7f7244e00640 1 -- 172.21.15.145:0/588788276 reap_dead start 2024-05-09T04:34:47.854 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.851+0000 7f7246e00640 1 -- 172.21.15.145:0/588788276 shutdown_connections 2024-05-09T04:34:47.854 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.851+0000 7f7246e00640 1 -- 172.21.15.145:0/588788276 >> 172.21.15.145:0/588788276 conn(0x7f724006c870 msgr2=0x7f724006cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:47.854 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.852+0000 7f7246e00640 1 -- 172.21.15.145:0/588788276 shutdown_connections 2024-05-09T04:34:47.854 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.852+0000 7f7246e00640 1 -- 172.21.15.145:0/588788276 wait complete. 2024-05-09T04:34:47.854 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.853+0000 7f7246e00640 1 Processor -- start 2024-05-09T04:34:47.854 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.854+0000 7f7246e00640 1 -- start start 2024-05-09T04:34:47.856 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.854+0000 7f7246e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7224004800 0x7f72401f5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:47.856 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.854+0000 7f7246e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7240145ac0 0x7f724015c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:47.856 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.855+0000 7f7246e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f724013cb40 0x7f724013ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:47.856 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.855+0000 7f7246e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7240146c00 con 0x7f7240145ac0 2024-05-09T04:34:47.856 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.855+0000 7f7246e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f72400b43f0 con 0x7f724013cb40 2024-05-09T04:34:47.857 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.855+0000 7f7246e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f7240146d80 con 0x7f7224004800 2024-05-09T04:34:47.857 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.855+0000 7f7237400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7240145ac0 0x7f724015c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:47.857 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.855+0000 7f7237400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7240145ac0 0x7f724015c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:59336/0 (socket says 172.21.15.145:59336) 2024-05-09T04:34:47.857 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.855+0000 7f7237400640 1 -- 172.21.15.145:0/3179193861 learned_addr learned my addr 172.21.15.145:0/3179193861 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:47.857 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.855+0000 7f7244e00640 1 --2- 172.21.15.145:0/3179193861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f724013cb40 0x7f724013ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:47.857 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.855+0000 7f723fe00640 1 --2- 172.21.15.145:0/3179193861 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7224004800 0x7f72401f5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:47.857 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.855+0000 7f7237400640 1 -- 172.21.15.145:0/3179193861 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7224004800 msgr2=0x7f72401f5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:47.857 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.855+0000 7f7237400640 1 --2- 172.21.15.145:0/3179193861 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7224004800 0x7f72401f5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:47.857 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.855+0000 7f7237400640 1 -- 172.21.15.145:0/3179193861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f724013cb40 msgr2=0x7f724013ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:47.857 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.855+0000 7f7237400640 1 --2- 172.21.15.145:0/3179193861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f724013cb40 0x7f724013ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:47.857 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.855+0000 7f7237400640 1 -- 172.21.15.145:0/3179193861 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72401f5520 con 0x7f7240145ac0 2024-05-09T04:34:47.857 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.856+0000 7f7237400640 1 --2- 172.21.15.145:0/3179193861 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7240145ac0 0x7f724015c910 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f722800c300 tx=0x7f7240161a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:47.857 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.856+0000 7f723cc00640 1 -- 172.21.15.145:0/3179193861 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7228043070 con 0x7f7240145ac0 2024-05-09T04:34:47.857 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.856+0000 7f723cc00640 1 -- 172.21.15.145:0/3179193861 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7228038040 con 0x7f7240145ac0 2024-05-09T04:34:47.857 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.856+0000 7f723cc00640 1 -- 172.21.15.145:0/3179193861 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7228007290 con 0x7f7240145ac0 2024-05-09T04:34:47.858 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.856+0000 7f7246e00640 1 -- 172.21.15.145:0/3179193861 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f724013d450 con 0x7f7240145ac0 2024-05-09T04:34:47.858 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.856+0000 7f7246e00640 1 -- 172.21.15.145:0/3179193861 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72401f5aa0 con 0x7f7240145ac0 2024-05-09T04:34:47.858 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.857+0000 7f7246e00640 1 -- 172.21.15.145:0/3179193861 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f724006a310 con 0x7f7240145ac0 2024-05-09T04:34:47.858 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.858+0000 7f723cc00640 1 -- 172.21.15.145:0/3179193861 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7228007430 con 0x7f7240145ac0 2024-05-09T04:34:47.859 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.859+0000 7f723cc00640 1 --2- 172.21.15.145:0/3179193861 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f721c0762d0 0x7f721c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:47.859 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.860+0000 7f723cc00640 1 -- 172.21.15.145:0/3179193861 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f72280b7120 con 0x7f7240145ac0 2024-05-09T04:34:47.859 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.860+0000 7f723fe00640 1 --2- 172.21.15.145:0/3179193861 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f721c0762d0 0x7f721c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:47.860 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.860+0000 7f723fe00640 1 --2- 172.21.15.145:0/3179193861 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f721c0762d0 0x7f721c078790 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7230006fd0 tx=0x7f7230008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:47.863 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:47.863+0000 7f723cc00640 1 -- 172.21.15.145:0/3179193861 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f722807a990 con 0x7f7240145ac0 2024-05-09T04:34:48.057 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.057+0000 7f7246e00640 1 -- 172.21.15.145:0/3179193861 --> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7240141720 con 0x7f721c0762d0 2024-05-09T04:34:48.077 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.077+0000 7f723cc00640 1 -- 172.21.15.145:0/3179193861 <== mgr.4104 v2:172.21.15.145:6832/2725855396 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251910 (secure 0 0 0) 0x7f7240141720 con 0x7f721c0762d0 2024-05-09T04:34:48.080 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:34:48.082 INFO:teuthology.orchestra.run.smithi145.stderr:dumped all 2024-05-09T04:34:48.083 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.083+0000 7f7246e00640 1 -- 172.21.15.145:0/3179193861 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f721c0762d0 msgr2=0x7f721c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:48.084 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.083+0000 7f7246e00640 1 --2- 172.21.15.145:0/3179193861 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f721c0762d0 0x7f721c078790 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7230006fd0 tx=0x7f7230008040 comp rx=0 tx=0).stop 2024-05-09T04:34:48.084 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.083+0000 7f7246e00640 1 -- 172.21.15.145:0/3179193861 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7240145ac0 msgr2=0x7f724015c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:48.084 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.083+0000 7f7246e00640 1 --2- 172.21.15.145:0/3179193861 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7240145ac0 0x7f724015c910 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f722800c300 tx=0x7f7240161a70 comp rx=0 tx=0).stop 2024-05-09T04:34:48.084 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.083+0000 7f7246e00640 1 -- 172.21.15.145:0/3179193861 shutdown_connections 2024-05-09T04:34:48.084 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.083+0000 7f7246e00640 1 --2- 172.21.15.145:0/3179193861 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f721c0762d0 0x7f721c078790 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:48.084 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.083+0000 7f7246e00640 1 --2- 172.21.15.145:0/3179193861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f724013cb40 0x7f724013ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:48.084 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.083+0000 7f7246e00640 1 --2- 172.21.15.145:0/3179193861 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7240145ac0 0x7f724015c910 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:48.084 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.083+0000 7f7246e00640 1 --2- 172.21.15.145:0/3179193861 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7224004800 0x7f72401f5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:48.084 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.083+0000 7f7246e00640 1 -- 172.21.15.145:0/3179193861 >> 172.21.15.145:0/3179193861 conn(0x7f724006c870 msgr2=0x7f724006cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:48.084 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.083+0000 7f7246e00640 1 -- 172.21.15.145:0/3179193861 shutdown_connections 2024-05-09T04:34:48.084 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.083+0000 7f7246e00640 1 -- 172.21.15.145:0/3179193861 wait complete. 2024-05-09T04:34:48.100 INFO:teuthology.orchestra.run.smithi145.stdout:{"pg_ready":true,"pg_map":{"version":59,"stamp":"2024-05-09T04:34:47.772516+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":99,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220108,"kb_used_data":3944,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754612,"statfs":{"total":773094113280,"available":772868722688,"internally_reserved":0,"allocated":4038656,"data_stored":2260508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001384"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.697871+0000","last_change":"2024-05-09T04:34:43.694568+0000","last_active":"2024-05-09T04:34:43.697871+0000","last_peered":"2024-05-09T04:34:43.697871+0000","last_clean":"2024-05-09T04:34:43.697871+0000","last_became_active":"2024-05-09T04:34:43.694286+0000","last_became_peered":"2024-05-09T04:34:43.694286+0000","last_unstale":"2024-05-09T04:34:43.697871+0000","last_undegraded":"2024-05-09T04:34:43.697871+0000","last_fullsized":"2024-05-09T04:34:43.697871+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:51:31.953118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.685487+0000","last_change":"2024-05-09T04:34:43.686641+0000","last_active":"2024-05-09T04:34:44.685487+0000","last_peered":"2024-05-09T04:34:44.685487+0000","last_clean":"2024-05-09T04:34:44.685487+0000","last_became_active":"2024-05-09T04:34:43.685885+0000","last_became_peered":"2024-05-09T04:34:43.685885+0000","last_unstale":"2024-05-09T04:34:44.685487+0000","last_undegraded":"2024-05-09T04:34:44.685487+0000","last_fullsized":"2024-05-09T04:34:44.685487+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:13:04.076200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.699881+0000","last_change":"2024-05-09T04:34:43.693338+0000","last_active":"2024-05-09T04:34:43.699881+0000","last_peered":"2024-05-09T04:34:43.699881+0000","last_clean":"2024-05-09T04:34:43.699881+0000","last_became_active":"2024-05-09T04:34:43.692872+0000","last_became_peered":"2024-05-09T04:34:43.692872+0000","last_unstale":"2024-05-09T04:34:43.699881+0000","last_undegraded":"2024-05-09T04:34:43.699881+0000","last_fullsized":"2024-05-09T04:34:43.699881+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:24:03.114715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.694484+0000","last_change":"2024-05-09T04:34:43.680331+0000","last_active":"2024-05-09T04:34:44.694484+0000","last_peered":"2024-05-09T04:34:44.694484+0000","last_clean":"2024-05-09T04:34:44.694484+0000","last_became_active":"2024-05-09T04:34:43.677025+0000","last_became_peered":"2024-05-09T04:34:43.677025+0000","last_unstale":"2024-05-09T04:34:44.694484+0000","last_undegraded":"2024-05-09T04:34:44.694484+0000","last_fullsized":"2024-05-09T04:34:44.694484+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:01:46.785782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.683911+0000","last_change":"2024-05-09T04:34:43.706018+0000","last_active":"2024-05-09T04:34:44.683911+0000","last_peered":"2024-05-09T04:34:44.683911+0000","last_clean":"2024-05-09T04:34:44.683911+0000","last_became_active":"2024-05-09T04:34:43.705710+0000","last_became_peered":"2024-05-09T04:34:43.705710+0000","last_unstale":"2024-05-09T04:34:44.683911+0000","last_undegraded":"2024-05-09T04:34:44.683911+0000","last_fullsized":"2024-05-09T04:34:44.683911+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:47:32.035300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.684890+0000","last_change":"2024-05-09T04:34:43.694085+0000","last_active":"2024-05-09T04:34:44.684890+0000","last_peered":"2024-05-09T04:34:44.684890+0000","last_clean":"2024-05-09T04:34:44.684890+0000","last_became_active":"2024-05-09T04:34:43.693754+0000","last_became_peered":"2024-05-09T04:34:43.693754+0000","last_unstale":"2024-05-09T04:34:44.684890+0000","last_undegraded":"2024-05-09T04:34:44.684890+0000","last_fullsized":"2024-05-09T04:34:44.684890+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:35:34.192112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.709340+0000","last_change":"2024-05-09T04:34:43.703595+0000","last_active":"2024-05-09T04:34:43.709340+0000","last_peered":"2024-05-09T04:34:43.709340+0000","last_clean":"2024-05-09T04:34:43.709340+0000","last_became_active":"2024-05-09T04:34:43.702622+0000","last_became_peered":"2024-05-09T04:34:43.702622+0000","last_unstale":"2024-05-09T04:34:43.709340+0000","last_undegraded":"2024-05-09T04:34:43.709340+0000","last_fullsized":"2024-05-09T04:34:43.709340+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:24:03.114715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.712902+0000","last_change":"2024-05-09T04:34:43.708716+0000","last_active":"2024-05-09T04:34:43.712902+0000","last_peered":"2024-05-09T04:34:43.712902+0000","last_clean":"2024-05-09T04:34:43.712902+0000","last_became_active":"2024-05-09T04:34:43.708404+0000","last_became_peered":"2024-05-09T04:34:43.708404+0000","last_unstale":"2024-05-09T04:34:43.712902+0000","last_undegraded":"2024-05-09T04:34:43.712902+0000","last_fullsized":"2024-05-09T04:34:43.712902+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:14:17.180834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.689170+0000","last_change":"2024-05-09T04:34:43.689680+0000","last_active":"2024-05-09T04:34:44.689170+0000","last_peered":"2024-05-09T04:34:44.689170+0000","last_clean":"2024-05-09T04:34:44.689170+0000","last_became_active":"2024-05-09T04:34:43.689417+0000","last_became_peered":"2024-05-09T04:34:43.689417+0000","last_unstale":"2024-05-09T04:34:44.689170+0000","last_undegraded":"2024-05-09T04:34:44.689170+0000","last_fullsized":"2024-05-09T04:34:44.689170+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:01:46.785782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.694722+0000","last_change":"2024-05-09T04:34:43.692250+0000","last_active":"2024-05-09T04:34:44.694722+0000","last_peered":"2024-05-09T04:34:44.694722+0000","last_clean":"2024-05-09T04:34:44.694722+0000","last_became_active":"2024-05-09T04:34:43.687629+0000","last_became_peered":"2024-05-09T04:34:43.687629+0000","last_unstale":"2024-05-09T04:34:44.694722+0000","last_undegraded":"2024-05-09T04:34:44.694722+0000","last_fullsized":"2024-05-09T04:34:44.694722+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:15:12.390868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.702542+0000","last_change":"2024-05-09T04:34:43.694594+0000","last_active":"2024-05-09T04:34:43.702542+0000","last_peered":"2024-05-09T04:34:43.702542+0000","last_clean":"2024-05-09T04:34:43.702542+0000","last_became_active":"2024-05-09T04:34:43.693590+0000","last_became_peered":"2024-05-09T04:34:43.693590+0000","last_unstale":"2024-05-09T04:34:43.702542+0000","last_undegraded":"2024-05-09T04:34:43.702542+0000","last_fullsized":"2024-05-09T04:34:43.702542+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:12:32.277722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.711048+0000","last_change":"2024-05-09T04:34:43.700356+0000","last_active":"2024-05-09T04:34:43.711048+0000","last_peered":"2024-05-09T04:34:43.711048+0000","last_clean":"2024-05-09T04:34:43.711048+0000","last_became_active":"2024-05-09T04:34:43.699317+0000","last_became_peered":"2024-05-09T04:34:43.699317+0000","last_unstale":"2024-05-09T04:34:43.711048+0000","last_undegraded":"2024-05-09T04:34:43.711048+0000","last_fullsized":"2024-05-09T04:34:43.711048+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:11:23.998791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.684280+0000","last_change":"2024-05-09T04:34:43.698708+0000","last_active":"2024-05-09T04:34:44.684280+0000","last_peered":"2024-05-09T04:34:44.684280+0000","last_clean":"2024-05-09T04:34:44.684280+0000","last_became_active":"2024-05-09T04:34:43.698113+0000","last_became_peered":"2024-05-09T04:34:43.698113+0000","last_unstale":"2024-05-09T04:34:44.684280+0000","last_undegraded":"2024-05-09T04:34:44.684280+0000","last_fullsized":"2024-05-09T04:34:44.684280+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:52:21.080889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.684970+0000","last_change":"2024-05-09T04:34:43.685359+0000","last_active":"2024-05-09T04:34:44.684970+0000","last_peered":"2024-05-09T04:34:44.684970+0000","last_clean":"2024-05-09T04:34:44.684970+0000","last_became_active":"2024-05-09T04:34:43.685035+0000","last_became_peered":"2024-05-09T04:34:43.685035+0000","last_unstale":"2024-05-09T04:34:44.684970+0000","last_undegraded":"2024-05-09T04:34:44.684970+0000","last_fullsized":"2024-05-09T04:34:44.684970+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:51.416480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.711008+0000","last_change":"2024-05-09T04:34:43.697607+0000","last_active":"2024-05-09T04:34:43.711008+0000","last_peered":"2024-05-09T04:34:43.711008+0000","last_clean":"2024-05-09T04:34:43.711008+0000","last_became_active":"2024-05-09T04:34:43.694675+0000","last_became_peered":"2024-05-09T04:34:43.694675+0000","last_unstale":"2024-05-09T04:34:43.711008+0000","last_undegraded":"2024-05-09T04:34:43.711008+0000","last_fullsized":"2024-05-09T04:34:43.711008+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:03:31.499077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.712969+0000","last_change":"2024-05-09T04:34:43.710358+0000","last_active":"2024-05-09T04:34:43.712969+0000","last_peered":"2024-05-09T04:34:43.712969+0000","last_clean":"2024-05-09T04:34:43.712969+0000","last_became_active":"2024-05-09T04:34:43.709867+0000","last_became_peered":"2024-05-09T04:34:43.709867+0000","last_unstale":"2024-05-09T04:34:43.712969+0000","last_undegraded":"2024-05-09T04:34:43.712969+0000","last_fullsized":"2024-05-09T04:34:43.712969+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:48:53.850522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.708928+0000","last_change":"2024-05-09T04:34:43.702030+0000","last_active":"2024-05-09T04:34:43.708928+0000","last_peered":"2024-05-09T04:34:43.708928+0000","last_clean":"2024-05-09T04:34:43.708928+0000","last_became_active":"2024-05-09T04:34:43.701700+0000","last_became_peered":"2024-05-09T04:34:43.701700+0000","last_unstale":"2024-05-09T04:34:43.708928+0000","last_undegraded":"2024-05-09T04:34:43.708928+0000","last_fullsized":"2024-05-09T04:34:43.708928+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:08:58.949795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.685568+0000","last_change":"2024-05-09T04:34:43.685331+0000","last_active":"2024-05-09T04:34:44.685568+0000","last_peered":"2024-05-09T04:34:44.685568+0000","last_clean":"2024-05-09T04:34:44.685568+0000","last_became_active":"2024-05-09T04:34:43.684724+0000","last_became_peered":"2024-05-09T04:34:43.684724+0000","last_unstale":"2024-05-09T04:34:44.685568+0000","last_undegraded":"2024-05-09T04:34:44.685568+0000","last_fullsized":"2024-05-09T04:34:44.685568+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:07:22.472142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.685308+0000","last_change":"2024-05-09T04:34:43.683637+0000","last_active":"2024-05-09T04:34:44.685308+0000","last_peered":"2024-05-09T04:34:44.685308+0000","last_clean":"2024-05-09T04:34:44.685308+0000","last_became_active":"2024-05-09T04:34:43.681702+0000","last_became_peered":"2024-05-09T04:34:43.681702+0000","last_unstale":"2024-05-09T04:34:44.685308+0000","last_undegraded":"2024-05-09T04:34:44.685308+0000","last_fullsized":"2024-05-09T04:34:44.685308+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:37:45.506660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.684185+0000","last_change":"2024-05-09T04:34:43.706300+0000","last_active":"2024-05-09T04:34:44.684185+0000","last_peered":"2024-05-09T04:34:44.684185+0000","last_clean":"2024-05-09T04:34:44.684185+0000","last_became_active":"2024-05-09T04:34:43.705959+0000","last_became_peered":"2024-05-09T04:34:43.705959+0000","last_unstale":"2024-05-09T04:34:44.684185+0000","last_undegraded":"2024-05-09T04:34:44.684185+0000","last_fullsized":"2024-05-09T04:34:44.684185+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:07:39.701863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.687944+0000","last_change":"2024-05-09T04:34:43.687539+0000","last_active":"2024-05-09T04:34:44.687944+0000","last_peered":"2024-05-09T04:34:44.687944+0000","last_clean":"2024-05-09T04:34:44.687944+0000","last_became_active":"2024-05-09T04:34:43.687172+0000","last_became_peered":"2024-05-09T04:34:43.687172+0000","last_unstale":"2024-05-09T04:34:44.687944+0000","last_undegraded":"2024-05-09T04:34:44.687944+0000","last_fullsized":"2024-05-09T04:34:44.687944+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:04:34.378647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.688537+0000","last_change":"2024-05-09T04:34:43.700328+0000","last_active":"2024-05-09T04:34:44.688537+0000","last_peered":"2024-05-09T04:34:44.688537+0000","last_clean":"2024-05-09T04:34:44.688537+0000","last_became_active":"2024-05-09T04:34:43.691266+0000","last_became_peered":"2024-05-09T04:34:43.691266+0000","last_unstale":"2024-05-09T04:34:44.688537+0000","last_undegraded":"2024-05-09T04:34:44.688537+0000","last_fullsized":"2024-05-09T04:34:44.688537+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:11:23.998791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.691117+0000","last_change":"2024-05-09T04:34:43.696613+0000","last_active":"2024-05-09T04:34:44.691117+0000","last_peered":"2024-05-09T04:34:44.691117+0000","last_clean":"2024-05-09T04:34:44.691117+0000","last_became_active":"2024-05-09T04:34:43.696168+0000","last_became_peered":"2024-05-09T04:34:43.696168+0000","last_unstale":"2024-05-09T04:34:44.691117+0000","last_undegraded":"2024-05-09T04:34:44.691117+0000","last_fullsized":"2024-05-09T04:34:44.691117+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:24:03.114715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.710902+0000","last_change":"2024-05-09T04:34:43.699392+0000","last_active":"2024-05-09T04:34:43.710902+0000","last_peered":"2024-05-09T04:34:43.710902+0000","last_clean":"2024-05-09T04:34:43.710902+0000","last_became_active":"2024-05-09T04:34:43.698996+0000","last_became_peered":"2024-05-09T04:34:43.698996+0000","last_unstale":"2024-05-09T04:34:43.710902+0000","last_undegraded":"2024-05-09T04:34:43.710902+0000","last_fullsized":"2024-05-09T04:34:43.710902+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:04:34.378647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.694876+0000","last_change":"2024-05-09T04:34:43.701850+0000","last_active":"2024-05-09T04:34:44.694876+0000","last_peered":"2024-05-09T04:34:44.694876+0000","last_clean":"2024-05-09T04:34:44.694876+0000","last_became_active":"2024-05-09T04:34:43.701601+0000","last_became_peered":"2024-05-09T04:34:43.701601+0000","last_unstale":"2024-05-09T04:34:44.694876+0000","last_undegraded":"2024-05-09T04:34:44.694876+0000","last_fullsized":"2024-05-09T04:34:44.694876+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:59:51.258217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.685646+0000","last_change":"2024-05-09T04:34:43.702958+0000","last_active":"2024-05-09T04:34:44.685646+0000","last_peered":"2024-05-09T04:34:44.685646+0000","last_clean":"2024-05-09T04:34:44.685646+0000","last_became_active":"2024-05-09T04:34:43.701288+0000","last_became_peered":"2024-05-09T04:34:43.701288+0000","last_unstale":"2024-05-09T04:34:44.685646+0000","last_undegraded":"2024-05-09T04:34:44.685646+0000","last_fullsized":"2024-05-09T04:34:44.685646+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:13:56.137847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.694794+0000","last_change":"2024-05-09T04:34:43.679591+0000","last_active":"2024-05-09T04:34:44.694794+0000","last_peered":"2024-05-09T04:34:44.694794+0000","last_clean":"2024-05-09T04:34:44.694794+0000","last_became_active":"2024-05-09T04:34:43.679358+0000","last_became_peered":"2024-05-09T04:34:43.679358+0000","last_unstale":"2024-05-09T04:34:44.694794+0000","last_undegraded":"2024-05-09T04:34:44.694794+0000","last_fullsized":"2024-05-09T04:34:44.694794+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:26.112798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.710929+0000","last_change":"2024-05-09T04:34:43.699577+0000","last_active":"2024-05-09T04:34:43.710929+0000","last_peered":"2024-05-09T04:34:43.710929+0000","last_clean":"2024-05-09T04:34:43.710929+0000","last_became_active":"2024-05-09T04:34:43.697445+0000","last_became_peered":"2024-05-09T04:34:43.697445+0000","last_unstale":"2024-05-09T04:34:43.710929+0000","last_undegraded":"2024-05-09T04:34:43.710929+0000","last_fullsized":"2024-05-09T04:34:43.710929+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:14:17.180834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.688007+0000","last_change":"2024-05-09T04:34:43.684497+0000","last_active":"2024-05-09T04:34:44.688007+0000","last_peered":"2024-05-09T04:34:44.688007+0000","last_clean":"2024-05-09T04:34:44.688007+0000","last_became_active":"2024-05-09T04:34:43.682936+0000","last_became_peered":"2024-05-09T04:34:43.682936+0000","last_unstale":"2024-05-09T04:34:44.688007+0000","last_undegraded":"2024-05-09T04:34:44.688007+0000","last_fullsized":"2024-05-09T04:34:44.688007+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:14:17.180834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.707825+0000","last_change":"2024-05-09T04:34:43.701903+0000","last_active":"2024-05-09T04:34:43.707825+0000","last_peered":"2024-05-09T04:34:43.707825+0000","last_clean":"2024-05-09T04:34:43.707825+0000","last_became_active":"2024-05-09T04:34:43.701501+0000","last_became_peered":"2024-05-09T04:34:43.701501+0000","last_unstale":"2024-05-09T04:34:43.707825+0000","last_undegraded":"2024-05-09T04:34:43.707825+0000","last_fullsized":"2024-05-09T04:34:43.707825+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:33:16.575080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.706265+0000","last_change":"2024-05-09T04:34:43.695924+0000","last_active":"2024-05-09T04:34:43.706265+0000","last_peered":"2024-05-09T04:34:43.706265+0000","last_clean":"2024-05-09T04:34:43.706265+0000","last_became_active":"2024-05-09T04:34:43.694655+0000","last_became_peered":"2024-05-09T04:34:43.694655+0000","last_unstale":"2024-05-09T04:34:43.706265+0000","last_undegraded":"2024-05-09T04:34:43.706265+0000","last_fullsized":"2024-05-09T04:34:43.706265+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:11:53.554104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.711829+0000","last_change":"2024-05-09T04:34:43.706759+0000","last_active":"2024-05-09T04:34:43.711829+0000","last_peered":"2024-05-09T04:34:43.711829+0000","last_clean":"2024-05-09T04:34:43.711829+0000","last_became_active":"2024-05-09T04:34:43.706056+0000","last_became_peered":"2024-05-09T04:34:43.706056+0000","last_unstale":"2024-05-09T04:34:43.711829+0000","last_undegraded":"2024-05-09T04:34:43.711829+0000","last_fullsized":"2024-05-09T04:34:43.711829+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:37:25.633746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.691611+0000","last_change":"2024-05-09T04:34:43.703345+0000","last_active":"2024-05-09T04:34:44.691611+0000","last_peered":"2024-05-09T04:34:44.691611+0000","last_clean":"2024-05-09T04:34:44.691611+0000","last_became_active":"2024-05-09T04:34:43.703050+0000","last_became_peered":"2024-05-09T04:34:43.703050+0000","last_unstale":"2024-05-09T04:34:44.691611+0000","last_undegraded":"2024-05-09T04:34:44.691611+0000","last_fullsized":"2024-05-09T04:34:44.691611+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:03:31.499077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"31'4","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.711301+0000","last_change":"2024-05-09T04:34:42.694521+0000","last_active":"2024-05-09T04:34:43.711301+0000","last_peered":"2024-05-09T04:34:43.711301+0000","last_clean":"2024-05-09T04:34:43.711301+0000","last_became_active":"2024-05-09T04:34:42.694213+0000","last_became_peered":"2024-05-09T04:34:42.694213+0000","last_unstale":"2024-05-09T04:34:43.711301+0000","last_undegraded":"2024-05-09T04:34:43.711301+0000","last_fullsized":"2024-05-09T04:34:43.711301+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:51:57.033937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.712217+0000","last_change":"2024-05-09T04:34:43.704482+0000","last_active":"2024-05-09T04:34:43.712217+0000","last_peered":"2024-05-09T04:34:43.712217+0000","last_clean":"2024-05-09T04:34:43.712217+0000","last_became_active":"2024-05-09T04:34:43.703894+0000","last_became_peered":"2024-05-09T04:34:43.703894+0000","last_unstale":"2024-05-09T04:34:43.712217+0000","last_undegraded":"2024-05-09T04:34:43.712217+0000","last_fullsized":"2024-05-09T04:34:43.712217+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:52:21.080889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"31'1","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.691382+0000","last_change":"2024-05-09T04:34:42.683305+0000","last_active":"2024-05-09T04:34:44.691382+0000","last_peered":"2024-05-09T04:34:44.691382+0000","last_clean":"2024-05-09T04:34:44.691382+0000","last_became_active":"2024-05-09T04:34:42.682498+0000","last_became_peered":"2024-05-09T04:34:42.682498+0000","last_unstale":"2024-05-09T04:34:44.691382+0000","last_undegraded":"2024-05-09T04:34:44.691382+0000","last_fullsized":"2024-05-09T04:34:44.691382+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:47:31.029274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.688298+0000","last_change":"2024-05-09T04:34:43.702064+0000","last_active":"2024-05-09T04:34:44.688298+0000","last_peered":"2024-05-09T04:34:44.688298+0000","last_clean":"2024-05-09T04:34:44.688298+0000","last_became_active":"2024-05-09T04:34:43.698587+0000","last_became_peered":"2024-05-09T04:34:43.698587+0000","last_unstale":"2024-05-09T04:34:44.688298+0000","last_undegraded":"2024-05-09T04:34:44.688298+0000","last_fullsized":"2024-05-09T04:34:44.688298+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:48:53.850522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"31'2","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.685419+0000","last_change":"2024-05-09T04:34:42.689552+0000","last_active":"2024-05-09T04:34:44.685419+0000","last_peered":"2024-05-09T04:34:44.685419+0000","last_clean":"2024-05-09T04:34:44.685419+0000","last_became_active":"2024-05-09T04:34:42.689243+0000","last_became_peered":"2024-05-09T04:34:42.689243+0000","last_unstale":"2024-05-09T04:34:44.685419+0000","last_undegraded":"2024-05-09T04:34:44.685419+0000","last_fullsized":"2024-05-09T04:34:44.685419+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:47:37.225318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.694776+0000","last_change":"2024-05-09T04:34:43.684249+0000","last_active":"2024-05-09T04:34:44.694776+0000","last_peered":"2024-05-09T04:34:44.694776+0000","last_clean":"2024-05-09T04:34:44.694776+0000","last_became_active":"2024-05-09T04:34:43.683787+0000","last_became_peered":"2024-05-09T04:34:43.683787+0000","last_unstale":"2024-05-09T04:34:44.694776+0000","last_undegraded":"2024-05-09T04:34:44.694776+0000","last_fullsized":"2024-05-09T04:34:44.694776+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:07:08.077009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"31'2","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.684599+0000","last_change":"2024-05-09T04:34:42.682139+0000","last_active":"2024-05-09T04:34:44.684599+0000","last_peered":"2024-05-09T04:34:44.684599+0000","last_clean":"2024-05-09T04:34:44.684599+0000","last_became_active":"2024-05-09T04:34:42.681890+0000","last_became_peered":"2024-05-09T04:34:42.681890+0000","last_unstale":"2024-05-09T04:34:44.684599+0000","last_undegraded":"2024-05-09T04:34:44.684599+0000","last_fullsized":"2024-05-09T04:34:44.684599+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:40:02.921068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.712815+0000","last_change":"2024-05-09T04:34:43.708295+0000","last_active":"2024-05-09T04:34:43.712815+0000","last_peered":"2024-05-09T04:34:43.712815+0000","last_clean":"2024-05-09T04:34:43.712815+0000","last_became_active":"2024-05-09T04:34:43.707982+0000","last_became_peered":"2024-05-09T04:34:43.707982+0000","last_unstale":"2024-05-09T04:34:43.712815+0000","last_undegraded":"2024-05-09T04:34:43.712815+0000","last_fullsized":"2024-05-09T04:34:43.712815+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:03:31.499077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"31'1","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.688189+0000","last_change":"2024-05-09T04:34:42.683110+0000","last_active":"2024-05-09T04:34:44.688189+0000","last_peered":"2024-05-09T04:34:44.688189+0000","last_clean":"2024-05-09T04:34:44.688189+0000","last_became_active":"2024-05-09T04:34:42.682779+0000","last_became_peered":"2024-05-09T04:34:42.682779+0000","last_unstale":"2024-05-09T04:34:44.688189+0000","last_undegraded":"2024-05-09T04:34:44.688189+0000","last_fullsized":"2024-05-09T04:34:44.688189+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:51:30.947092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.710169+0000","last_change":"2024-05-09T04:34:43.696718+0000","last_active":"2024-05-09T04:34:43.710169+0000","last_peered":"2024-05-09T04:34:43.710169+0000","last_clean":"2024-05-09T04:34:43.710169+0000","last_became_active":"2024-05-09T04:34:43.696110+0000","last_became_peered":"2024-05-09T04:34:43.696110+0000","last_unstale":"2024-05-09T04:34:43.710169+0000","last_undegraded":"2024-05-09T04:34:43.710169+0000","last_fullsized":"2024-05-09T04:34:43.710169+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:13:44.029814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"31'1","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.688912+0000","last_change":"2024-05-09T04:34:42.683355+0000","last_active":"2024-05-09T04:34:44.688912+0000","last_peered":"2024-05-09T04:34:44.688912+0000","last_clean":"2024-05-09T04:34:44.688912+0000","last_became_active":"2024-05-09T04:34:42.682838+0000","last_became_peered":"2024-05-09T04:34:42.682838+0000","last_unstale":"2024-05-09T04:34:44.688912+0000","last_undegraded":"2024-05-09T04:34:44.688912+0000","last_fullsized":"2024-05-09T04:34:44.688912+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:14:41.788104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.687580+0000","last_change":"2024-05-09T04:34:43.701035+0000","last_active":"2024-05-09T04:34:44.687580+0000","last_peered":"2024-05-09T04:34:44.687580+0000","last_clean":"2024-05-09T04:34:44.687580+0000","last_became_active":"2024-05-09T04:34:43.691724+0000","last_became_peered":"2024-05-09T04:34:43.691724+0000","last_unstale":"2024-05-09T04:34:44.687580+0000","last_undegraded":"2024-05-09T04:34:44.687580+0000","last_fullsized":"2024-05-09T04:34:44.687580+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:03:31.499077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.684757+0000","last_change":"2024-05-09T04:34:42.681132+0000","last_active":"2024-05-09T04:34:44.684757+0000","last_peered":"2024-05-09T04:34:44.684757+0000","last_clean":"2024-05-09T04:34:44.684757+0000","last_became_active":"2024-05-09T04:34:42.680821+0000","last_became_peered":"2024-05-09T04:34:42.680821+0000","last_unstale":"2024-05-09T04:34:44.684757+0000","last_undegraded":"2024-05-09T04:34:44.684757+0000","last_fullsized":"2024-05-09T04:34:44.684757+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:37:24.627720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.694163+0000","last_change":"2024-05-09T04:34:43.690719+0000","last_active":"2024-05-09T04:34:44.694163+0000","last_peered":"2024-05-09T04:34:44.694163+0000","last_clean":"2024-05-09T04:34:44.694163+0000","last_became_active":"2024-05-09T04:34:43.689745+0000","last_became_peered":"2024-05-09T04:34:43.689745+0000","last_unstale":"2024-05-09T04:34:44.694163+0000","last_undegraded":"2024-05-09T04:34:44.694163+0000","last_fullsized":"2024-05-09T04:34:44.694163+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:59:34.370275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.689255+0000","last_change":"2024-05-09T04:34:42.689959+0000","last_active":"2024-05-09T04:34:44.689255+0000","last_peered":"2024-05-09T04:34:44.689255+0000","last_clean":"2024-05-09T04:34:44.689255+0000","last_became_active":"2024-05-09T04:34:42.689628+0000","last_became_peered":"2024-05-09T04:34:42.689628+0000","last_unstale":"2024-05-09T04:34:44.689255+0000","last_undegraded":"2024-05-09T04:34:44.689255+0000","last_fullsized":"2024-05-09T04:34:44.689255+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:04:49.709155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.711371+0000","last_change":"2024-05-09T04:34:43.707580+0000","last_active":"2024-05-09T04:34:43.711371+0000","last_peered":"2024-05-09T04:34:43.711371+0000","last_clean":"2024-05-09T04:34:43.711371+0000","last_became_active":"2024-05-09T04:34:43.707303+0000","last_became_peered":"2024-05-09T04:34:43.707303+0000","last_unstale":"2024-05-09T04:34:43.711371+0000","last_undegraded":"2024-05-09T04:34:43.711371+0000","last_fullsized":"2024-05-09T04:34:43.711371+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:59:34.370275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.685791+0000","last_change":"2024-05-09T04:34:42.688714+0000","last_active":"2024-05-09T04:34:44.685791+0000","last_peered":"2024-05-09T04:34:44.685791+0000","last_clean":"2024-05-09T04:34:44.685791+0000","last_became_active":"2024-05-09T04:34:42.687729+0000","last_became_peered":"2024-05-09T04:34:42.687729+0000","last_unstale":"2024-05-09T04:34:44.685791+0000","last_undegraded":"2024-05-09T04:34:44.685791+0000","last_fullsized":"2024-05-09T04:34:44.685791+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:03:30.493050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.685880+0000","last_change":"2024-05-09T04:34:43.692687+0000","last_active":"2024-05-09T04:34:44.685880+0000","last_peered":"2024-05-09T04:34:44.685880+0000","last_clean":"2024-05-09T04:34:44.685880+0000","last_became_active":"2024-05-09T04:34:43.692325+0000","last_became_peered":"2024-05-09T04:34:43.692325+0000","last_unstale":"2024-05-09T04:34:44.685880+0000","last_undegraded":"2024-05-09T04:34:44.685880+0000","last_fullsized":"2024-05-09T04:34:44.685880+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:50:14.994456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.691491+0000","last_change":"2024-05-09T04:34:42.683380+0000","last_active":"2024-05-09T04:34:44.691491+0000","last_peered":"2024-05-09T04:34:44.691491+0000","last_clean":"2024-05-09T04:34:44.691491+0000","last_became_active":"2024-05-09T04:34:42.681564+0000","last_became_peered":"2024-05-09T04:34:42.681564+0000","last_unstale":"2024-05-09T04:34:44.691491+0000","last_undegraded":"2024-05-09T04:34:44.691491+0000","last_fullsized":"2024-05-09T04:34:44.691491+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:51:30.947092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.685324+0000","last_change":"2024-05-09T04:34:43.709223+0000","last_active":"2024-05-09T04:34:44.685324+0000","last_peered":"2024-05-09T04:34:44.685324+0000","last_clean":"2024-05-09T04:34:44.685324+0000","last_became_active":"2024-05-09T04:34:43.708861+0000","last_became_peered":"2024-05-09T04:34:43.708861+0000","last_unstale":"2024-05-09T04:34:44.685324+0000","last_undegraded":"2024-05-09T04:34:44.685324+0000","last_fullsized":"2024-05-09T04:34:44.685324+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:54:33.072668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"31'3","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.711006+0000","last_change":"2024-05-09T04:34:42.683525+0000","last_active":"2024-05-09T04:34:43.711006+0000","last_peered":"2024-05-09T04:34:43.711006+0000","last_clean":"2024-05-09T04:34:43.711006+0000","last_became_active":"2024-05-09T04:34:42.683151+0000","last_became_peered":"2024-05-09T04:34:42.683151+0000","last_unstale":"2024-05-09T04:34:43.711006+0000","last_undegraded":"2024-05-09T04:34:43.711006+0000","last_fullsized":"2024-05-09T04:34:43.711006+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:47:37.225318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.699775+0000","last_change":"2024-05-09T04:34:43.693128+0000","last_active":"2024-05-09T04:34:43.699775+0000","last_peered":"2024-05-09T04:34:43.699775+0000","last_clean":"2024-05-09T04:34:43.699775+0000","last_became_active":"2024-05-09T04:34:43.692576+0000","last_became_peered":"2024-05-09T04:34:43.692576+0000","last_unstale":"2024-05-09T04:34:43.699775+0000","last_undegraded":"2024-05-09T04:34:43.699775+0000","last_fullsized":"2024-05-09T04:34:43.699775+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:08:58.949795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"31'6","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.685393+0000","last_change":"2024-05-09T04:34:42.686738+0000","last_active":"2024-05-09T04:34:44.685393+0000","last_peered":"2024-05-09T04:34:44.685393+0000","last_clean":"2024-05-09T04:34:44.685393+0000","last_became_active":"2024-05-09T04:34:42.685624+0000","last_became_peered":"2024-05-09T04:34:42.685624+0000","last_unstale":"2024-05-09T04:34:44.685393+0000","last_undegraded":"2024-05-09T04:34:44.685393+0000","last_fullsized":"2024-05-09T04:34:44.685393+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:04:33.372621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.694112+0000","last_change":"2024-05-09T04:34:43.689872+0000","last_active":"2024-05-09T04:34:44.694112+0000","last_peered":"2024-05-09T04:34:44.694112+0000","last_clean":"2024-05-09T04:34:44.694112+0000","last_became_active":"2024-05-09T04:34:43.689565+0000","last_became_peered":"2024-05-09T04:34:43.689565+0000","last_unstale":"2024-05-09T04:34:44.694112+0000","last_undegraded":"2024-05-09T04:34:44.694112+0000","last_fullsized":"2024-05-09T04:34:44.694112+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:11:23.998791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"31'2","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.688316+0000","last_change":"2024-05-09T04:34:42.691436+0000","last_active":"2024-05-09T04:34:44.688316+0000","last_peered":"2024-05-09T04:34:44.688316+0000","last_clean":"2024-05-09T04:34:44.688316+0000","last_became_active":"2024-05-09T04:34:42.691128+0000","last_became_peered":"2024-05-09T04:34:42.691128+0000","last_unstale":"2024-05-09T04:34:44.688316+0000","last_undegraded":"2024-05-09T04:34:44.688316+0000","last_fullsized":"2024-05-09T04:34:44.688316+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:51:57.033937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.685468+0000","last_change":"2024-05-09T04:34:43.701843+0000","last_active":"2024-05-09T04:34:44.685468+0000","last_peered":"2024-05-09T04:34:44.685468+0000","last_clean":"2024-05-09T04:34:44.685468+0000","last_became_active":"2024-05-09T04:34:43.701553+0000","last_became_peered":"2024-05-09T04:34:43.701553+0000","last_unstale":"2024-05-09T04:34:44.685468+0000","last_undegraded":"2024-05-09T04:34:44.685468+0000","last_fullsized":"2024-05-09T04:34:44.685468+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:57:06.820576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.694165+0000","last_change":"2024-05-09T04:34:42.691963+0000","last_active":"2024-05-09T04:34:44.694165+0000","last_peered":"2024-05-09T04:34:44.694165+0000","last_clean":"2024-05-09T04:34:44.694165+0000","last_became_active":"2024-05-09T04:34:42.691689+0000","last_became_peered":"2024-05-09T04:34:42.691689+0000","last_unstale":"2024-05-09T04:34:44.694165+0000","last_undegraded":"2024-05-09T04:34:44.694165+0000","last_fullsized":"2024-05-09T04:34:44.694165+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:47:37.225318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.711799+0000","last_change":"2024-05-09T04:34:43.706306+0000","last_active":"2024-05-09T04:34:43.711799+0000","last_peered":"2024-05-09T04:34:43.711799+0000","last_clean":"2024-05-09T04:34:43.711799+0000","last_became_active":"2024-05-09T04:34:43.705631+0000","last_became_peered":"2024-05-09T04:34:43.705631+0000","last_unstale":"2024-05-09T04:34:43.711799+0000","last_undegraded":"2024-05-09T04:34:43.711799+0000","last_fullsized":"2024-05-09T04:34:43.711799+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:13:44.029814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.684629+0000","last_change":"2024-05-09T04:34:42.687735+0000","last_active":"2024-05-09T04:34:44.684629+0000","last_peered":"2024-05-09T04:34:44.684629+0000","last_clean":"2024-05-09T04:34:44.684629+0000","last_became_active":"2024-05-09T04:34:42.687447+0000","last_became_peered":"2024-05-09T04:34:42.687447+0000","last_unstale":"2024-05-09T04:34:44.684629+0000","last_undegraded":"2024-05-09T04:34:44.684629+0000","last_fullsized":"2024-05-09T04:34:44.684629+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:13:43.023788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.706308+0000","last_change":"2024-05-09T04:34:43.700026+0000","last_active":"2024-05-09T04:34:43.706308+0000","last_peered":"2024-05-09T04:34:43.706308+0000","last_clean":"2024-05-09T04:34:43.706308+0000","last_became_active":"2024-05-09T04:34:43.699713+0000","last_became_peered":"2024-05-09T04:34:43.699713+0000","last_unstale":"2024-05-09T04:34:43.706308+0000","last_undegraded":"2024-05-09T04:34:43.706308+0000","last_fullsized":"2024-05-09T04:34:43.706308+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:07:39.701863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"31'8","reported_seq":22,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.684636+0000","last_change":"2024-05-09T04:34:42.689511+0000","last_active":"2024-05-09T04:34:44.684636+0000","last_peered":"2024-05-09T04:34:44.684636+0000","last_clean":"2024-05-09T04:34:44.684636+0000","last_became_active":"2024-05-09T04:34:42.688929+0000","last_became_peered":"2024-05-09T04:34:42.688929+0000","last_unstale":"2024-05-09T04:34:44.684636+0000","last_undegraded":"2024-05-09T04:34:44.684636+0000","last_fullsized":"2024-05-09T04:34:44.684636+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:24:02.108688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.688863+0000","last_change":"2024-05-09T04:34:43.695050+0000","last_active":"2024-05-09T04:34:44.688863+0000","last_peered":"2024-05-09T04:34:44.688863+0000","last_clean":"2024-05-09T04:34:44.688863+0000","last_became_active":"2024-05-09T04:34:43.694756+0000","last_became_peered":"2024-05-09T04:34:43.694756+0000","last_unstale":"2024-05-09T04:34:44.688863+0000","last_undegraded":"2024-05-09T04:34:44.688863+0000","last_fullsized":"2024-05-09T04:34:44.688863+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:59:34.370275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"31'1","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.694989+0000","last_change":"2024-05-09T04:34:42.684827+0000","last_active":"2024-05-09T04:34:44.694989+0000","last_peered":"2024-05-09T04:34:44.694989+0000","last_clean":"2024-05-09T04:34:44.694989+0000","last_became_active":"2024-05-09T04:34:42.684506+0000","last_became_peered":"2024-05-09T04:34:42.684506+0000","last_unstale":"2024-05-09T04:34:44.694989+0000","last_undegraded":"2024-05-09T04:34:44.694989+0000","last_fullsized":"2024-05-09T04:34:44.694989+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:12:31.271696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.711007+0000","last_change":"2024-05-09T04:34:43.699790+0000","last_active":"2024-05-09T04:34:43.711007+0000","last_peered":"2024-05-09T04:34:43.711007+0000","last_clean":"2024-05-09T04:34:43.711007+0000","last_became_active":"2024-05-09T04:34:43.699474+0000","last_became_peered":"2024-05-09T04:34:43.699474+0000","last_unstale":"2024-05-09T04:34:43.711007+0000","last_undegraded":"2024-05-09T04:34:43.711007+0000","last_fullsized":"2024-05-09T04:34:43.711007+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:48:53.850522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"31'2","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.688853+0000","last_change":"2024-05-09T04:34:42.692093+0000","last_active":"2024-05-09T04:34:44.688853+0000","last_peered":"2024-05-09T04:34:44.688853+0000","last_clean":"2024-05-09T04:34:44.688853+0000","last_became_active":"2024-05-09T04:34:42.691523+0000","last_became_peered":"2024-05-09T04:34:42.691523+0000","last_unstale":"2024-05-09T04:34:44.688853+0000","last_undegraded":"2024-05-09T04:34:44.688853+0000","last_fullsized":"2024-05-09T04:34:44.688853+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:11:52.548077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.694644+0000","last_change":"2024-05-09T04:34:43.691404+0000","last_active":"2024-05-09T04:34:44.694644+0000","last_peered":"2024-05-09T04:34:44.694644+0000","last_clean":"2024-05-09T04:34:44.694644+0000","last_became_active":"2024-05-09T04:34:43.691036+0000","last_became_peered":"2024-05-09T04:34:43.691036+0000","last_unstale":"2024-05-09T04:34:44.694644+0000","last_undegraded":"2024-05-09T04:34:44.694644+0000","last_fullsized":"2024-05-09T04:34:44.694644+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:21:15.637406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"31'1","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.688639+0000","last_change":"2024-05-09T04:34:42.680457+0000","last_active":"2024-05-09T04:34:44.688639+0000","last_peered":"2024-05-09T04:34:44.688639+0000","last_clean":"2024-05-09T04:34:44.688639+0000","last_became_active":"2024-05-09T04:34:42.679663+0000","last_became_peered":"2024-05-09T04:34:42.679663+0000","last_unstale":"2024-05-09T04:34:44.688639+0000","last_undegraded":"2024-05-09T04:34:44.688639+0000","last_fullsized":"2024-05-09T04:34:44.688639+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:47:31.029274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.687040+0000","last_change":"2024-05-09T04:34:43.704738+0000","last_active":"2024-05-09T04:34:44.687040+0000","last_peered":"2024-05-09T04:34:44.687040+0000","last_clean":"2024-05-09T04:34:44.687040+0000","last_became_active":"2024-05-09T04:34:43.704422+0000","last_became_peered":"2024-05-09T04:34:43.704422+0000","last_unstale":"2024-05-09T04:34:44.687040+0000","last_undegraded":"2024-05-09T04:34:44.687040+0000","last_fullsized":"2024-05-09T04:34:44.687040+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:33:16.575080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.688513+0000","last_change":"2024-05-09T04:34:42.683053+0000","last_active":"2024-05-09T04:34:44.688513+0000","last_peered":"2024-05-09T04:34:44.688513+0000","last_clean":"2024-05-09T04:34:44.688513+0000","last_became_active":"2024-05-09T04:34:42.682678+0000","last_became_peered":"2024-05-09T04:34:42.682678+0000","last_unstale":"2024-05-09T04:34:44.688513+0000","last_undegraded":"2024-05-09T04:34:44.688513+0000","last_fullsized":"2024-05-09T04:34:44.688513+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:10:00.199341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.683874+0000","last_change":"2024-05-09T04:34:43.691730+0000","last_active":"2024-05-09T04:34:44.683874+0000","last_peered":"2024-05-09T04:34:44.683874+0000","last_clean":"2024-05-09T04:34:44.683874+0000","last_became_active":"2024-05-09T04:34:43.691411+0000","last_became_peered":"2024-05-09T04:34:43.691411+0000","last_unstale":"2024-05-09T04:34:44.683874+0000","last_undegraded":"2024-05-09T04:34:44.683874+0000","last_fullsized":"2024-05-09T04:34:44.683874+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:10:01.205367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"31'2","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.691432+0000","last_change":"2024-05-09T04:34:42.678125+0000","last_active":"2024-05-09T04:34:44.691432+0000","last_peered":"2024-05-09T04:34:44.691432+0000","last_clean":"2024-05-09T04:34:44.691432+0000","last_became_active":"2024-05-09T04:34:42.677754+0000","last_became_peered":"2024-05-09T04:34:42.677754+0000","last_unstale":"2024-05-09T04:34:44.691432+0000","last_undegraded":"2024-05-09T04:34:44.691432+0000","last_fullsized":"2024-05-09T04:34:44.691432+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:07:38.695837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.684616+0000","last_change":"2024-05-09T04:34:43.704328+0000","last_active":"2024-05-09T04:34:44.684616+0000","last_peered":"2024-05-09T04:34:44.684616+0000","last_clean":"2024-05-09T04:34:44.684616+0000","last_became_active":"2024-05-09T04:34:43.698460+0000","last_became_peered":"2024-05-09T04:34:43.698460+0000","last_unstale":"2024-05-09T04:34:44.684616+0000","last_undegraded":"2024-05-09T04:34:44.684616+0000","last_fullsized":"2024-05-09T04:34:44.684616+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:12:32.277722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.681608+0000","last_change":"2024-05-09T04:34:42.694370+0000","last_active":"2024-05-09T04:34:43.681608+0000","last_peered":"2024-05-09T04:34:43.681608+0000","last_clean":"2024-05-09T04:34:43.681608+0000","last_became_active":"2024-05-09T04:34:42.693975+0000","last_became_peered":"2024-05-09T04:34:42.693975+0000","last_unstale":"2024-05-09T04:34:43.681608+0000","last_undegraded":"2024-05-09T04:34:43.681608+0000","last_fullsized":"2024-05-09T04:34:43.681608+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:53:40.510377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.684147+0000","last_change":"2024-05-09T04:34:43.697017+0000","last_active":"2024-05-09T04:34:44.684147+0000","last_peered":"2024-05-09T04:34:44.684147+0000","last_clean":"2024-05-09T04:34:44.684147+0000","last_became_active":"2024-05-09T04:34:43.696762+0000","last_became_peered":"2024-05-09T04:34:43.696762+0000","last_unstale":"2024-05-09T04:34:44.684147+0000","last_undegraded":"2024-05-09T04:34:44.684147+0000","last_fullsized":"2024-05-09T04:34:44.684147+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:11:53.554104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.687356+0000","last_change":"2024-05-09T04:34:42.685942+0000","last_active":"2024-05-09T04:34:44.687356+0000","last_peered":"2024-05-09T04:34:44.687356+0000","last_clean":"2024-05-09T04:34:44.687356+0000","last_became_active":"2024-05-09T04:34:42.685657+0000","last_became_peered":"2024-05-09T04:34:42.685657+0000","last_unstale":"2024-05-09T04:34:44.687356+0000","last_undegraded":"2024-05-09T04:34:44.687356+0000","last_fullsized":"2024-05-09T04:34:44.687356+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:42:56.870686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.707858+0000","last_change":"2024-05-09T04:34:43.701922+0000","last_active":"2024-05-09T04:34:43.707858+0000","last_peered":"2024-05-09T04:34:43.707858+0000","last_clean":"2024-05-09T04:34:43.707858+0000","last_became_active":"2024-05-09T04:34:43.701517+0000","last_became_peered":"2024-05-09T04:34:43.701517+0000","last_unstale":"2024-05-09T04:34:43.707858+0000","last_undegraded":"2024-05-09T04:34:43.707858+0000","last_fullsized":"2024-05-09T04:34:43.707858+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:47:38.231344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.685638+0000","last_change":"2024-05-09T04:34:42.682776+0000","last_active":"2024-05-09T04:34:43.685638+0000","last_peered":"2024-05-09T04:34:43.685638+0000","last_clean":"2024-05-09T04:34:43.685638+0000","last_became_active":"2024-05-09T04:34:42.682496+0000","last_became_peered":"2024-05-09T04:34:42.682496+0000","last_unstale":"2024-05-09T04:34:43.685638+0000","last_undegraded":"2024-05-09T04:34:43.685638+0000","last_fullsized":"2024-05-09T04:34:43.685638+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:33:15.569054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.683993+0000","last_change":"2024-05-09T04:34:43.695781+0000","last_active":"2024-05-09T04:34:44.683993+0000","last_peered":"2024-05-09T04:34:44.683993+0000","last_clean":"2024-05-09T04:34:44.683993+0000","last_became_active":"2024-05-09T04:34:43.695475+0000","last_became_peered":"2024-05-09T04:34:43.695475+0000","last_unstale":"2024-05-09T04:34:44.683993+0000","last_undegraded":"2024-05-09T04:34:44.683993+0000","last_fullsized":"2024-05-09T04:34:44.683993+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:51:31.953118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"31'2","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.694990+0000","last_change":"2024-05-09T04:34:42.675499+0000","last_active":"2024-05-09T04:34:44.694990+0000","last_peered":"2024-05-09T04:34:44.694990+0000","last_clean":"2024-05-09T04:34:44.694990+0000","last_became_active":"2024-05-09T04:34:42.672972+0000","last_became_peered":"2024-05-09T04:34:42.672972+0000","last_unstale":"2024-05-09T04:34:44.694990+0000","last_undegraded":"2024-05-09T04:34:44.694990+0000","last_fullsized":"2024-05-09T04:34:44.694990+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:33:15.569054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.710198+0000","last_change":"2024-05-09T04:34:43.696910+0000","last_active":"2024-05-09T04:34:43.710198+0000","last_peered":"2024-05-09T04:34:43.710198+0000","last_clean":"2024-05-09T04:34:43.710198+0000","last_became_active":"2024-05-09T04:34:43.694234+0000","last_became_peered":"2024-05-09T04:34:43.694234+0000","last_unstale":"2024-05-09T04:34:43.710198+0000","last_undegraded":"2024-05-09T04:34:43.710198+0000","last_fullsized":"2024-05-09T04:34:43.710198+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:37:25.633746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"32'1","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.693466+0000","last_change":"2024-05-09T04:34:42.674612+0000","last_active":"2024-05-09T04:34:44.693466+0000","last_peered":"2024-05-09T04:34:44.693466+0000","last_clean":"2024-05-09T04:34:44.693466+0000","last_became_active":"2024-05-09T04:34:42.674289+0000","last_became_peered":"2024-05-09T04:34:42.674289+0000","last_unstale":"2024-05-09T04:34:44.693466+0000","last_undegraded":"2024-05-09T04:34:44.693466+0000","last_fullsized":"2024-05-09T04:34:44.693466+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:11:52.548077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.691295+0000","last_change":"2024-05-09T04:34:43.688662+0000","last_active":"2024-05-09T04:34:44.691295+0000","last_peered":"2024-05-09T04:34:44.691295+0000","last_clean":"2024-05-09T04:34:44.691295+0000","last_became_active":"2024-05-09T04:34:43.688144+0000","last_became_peered":"2024-05-09T04:34:43.688144+0000","last_unstale":"2024-05-09T04:34:44.691295+0000","last_undegraded":"2024-05-09T04:34:44.691295+0000","last_fullsized":"2024-05-09T04:34:44.691295+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:13:44.029814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.694582+0000","last_change":"2024-05-09T04:34:42.675062+0000","last_active":"2024-05-09T04:34:44.694582+0000","last_peered":"2024-05-09T04:34:44.694582+0000","last_clean":"2024-05-09T04:34:44.694582+0000","last_became_active":"2024-05-09T04:34:42.674612+0000","last_became_peered":"2024-05-09T04:34:42.674612+0000","last_unstale":"2024-05-09T04:34:44.694582+0000","last_undegraded":"2024-05-09T04:34:44.694582+0000","last_fullsized":"2024-05-09T04:34:44.694582+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:52:20.074863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.694974+0000","last_change":"2024-05-09T04:34:43.690599+0000","last_active":"2024-05-09T04:34:43.694974+0000","last_peered":"2024-05-09T04:34:43.694974+0000","last_clean":"2024-05-09T04:34:43.694974+0000","last_became_active":"2024-05-09T04:34:43.690279+0000","last_became_peered":"2024-05-09T04:34:43.690279+0000","last_unstale":"2024-05-09T04:34:43.694974+0000","last_undegraded":"2024-05-09T04:34:43.694974+0000","last_fullsized":"2024-05-09T04:34:43.694974+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:47:32.035300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.691522+0000","last_change":"2024-05-09T04:34:42.678410+0000","last_active":"2024-05-09T04:34:44.691522+0000","last_peered":"2024-05-09T04:34:44.691522+0000","last_clean":"2024-05-09T04:34:44.691522+0000","last_became_active":"2024-05-09T04:34:42.678103+0000","last_became_peered":"2024-05-09T04:34:42.678103+0000","last_unstale":"2024-05-09T04:34:44.691522+0000","last_undegraded":"2024-05-09T04:34:44.691522+0000","last_fullsized":"2024-05-09T04:34:44.691522+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:10:00.199341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.691400+0000","last_change":"2024-05-09T04:34:43.699825+0000","last_active":"2024-05-09T04:34:44.691400+0000","last_peered":"2024-05-09T04:34:44.691400+0000","last_clean":"2024-05-09T04:34:44.691400+0000","last_became_active":"2024-05-09T04:34:43.694720+0000","last_became_peered":"2024-05-09T04:34:43.694720+0000","last_unstale":"2024-05-09T04:34:44.691400+0000","last_undegraded":"2024-05-09T04:34:44.691400+0000","last_fullsized":"2024-05-09T04:34:44.691400+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:37:25.633746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.694247+0000","last_change":"2024-05-09T04:34:42.684510+0000","last_active":"2024-05-09T04:34:44.694247+0000","last_peered":"2024-05-09T04:34:44.694247+0000","last_clean":"2024-05-09T04:34:44.694247+0000","last_became_active":"2024-05-09T04:34:42.684164+0000","last_became_peered":"2024-05-09T04:34:42.684164+0000","last_unstale":"2024-05-09T04:34:44.694247+0000","last_undegraded":"2024-05-09T04:34:44.694247+0000","last_fullsized":"2024-05-09T04:34:44.694247+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:07:38.695837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.684536+0000","last_change":"2024-05-09T04:34:43.687222+0000","last_active":"2024-05-09T04:34:44.684536+0000","last_peered":"2024-05-09T04:34:44.684536+0000","last_clean":"2024-05-09T04:34:44.684536+0000","last_became_active":"2024-05-09T04:34:43.684963+0000","last_became_peered":"2024-05-09T04:34:43.684963+0000","last_unstale":"2024-05-09T04:34:44.684536+0000","last_undegraded":"2024-05-09T04:34:44.684536+0000","last_fullsized":"2024-05-09T04:34:44.684536+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:29:30.265815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.687653+0000","last_change":"2024-05-09T04:34:42.680340+0000","last_active":"2024-05-09T04:34:44.687653+0000","last_peered":"2024-05-09T04:34:44.687653+0000","last_clean":"2024-05-09T04:34:44.687653+0000","last_became_active":"2024-05-09T04:34:42.680044+0000","last_became_peered":"2024-05-09T04:34:42.680044+0000","last_unstale":"2024-05-09T04:34:44.687653+0000","last_undegraded":"2024-05-09T04:34:44.687653+0000","last_fullsized":"2024-05-09T04:34:44.687653+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:53:40.510377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.687371+0000","last_change":"2024-05-09T04:34:43.687573+0000","last_active":"2024-05-09T04:34:44.687371+0000","last_peered":"2024-05-09T04:34:44.687371+0000","last_clean":"2024-05-09T04:34:44.687371+0000","last_became_active":"2024-05-09T04:34:43.684669+0000","last_became_peered":"2024-05-09T04:34:43.684669+0000","last_unstale":"2024-05-09T04:34:44.687371+0000","last_undegraded":"2024-05-09T04:34:44.687371+0000","last_fullsized":"2024-05-09T04:34:44.687371+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:37:25.633746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.684573+0000","last_change":"2024-05-09T04:34:42.687616+0000","last_active":"2024-05-09T04:34:44.684573+0000","last_peered":"2024-05-09T04:34:44.684573+0000","last_clean":"2024-05-09T04:34:44.684573+0000","last_became_active":"2024-05-09T04:34:42.687338+0000","last_became_peered":"2024-05-09T04:34:42.687338+0000","last_unstale":"2024-05-09T04:34:44.684573+0000","last_undegraded":"2024-05-09T04:34:44.684573+0000","last_fullsized":"2024-05-09T04:34:44.684573+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:08:57.943769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.712900+0000","last_change":"2024-05-09T04:34:43.708639+0000","last_active":"2024-05-09T04:34:43.712900+0000","last_peered":"2024-05-09T04:34:43.712900+0000","last_clean":"2024-05-09T04:34:43.712900+0000","last_became_active":"2024-05-09T04:34:43.708340+0000","last_became_peered":"2024-05-09T04:34:43.708340+0000","last_unstale":"2024-05-09T04:34:43.712900+0000","last_undegraded":"2024-05-09T04:34:43.712900+0000","last_fullsized":"2024-05-09T04:34:43.712900+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:04:34.378647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.684736+0000","last_change":"2024-05-09T04:34:42.695559+0000","last_active":"2024-05-09T04:34:43.684736+0000","last_peered":"2024-05-09T04:34:43.684736+0000","last_clean":"2024-05-09T04:34:43.684736+0000","last_became_active":"2024-05-09T04:34:42.695234+0000","last_became_peered":"2024-05-09T04:34:42.695234+0000","last_unstale":"2024-05-09T04:34:43.684736+0000","last_undegraded":"2024-05-09T04:34:43.684736+0000","last_fullsized":"2024-05-09T04:34:43.684736+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:10:00.199341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":130,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.683672+0000","last_change":"2024-05-09T04:33:41.092063+0000","last_active":"2024-05-09T04:34:43.683672+0000","last_peered":"2024-05-09T04:34:43.683672+0000","last_clean":"2024-05-09T04:34:43.683672+0000","last_became_active":"2024-05-09T04:33:41.091194+0000","last_became_peered":"2024-05-09T04:33:41.091194+0000","last_unstale":"2024-05-09T04:34:43.683672+0000","last_undegraded":"2024-05-09T04:34:43.683672+0000","last_fullsized":"2024-05-09T04:34:43.683672+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:40.045339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:40.045339+0000","last_clean_scrub_stamp":"2024-05-09T04:33:40.045339+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:04.908456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":946176,"data_stored":724827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":356352,"data_stored":134985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":356352,"data_stored":135041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":356352,"data_stored":134473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":364544,"data_stored":136003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":352256,"data_stored":134439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":364544,"data_stored":136003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":942080,"data_stored":724737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T04:34:48.105 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T04:34:48.342 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.342+0000 7fa51c000640 1 Processor -- start 2024-05-09T04:34:48.342 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.342+0000 7fa51c000640 1 -- start start 2024-05-09T04:34:48.345 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.344+0000 7fa51c000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fa51406e860 0x7fa51406ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:48.345 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.344+0000 7fa51c000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa51415d780 0x7fa514160c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:48.345 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.344+0000 7fa51c000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa51406f180 con 0x7fa5140ce620 2024-05-09T04:34:48.345 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.344+0000 7fa51c000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fa5141615c0 con 0x7fa51406e860 2024-05-09T04:34:48.345 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.344+0000 7fa51c000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fa5141618a0 con 0x7fa51415d780 2024-05-09T04:34:48.346 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.344+0000 7fa519600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fa51406e860 0x7fa51406ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:48.346 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.344+0000 7fa51a000640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7fa5140ce620 0x7fa5140cea60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:49050/0 (socket says 172.21.15.145:49050) 2024-05-09T04:34:48.346 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.344+0000 7fa51a000640 1 -- 172.21.15.145:0/118472825 learned_addr learned my addr 172.21.15.145:0/118472825 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:34:48.346 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.345+0000 7fa518c00640 1 --2- 172.21.15.145:0/118472825 >> v2:172.21.15.192:3300/0 conn(0x7fa51415d780 0x7fa514160c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:48.346 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.345+0000 7fa513e00640 1 -- 172.21.15.145:0/118472825 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2946091860 0 0) 0x7fa51406f180 con 0x7fa5140ce620 2024-05-09T04:34:48.346 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.345+0000 7fa513e00640 1 -- 172.21.15.145:0/118472825 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa4fc003520 con 0x7fa5140ce620 2024-05-09T04:34:48.346 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.345+0000 7fa519600640 1 -- 172.21.15.145:0/118472825 >> v2:172.21.15.192:3300/0 conn(0x7fa51415d780 msgr2=0x7fa514160c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:48.346 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.345+0000 7fa519600640 1 --2- 172.21.15.145:0/118472825 >> v2:172.21.15.192:3300/0 conn(0x7fa51415d780 0x7fa514160c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:48.346 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.345+0000 7fa519600640 1 -- 172.21.15.145:0/118472825 >> v1:172.21.15.145:6789/0 conn(0x7fa5140ce620 legacy=0x7fa5140cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:48.346 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.345+0000 7fa519600640 1 -- 172.21.15.145:0/118472825 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5140b4110 con 0x7fa51406e860 2024-05-09T04:34:48.348 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.346+0000 7fa519600640 1 --2- 172.21.15.145:0/118472825 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa51406e860 0x7fa51406ec40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa5140d0e20 tx=0x7fa50802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f803abfd922f61e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:48.348 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.346+0000 7fa513e00640 1 -- 172.21.15.145:0/118472825 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa508037070 con 0x7fa51406e860 2024-05-09T04:34:48.348 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.346+0000 7fa513e00640 1 -- 172.21.15.145:0/118472825 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa508032440 con 0x7fa51406e860 2024-05-09T04:34:48.348 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.346+0000 7fa513e00640 1 -- 172.21.15.145:0/118472825 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa508032810 con 0x7fa51406e860 2024-05-09T04:34:48.348 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.347+0000 7fa51c000640 1 -- 172.21.15.145:0/118472825 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa51406e860 msgr2=0x7fa51406ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:48.348 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.347+0000 7fa51c000640 1 --2- 172.21.15.145:0/118472825 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa51406e860 0x7fa51406ec40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa5140d0e20 tx=0x7fa50802a1a0 comp rx=0 tx=0).stop 2024-05-09T04:34:48.348 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.347+0000 7fa51c000640 1 -- 172.21.15.145:0/118472825 shutdown_connections 2024-05-09T04:34:48.348 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.347+0000 7fa51c000640 1 --2- 172.21.15.145:0/118472825 >> v2:172.21.15.192:3300/0 conn(0x7fa51415d780 0x7fa514160c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:48.348 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.347+0000 7fa51c000640 1 --2- 172.21.15.145:0/118472825 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa51406e860 0x7fa51406ec40 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:48.348 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.347+0000 7fa51c000640 1 -- 172.21.15.145:0/118472825 >> 172.21.15.145:0/118472825 conn(0x7fa51406c870 msgr2=0x7fa51406cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:48.348 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.347+0000 7fa51c000640 1 -- 172.21.15.145:0/118472825 shutdown_connections 2024-05-09T04:34:48.349 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.347+0000 7fa51c000640 1 -- 172.21.15.145:0/118472825 wait complete. 2024-05-09T04:34:48.351 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.349+0000 7fa51c000640 1 Processor -- start 2024-05-09T04:34:48.352 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.349+0000 7fa51c000640 1 -- start start 2024-05-09T04:34:48.352 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.350+0000 7fa51c000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa51406e860 0x7fa5141f43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:48.352 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.350+0000 7fa51c000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa5140ce620 0x7fa514072be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:48.352 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.350+0000 7fa51c000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa51415d780 0x7fa5141e6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:48.352 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.350+0000 7fa51c000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa514146d80 con 0x7fa51415d780 2024-05-09T04:34:48.352 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.350+0000 7fa51c000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa514146c00 con 0x7fa51406e860 2024-05-09T04:34:48.352 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.350+0000 7fa51c000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa514161190 con 0x7fa5140ce620 2024-05-09T04:34:48.352 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.350+0000 7fa51a000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa51415d780 0x7fa5141e6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:48.352 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.350+0000 7fa51a000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa51415d780 0x7fa5141e6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:59352/0 (socket says 172.21.15.145:59352) 2024-05-09T04:34:48.352 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.350+0000 7fa51a000640 1 -- 172.21.15.145:0/3948502642 learned_addr learned my addr 172.21.15.145:0/3948502642 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:48.352 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.351+0000 7fa519600640 1 --2- 172.21.15.145:0/3948502642 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa51406e860 0x7fa5141f43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:48.352 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.351+0000 7fa518c00640 1 --2- 172.21.15.145:0/3948502642 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa5140ce620 0x7fa514072be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:48.353 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.351+0000 7fa51a000640 1 -- 172.21.15.145:0/3948502642 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa5140ce620 msgr2=0x7fa514072be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:48.353 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.351+0000 7fa51a000640 1 --2- 172.21.15.145:0/3948502642 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa5140ce620 0x7fa514072be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:48.353 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.351+0000 7fa51a000640 1 -- 172.21.15.145:0/3948502642 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa51406e860 msgr2=0x7fa5141f43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:48.353 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.351+0000 7fa51a000640 1 --2- 172.21.15.145:0/3948502642 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa51406e860 0x7fa5141f43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:48.353 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.351+0000 7fa51a000640 1 -- 172.21.15.145:0/3948502642 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa514073610 con 0x7fa51415d780 2024-05-09T04:34:48.353 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.351+0000 7fa51a000640 1 --2- 172.21.15.145:0/3948502642 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa51415d780 0x7fa5141e6a70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa50c00cb70 tx=0x7fa50c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:48.353 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.351+0000 7fa511600640 1 -- 172.21.15.145:0/3948502642 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa50c00ace0 con 0x7fa51415d780 2024-05-09T04:34:48.353 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.351+0000 7fa511600640 1 -- 172.21.15.145:0/3948502642 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa50c012400 con 0x7fa51415d780 2024-05-09T04:34:48.353 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.351+0000 7fa511600640 1 -- 172.21.15.145:0/3948502642 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa50c0127b0 con 0x7fa51415d780 2024-05-09T04:34:48.353 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.351+0000 7fa51c000640 1 -- 172.21.15.145:0/3948502642 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5140738f0 con 0x7fa51415d780 2024-05-09T04:34:48.353 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.351+0000 7fa51c000640 1 -- 172.21.15.145:0/3948502642 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5141442d0 con 0x7fa51415d780 2024-05-09T04:34:48.354 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.352+0000 7fa51c000640 1 -- 172.21.15.145:0/3948502642 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa51406a310 con 0x7fa51415d780 2024-05-09T04:34:48.354 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.354+0000 7fa511600640 1 -- 172.21.15.145:0/3948502642 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa50c012950 con 0x7fa51415d780 2024-05-09T04:34:48.355 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.355+0000 7fa511600640 1 --2- 172.21.15.145:0/3948502642 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fa4f40762d0 0x7fa4f4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:48.355 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.355+0000 7fa511600640 1 -- 172.21.15.145:0/3948502642 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fa50c097cf0 con 0x7fa51415d780 2024-05-09T04:34:48.355 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.355+0000 7fa519600640 1 --2- 172.21.15.145:0/3948502642 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fa4f40762d0 0x7fa4f4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:48.356 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.356+0000 7fa519600640 1 --2- 172.21.15.145:0/3948502642 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fa4f40762d0 0x7fa4f4078790 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa508034000 tx=0x7fa50802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:48.359 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.360+0000 7fa511600640 1 -- 172.21.15.145:0/3948502642 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa50c061640 con 0x7fa51415d780 2024-05-09T04:34:48.564 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.564+0000 7fa51c000640 1 -- 172.21.15.145:0/3948502642 --> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa5141444a0 con 0x7fa4f40762d0 2024-05-09T04:34:48.575 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.575+0000 7fa511600640 1 -- 172.21.15.145:0/3948502642 <== mgr.4104 v2:172.21.15.145:6832/2725855396 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251910 (secure 0 0 0) 0x7fa5141444a0 con 0x7fa4f40762d0 2024-05-09T04:34:48.579 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:34:48.581 INFO:teuthology.orchestra.run.smithi145.stderr:dumped all 2024-05-09T04:34:48.582 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.581+0000 7fa51c000640 1 -- 172.21.15.145:0/3948502642 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fa4f40762d0 msgr2=0x7fa4f4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:48.582 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.581+0000 7fa51c000640 1 --2- 172.21.15.145:0/3948502642 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fa4f40762d0 0x7fa4f4078790 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa508034000 tx=0x7fa50802e000 comp rx=0 tx=0).stop 2024-05-09T04:34:48.582 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.581+0000 7fa51c000640 1 -- 172.21.15.145:0/3948502642 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa51415d780 msgr2=0x7fa5141e6a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:48.582 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.581+0000 7fa51c000640 1 --2- 172.21.15.145:0/3948502642 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa51415d780 0x7fa5141e6a70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa50c00cb70 tx=0x7fa50c00a430 comp rx=0 tx=0).stop 2024-05-09T04:34:48.582 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.581+0000 7fa51c000640 1 -- 172.21.15.145:0/3948502642 shutdown_connections 2024-05-09T04:34:48.583 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.581+0000 7fa51c000640 1 --2- 172.21.15.145:0/3948502642 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fa4f40762d0 0x7fa4f4078790 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:48.583 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.581+0000 7fa51c000640 1 --2- 172.21.15.145:0/3948502642 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa51415d780 0x7fa5141e6a70 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:48.583 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.581+0000 7fa51c000640 1 --2- 172.21.15.145:0/3948502642 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa5140ce620 0x7fa514072be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:48.583 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.581+0000 7fa51c000640 1 --2- 172.21.15.145:0/3948502642 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa51406e860 0x7fa5141f43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:48.583 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.581+0000 7fa51c000640 1 -- 172.21.15.145:0/3948502642 >> 172.21.15.145:0/3948502642 conn(0x7fa51406c870 msgr2=0x7fa5140b4810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:48.583 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.581+0000 7fa51c000640 1 -- 172.21.15.145:0/3948502642 shutdown_connections 2024-05-09T04:34:48.583 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.581+0000 7fa51c000640 1 -- 172.21.15.145:0/3948502642 wait complete. 2024-05-09T04:34:48.600 INFO:teuthology.orchestra.run.smithi145.stdout:{"pg_ready":true,"pg_map":{"version":59,"stamp":"2024-05-09T04:34:47.772516+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":99,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220108,"kb_used_data":3944,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754612,"statfs":{"total":773094113280,"available":772868722688,"internally_reserved":0,"allocated":4038656,"data_stored":2260508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001384"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.697871+0000","last_change":"2024-05-09T04:34:43.694568+0000","last_active":"2024-05-09T04:34:43.697871+0000","last_peered":"2024-05-09T04:34:43.697871+0000","last_clean":"2024-05-09T04:34:43.697871+0000","last_became_active":"2024-05-09T04:34:43.694286+0000","last_became_peered":"2024-05-09T04:34:43.694286+0000","last_unstale":"2024-05-09T04:34:43.697871+0000","last_undegraded":"2024-05-09T04:34:43.697871+0000","last_fullsized":"2024-05-09T04:34:43.697871+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:51:31.953118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.685487+0000","last_change":"2024-05-09T04:34:43.686641+0000","last_active":"2024-05-09T04:34:44.685487+0000","last_peered":"2024-05-09T04:34:44.685487+0000","last_clean":"2024-05-09T04:34:44.685487+0000","last_became_active":"2024-05-09T04:34:43.685885+0000","last_became_peered":"2024-05-09T04:34:43.685885+0000","last_unstale":"2024-05-09T04:34:44.685487+0000","last_undegraded":"2024-05-09T04:34:44.685487+0000","last_fullsized":"2024-05-09T04:34:44.685487+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:13:04.076200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.699881+0000","last_change":"2024-05-09T04:34:43.693338+0000","last_active":"2024-05-09T04:34:43.699881+0000","last_peered":"2024-05-09T04:34:43.699881+0000","last_clean":"2024-05-09T04:34:43.699881+0000","last_became_active":"2024-05-09T04:34:43.692872+0000","last_became_peered":"2024-05-09T04:34:43.692872+0000","last_unstale":"2024-05-09T04:34:43.699881+0000","last_undegraded":"2024-05-09T04:34:43.699881+0000","last_fullsized":"2024-05-09T04:34:43.699881+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:24:03.114715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.694484+0000","last_change":"2024-05-09T04:34:43.680331+0000","last_active":"2024-05-09T04:34:44.694484+0000","last_peered":"2024-05-09T04:34:44.694484+0000","last_clean":"2024-05-09T04:34:44.694484+0000","last_became_active":"2024-05-09T04:34:43.677025+0000","last_became_peered":"2024-05-09T04:34:43.677025+0000","last_unstale":"2024-05-09T04:34:44.694484+0000","last_undegraded":"2024-05-09T04:34:44.694484+0000","last_fullsized":"2024-05-09T04:34:44.694484+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:01:46.785782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.683911+0000","last_change":"2024-05-09T04:34:43.706018+0000","last_active":"2024-05-09T04:34:44.683911+0000","last_peered":"2024-05-09T04:34:44.683911+0000","last_clean":"2024-05-09T04:34:44.683911+0000","last_became_active":"2024-05-09T04:34:43.705710+0000","last_became_peered":"2024-05-09T04:34:43.705710+0000","last_unstale":"2024-05-09T04:34:44.683911+0000","last_undegraded":"2024-05-09T04:34:44.683911+0000","last_fullsized":"2024-05-09T04:34:44.683911+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:47:32.035300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.684890+0000","last_change":"2024-05-09T04:34:43.694085+0000","last_active":"2024-05-09T04:34:44.684890+0000","last_peered":"2024-05-09T04:34:44.684890+0000","last_clean":"2024-05-09T04:34:44.684890+0000","last_became_active":"2024-05-09T04:34:43.693754+0000","last_became_peered":"2024-05-09T04:34:43.693754+0000","last_unstale":"2024-05-09T04:34:44.684890+0000","last_undegraded":"2024-05-09T04:34:44.684890+0000","last_fullsized":"2024-05-09T04:34:44.684890+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:35:34.192112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.709340+0000","last_change":"2024-05-09T04:34:43.703595+0000","last_active":"2024-05-09T04:34:43.709340+0000","last_peered":"2024-05-09T04:34:43.709340+0000","last_clean":"2024-05-09T04:34:43.709340+0000","last_became_active":"2024-05-09T04:34:43.702622+0000","last_became_peered":"2024-05-09T04:34:43.702622+0000","last_unstale":"2024-05-09T04:34:43.709340+0000","last_undegraded":"2024-05-09T04:34:43.709340+0000","last_fullsized":"2024-05-09T04:34:43.709340+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:24:03.114715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.712902+0000","last_change":"2024-05-09T04:34:43.708716+0000","last_active":"2024-05-09T04:34:43.712902+0000","last_peered":"2024-05-09T04:34:43.712902+0000","last_clean":"2024-05-09T04:34:43.712902+0000","last_became_active":"2024-05-09T04:34:43.708404+0000","last_became_peered":"2024-05-09T04:34:43.708404+0000","last_unstale":"2024-05-09T04:34:43.712902+0000","last_undegraded":"2024-05-09T04:34:43.712902+0000","last_fullsized":"2024-05-09T04:34:43.712902+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:14:17.180834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.689170+0000","last_change":"2024-05-09T04:34:43.689680+0000","last_active":"2024-05-09T04:34:44.689170+0000","last_peered":"2024-05-09T04:34:44.689170+0000","last_clean":"2024-05-09T04:34:44.689170+0000","last_became_active":"2024-05-09T04:34:43.689417+0000","last_became_peered":"2024-05-09T04:34:43.689417+0000","last_unstale":"2024-05-09T04:34:44.689170+0000","last_undegraded":"2024-05-09T04:34:44.689170+0000","last_fullsized":"2024-05-09T04:34:44.689170+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:01:46.785782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.694722+0000","last_change":"2024-05-09T04:34:43.692250+0000","last_active":"2024-05-09T04:34:44.694722+0000","last_peered":"2024-05-09T04:34:44.694722+0000","last_clean":"2024-05-09T04:34:44.694722+0000","last_became_active":"2024-05-09T04:34:43.687629+0000","last_became_peered":"2024-05-09T04:34:43.687629+0000","last_unstale":"2024-05-09T04:34:44.694722+0000","last_undegraded":"2024-05-09T04:34:44.694722+0000","last_fullsized":"2024-05-09T04:34:44.694722+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:15:12.390868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.702542+0000","last_change":"2024-05-09T04:34:43.694594+0000","last_active":"2024-05-09T04:34:43.702542+0000","last_peered":"2024-05-09T04:34:43.702542+0000","last_clean":"2024-05-09T04:34:43.702542+0000","last_became_active":"2024-05-09T04:34:43.693590+0000","last_became_peered":"2024-05-09T04:34:43.693590+0000","last_unstale":"2024-05-09T04:34:43.702542+0000","last_undegraded":"2024-05-09T04:34:43.702542+0000","last_fullsized":"2024-05-09T04:34:43.702542+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:12:32.277722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.711048+0000","last_change":"2024-05-09T04:34:43.700356+0000","last_active":"2024-05-09T04:34:43.711048+0000","last_peered":"2024-05-09T04:34:43.711048+0000","last_clean":"2024-05-09T04:34:43.711048+0000","last_became_active":"2024-05-09T04:34:43.699317+0000","last_became_peered":"2024-05-09T04:34:43.699317+0000","last_unstale":"2024-05-09T04:34:43.711048+0000","last_undegraded":"2024-05-09T04:34:43.711048+0000","last_fullsized":"2024-05-09T04:34:43.711048+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:11:23.998791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.684280+0000","last_change":"2024-05-09T04:34:43.698708+0000","last_active":"2024-05-09T04:34:44.684280+0000","last_peered":"2024-05-09T04:34:44.684280+0000","last_clean":"2024-05-09T04:34:44.684280+0000","last_became_active":"2024-05-09T04:34:43.698113+0000","last_became_peered":"2024-05-09T04:34:43.698113+0000","last_unstale":"2024-05-09T04:34:44.684280+0000","last_undegraded":"2024-05-09T04:34:44.684280+0000","last_fullsized":"2024-05-09T04:34:44.684280+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:52:21.080889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.684970+0000","last_change":"2024-05-09T04:34:43.685359+0000","last_active":"2024-05-09T04:34:44.684970+0000","last_peered":"2024-05-09T04:34:44.684970+0000","last_clean":"2024-05-09T04:34:44.684970+0000","last_became_active":"2024-05-09T04:34:43.685035+0000","last_became_peered":"2024-05-09T04:34:43.685035+0000","last_unstale":"2024-05-09T04:34:44.684970+0000","last_undegraded":"2024-05-09T04:34:44.684970+0000","last_fullsized":"2024-05-09T04:34:44.684970+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:39:51.416480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.711008+0000","last_change":"2024-05-09T04:34:43.697607+0000","last_active":"2024-05-09T04:34:43.711008+0000","last_peered":"2024-05-09T04:34:43.711008+0000","last_clean":"2024-05-09T04:34:43.711008+0000","last_became_active":"2024-05-09T04:34:43.694675+0000","last_became_peered":"2024-05-09T04:34:43.694675+0000","last_unstale":"2024-05-09T04:34:43.711008+0000","last_undegraded":"2024-05-09T04:34:43.711008+0000","last_fullsized":"2024-05-09T04:34:43.711008+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:03:31.499077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.712969+0000","last_change":"2024-05-09T04:34:43.710358+0000","last_active":"2024-05-09T04:34:43.712969+0000","last_peered":"2024-05-09T04:34:43.712969+0000","last_clean":"2024-05-09T04:34:43.712969+0000","last_became_active":"2024-05-09T04:34:43.709867+0000","last_became_peered":"2024-05-09T04:34:43.709867+0000","last_unstale":"2024-05-09T04:34:43.712969+0000","last_undegraded":"2024-05-09T04:34:43.712969+0000","last_fullsized":"2024-05-09T04:34:43.712969+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:48:53.850522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.708928+0000","last_change":"2024-05-09T04:34:43.702030+0000","last_active":"2024-05-09T04:34:43.708928+0000","last_peered":"2024-05-09T04:34:43.708928+0000","last_clean":"2024-05-09T04:34:43.708928+0000","last_became_active":"2024-05-09T04:34:43.701700+0000","last_became_peered":"2024-05-09T04:34:43.701700+0000","last_unstale":"2024-05-09T04:34:43.708928+0000","last_undegraded":"2024-05-09T04:34:43.708928+0000","last_fullsized":"2024-05-09T04:34:43.708928+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:08:58.949795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.685568+0000","last_change":"2024-05-09T04:34:43.685331+0000","last_active":"2024-05-09T04:34:44.685568+0000","last_peered":"2024-05-09T04:34:44.685568+0000","last_clean":"2024-05-09T04:34:44.685568+0000","last_became_active":"2024-05-09T04:34:43.684724+0000","last_became_peered":"2024-05-09T04:34:43.684724+0000","last_unstale":"2024-05-09T04:34:44.685568+0000","last_undegraded":"2024-05-09T04:34:44.685568+0000","last_fullsized":"2024-05-09T04:34:44.685568+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:07:22.472142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.685308+0000","last_change":"2024-05-09T04:34:43.683637+0000","last_active":"2024-05-09T04:34:44.685308+0000","last_peered":"2024-05-09T04:34:44.685308+0000","last_clean":"2024-05-09T04:34:44.685308+0000","last_became_active":"2024-05-09T04:34:43.681702+0000","last_became_peered":"2024-05-09T04:34:43.681702+0000","last_unstale":"2024-05-09T04:34:44.685308+0000","last_undegraded":"2024-05-09T04:34:44.685308+0000","last_fullsized":"2024-05-09T04:34:44.685308+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:37:45.506660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.684185+0000","last_change":"2024-05-09T04:34:43.706300+0000","last_active":"2024-05-09T04:34:44.684185+0000","last_peered":"2024-05-09T04:34:44.684185+0000","last_clean":"2024-05-09T04:34:44.684185+0000","last_became_active":"2024-05-09T04:34:43.705959+0000","last_became_peered":"2024-05-09T04:34:43.705959+0000","last_unstale":"2024-05-09T04:34:44.684185+0000","last_undegraded":"2024-05-09T04:34:44.684185+0000","last_fullsized":"2024-05-09T04:34:44.684185+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:07:39.701863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.687944+0000","last_change":"2024-05-09T04:34:43.687539+0000","last_active":"2024-05-09T04:34:44.687944+0000","last_peered":"2024-05-09T04:34:44.687944+0000","last_clean":"2024-05-09T04:34:44.687944+0000","last_became_active":"2024-05-09T04:34:43.687172+0000","last_became_peered":"2024-05-09T04:34:43.687172+0000","last_unstale":"2024-05-09T04:34:44.687944+0000","last_undegraded":"2024-05-09T04:34:44.687944+0000","last_fullsized":"2024-05-09T04:34:44.687944+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:04:34.378647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.688537+0000","last_change":"2024-05-09T04:34:43.700328+0000","last_active":"2024-05-09T04:34:44.688537+0000","last_peered":"2024-05-09T04:34:44.688537+0000","last_clean":"2024-05-09T04:34:44.688537+0000","last_became_active":"2024-05-09T04:34:43.691266+0000","last_became_peered":"2024-05-09T04:34:43.691266+0000","last_unstale":"2024-05-09T04:34:44.688537+0000","last_undegraded":"2024-05-09T04:34:44.688537+0000","last_fullsized":"2024-05-09T04:34:44.688537+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:11:23.998791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.691117+0000","last_change":"2024-05-09T04:34:43.696613+0000","last_active":"2024-05-09T04:34:44.691117+0000","last_peered":"2024-05-09T04:34:44.691117+0000","last_clean":"2024-05-09T04:34:44.691117+0000","last_became_active":"2024-05-09T04:34:43.696168+0000","last_became_peered":"2024-05-09T04:34:43.696168+0000","last_unstale":"2024-05-09T04:34:44.691117+0000","last_undegraded":"2024-05-09T04:34:44.691117+0000","last_fullsized":"2024-05-09T04:34:44.691117+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:24:03.114715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.710902+0000","last_change":"2024-05-09T04:34:43.699392+0000","last_active":"2024-05-09T04:34:43.710902+0000","last_peered":"2024-05-09T04:34:43.710902+0000","last_clean":"2024-05-09T04:34:43.710902+0000","last_became_active":"2024-05-09T04:34:43.698996+0000","last_became_peered":"2024-05-09T04:34:43.698996+0000","last_unstale":"2024-05-09T04:34:43.710902+0000","last_undegraded":"2024-05-09T04:34:43.710902+0000","last_fullsized":"2024-05-09T04:34:43.710902+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:04:34.378647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.694876+0000","last_change":"2024-05-09T04:34:43.701850+0000","last_active":"2024-05-09T04:34:44.694876+0000","last_peered":"2024-05-09T04:34:44.694876+0000","last_clean":"2024-05-09T04:34:44.694876+0000","last_became_active":"2024-05-09T04:34:43.701601+0000","last_became_peered":"2024-05-09T04:34:43.701601+0000","last_unstale":"2024-05-09T04:34:44.694876+0000","last_undegraded":"2024-05-09T04:34:44.694876+0000","last_fullsized":"2024-05-09T04:34:44.694876+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:59:51.258217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.685646+0000","last_change":"2024-05-09T04:34:43.702958+0000","last_active":"2024-05-09T04:34:44.685646+0000","last_peered":"2024-05-09T04:34:44.685646+0000","last_clean":"2024-05-09T04:34:44.685646+0000","last_became_active":"2024-05-09T04:34:43.701288+0000","last_became_peered":"2024-05-09T04:34:43.701288+0000","last_unstale":"2024-05-09T04:34:44.685646+0000","last_undegraded":"2024-05-09T04:34:44.685646+0000","last_fullsized":"2024-05-09T04:34:44.685646+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:13:56.137847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.694794+0000","last_change":"2024-05-09T04:34:43.679591+0000","last_active":"2024-05-09T04:34:44.694794+0000","last_peered":"2024-05-09T04:34:44.694794+0000","last_clean":"2024-05-09T04:34:44.694794+0000","last_became_active":"2024-05-09T04:34:43.679358+0000","last_became_peered":"2024-05-09T04:34:43.679358+0000","last_unstale":"2024-05-09T04:34:44.694794+0000","last_undegraded":"2024-05-09T04:34:44.694794+0000","last_fullsized":"2024-05-09T04:34:44.694794+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:05:26.112798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.710929+0000","last_change":"2024-05-09T04:34:43.699577+0000","last_active":"2024-05-09T04:34:43.710929+0000","last_peered":"2024-05-09T04:34:43.710929+0000","last_clean":"2024-05-09T04:34:43.710929+0000","last_became_active":"2024-05-09T04:34:43.697445+0000","last_became_peered":"2024-05-09T04:34:43.697445+0000","last_unstale":"2024-05-09T04:34:43.710929+0000","last_undegraded":"2024-05-09T04:34:43.710929+0000","last_fullsized":"2024-05-09T04:34:43.710929+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:14:17.180834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.688007+0000","last_change":"2024-05-09T04:34:43.684497+0000","last_active":"2024-05-09T04:34:44.688007+0000","last_peered":"2024-05-09T04:34:44.688007+0000","last_clean":"2024-05-09T04:34:44.688007+0000","last_became_active":"2024-05-09T04:34:43.682936+0000","last_became_peered":"2024-05-09T04:34:43.682936+0000","last_unstale":"2024-05-09T04:34:44.688007+0000","last_undegraded":"2024-05-09T04:34:44.688007+0000","last_fullsized":"2024-05-09T04:34:44.688007+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:14:17.180834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.707825+0000","last_change":"2024-05-09T04:34:43.701903+0000","last_active":"2024-05-09T04:34:43.707825+0000","last_peered":"2024-05-09T04:34:43.707825+0000","last_clean":"2024-05-09T04:34:43.707825+0000","last_became_active":"2024-05-09T04:34:43.701501+0000","last_became_peered":"2024-05-09T04:34:43.701501+0000","last_unstale":"2024-05-09T04:34:43.707825+0000","last_undegraded":"2024-05-09T04:34:43.707825+0000","last_fullsized":"2024-05-09T04:34:43.707825+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:33:16.575080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.706265+0000","last_change":"2024-05-09T04:34:43.695924+0000","last_active":"2024-05-09T04:34:43.706265+0000","last_peered":"2024-05-09T04:34:43.706265+0000","last_clean":"2024-05-09T04:34:43.706265+0000","last_became_active":"2024-05-09T04:34:43.694655+0000","last_became_peered":"2024-05-09T04:34:43.694655+0000","last_unstale":"2024-05-09T04:34:43.706265+0000","last_undegraded":"2024-05-09T04:34:43.706265+0000","last_fullsized":"2024-05-09T04:34:43.706265+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:11:53.554104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.711829+0000","last_change":"2024-05-09T04:34:43.706759+0000","last_active":"2024-05-09T04:34:43.711829+0000","last_peered":"2024-05-09T04:34:43.711829+0000","last_clean":"2024-05-09T04:34:43.711829+0000","last_became_active":"2024-05-09T04:34:43.706056+0000","last_became_peered":"2024-05-09T04:34:43.706056+0000","last_unstale":"2024-05-09T04:34:43.711829+0000","last_undegraded":"2024-05-09T04:34:43.711829+0000","last_fullsized":"2024-05-09T04:34:43.711829+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:37:25.633746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.691611+0000","last_change":"2024-05-09T04:34:43.703345+0000","last_active":"2024-05-09T04:34:44.691611+0000","last_peered":"2024-05-09T04:34:44.691611+0000","last_clean":"2024-05-09T04:34:44.691611+0000","last_became_active":"2024-05-09T04:34:43.703050+0000","last_became_peered":"2024-05-09T04:34:43.703050+0000","last_unstale":"2024-05-09T04:34:44.691611+0000","last_undegraded":"2024-05-09T04:34:44.691611+0000","last_fullsized":"2024-05-09T04:34:44.691611+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:03:31.499077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"31'4","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.711301+0000","last_change":"2024-05-09T04:34:42.694521+0000","last_active":"2024-05-09T04:34:43.711301+0000","last_peered":"2024-05-09T04:34:43.711301+0000","last_clean":"2024-05-09T04:34:43.711301+0000","last_became_active":"2024-05-09T04:34:42.694213+0000","last_became_peered":"2024-05-09T04:34:42.694213+0000","last_unstale":"2024-05-09T04:34:43.711301+0000","last_undegraded":"2024-05-09T04:34:43.711301+0000","last_fullsized":"2024-05-09T04:34:43.711301+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:51:57.033937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.712217+0000","last_change":"2024-05-09T04:34:43.704482+0000","last_active":"2024-05-09T04:34:43.712217+0000","last_peered":"2024-05-09T04:34:43.712217+0000","last_clean":"2024-05-09T04:34:43.712217+0000","last_became_active":"2024-05-09T04:34:43.703894+0000","last_became_peered":"2024-05-09T04:34:43.703894+0000","last_unstale":"2024-05-09T04:34:43.712217+0000","last_undegraded":"2024-05-09T04:34:43.712217+0000","last_fullsized":"2024-05-09T04:34:43.712217+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:52:21.080889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"31'1","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.691382+0000","last_change":"2024-05-09T04:34:42.683305+0000","last_active":"2024-05-09T04:34:44.691382+0000","last_peered":"2024-05-09T04:34:44.691382+0000","last_clean":"2024-05-09T04:34:44.691382+0000","last_became_active":"2024-05-09T04:34:42.682498+0000","last_became_peered":"2024-05-09T04:34:42.682498+0000","last_unstale":"2024-05-09T04:34:44.691382+0000","last_undegraded":"2024-05-09T04:34:44.691382+0000","last_fullsized":"2024-05-09T04:34:44.691382+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:47:31.029274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.688298+0000","last_change":"2024-05-09T04:34:43.702064+0000","last_active":"2024-05-09T04:34:44.688298+0000","last_peered":"2024-05-09T04:34:44.688298+0000","last_clean":"2024-05-09T04:34:44.688298+0000","last_became_active":"2024-05-09T04:34:43.698587+0000","last_became_peered":"2024-05-09T04:34:43.698587+0000","last_unstale":"2024-05-09T04:34:44.688298+0000","last_undegraded":"2024-05-09T04:34:44.688298+0000","last_fullsized":"2024-05-09T04:34:44.688298+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:48:53.850522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"31'2","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.685419+0000","last_change":"2024-05-09T04:34:42.689552+0000","last_active":"2024-05-09T04:34:44.685419+0000","last_peered":"2024-05-09T04:34:44.685419+0000","last_clean":"2024-05-09T04:34:44.685419+0000","last_became_active":"2024-05-09T04:34:42.689243+0000","last_became_peered":"2024-05-09T04:34:42.689243+0000","last_unstale":"2024-05-09T04:34:44.685419+0000","last_undegraded":"2024-05-09T04:34:44.685419+0000","last_fullsized":"2024-05-09T04:34:44.685419+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:47:37.225318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.694776+0000","last_change":"2024-05-09T04:34:43.684249+0000","last_active":"2024-05-09T04:34:44.694776+0000","last_peered":"2024-05-09T04:34:44.694776+0000","last_clean":"2024-05-09T04:34:44.694776+0000","last_became_active":"2024-05-09T04:34:43.683787+0000","last_became_peered":"2024-05-09T04:34:43.683787+0000","last_unstale":"2024-05-09T04:34:44.694776+0000","last_undegraded":"2024-05-09T04:34:44.694776+0000","last_fullsized":"2024-05-09T04:34:44.694776+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:07:08.077009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"31'2","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.684599+0000","last_change":"2024-05-09T04:34:42.682139+0000","last_active":"2024-05-09T04:34:44.684599+0000","last_peered":"2024-05-09T04:34:44.684599+0000","last_clean":"2024-05-09T04:34:44.684599+0000","last_became_active":"2024-05-09T04:34:42.681890+0000","last_became_peered":"2024-05-09T04:34:42.681890+0000","last_unstale":"2024-05-09T04:34:44.684599+0000","last_undegraded":"2024-05-09T04:34:44.684599+0000","last_fullsized":"2024-05-09T04:34:44.684599+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:40:02.921068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.712815+0000","last_change":"2024-05-09T04:34:43.708295+0000","last_active":"2024-05-09T04:34:43.712815+0000","last_peered":"2024-05-09T04:34:43.712815+0000","last_clean":"2024-05-09T04:34:43.712815+0000","last_became_active":"2024-05-09T04:34:43.707982+0000","last_became_peered":"2024-05-09T04:34:43.707982+0000","last_unstale":"2024-05-09T04:34:43.712815+0000","last_undegraded":"2024-05-09T04:34:43.712815+0000","last_fullsized":"2024-05-09T04:34:43.712815+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:03:31.499077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"31'1","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.688189+0000","last_change":"2024-05-09T04:34:42.683110+0000","last_active":"2024-05-09T04:34:44.688189+0000","last_peered":"2024-05-09T04:34:44.688189+0000","last_clean":"2024-05-09T04:34:44.688189+0000","last_became_active":"2024-05-09T04:34:42.682779+0000","last_became_peered":"2024-05-09T04:34:42.682779+0000","last_unstale":"2024-05-09T04:34:44.688189+0000","last_undegraded":"2024-05-09T04:34:44.688189+0000","last_fullsized":"2024-05-09T04:34:44.688189+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:51:30.947092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.710169+0000","last_change":"2024-05-09T04:34:43.696718+0000","last_active":"2024-05-09T04:34:43.710169+0000","last_peered":"2024-05-09T04:34:43.710169+0000","last_clean":"2024-05-09T04:34:43.710169+0000","last_became_active":"2024-05-09T04:34:43.696110+0000","last_became_peered":"2024-05-09T04:34:43.696110+0000","last_unstale":"2024-05-09T04:34:43.710169+0000","last_undegraded":"2024-05-09T04:34:43.710169+0000","last_fullsized":"2024-05-09T04:34:43.710169+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:13:44.029814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"31'1","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.688912+0000","last_change":"2024-05-09T04:34:42.683355+0000","last_active":"2024-05-09T04:34:44.688912+0000","last_peered":"2024-05-09T04:34:44.688912+0000","last_clean":"2024-05-09T04:34:44.688912+0000","last_became_active":"2024-05-09T04:34:42.682838+0000","last_became_peered":"2024-05-09T04:34:42.682838+0000","last_unstale":"2024-05-09T04:34:44.688912+0000","last_undegraded":"2024-05-09T04:34:44.688912+0000","last_fullsized":"2024-05-09T04:34:44.688912+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:14:41.788104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.687580+0000","last_change":"2024-05-09T04:34:43.701035+0000","last_active":"2024-05-09T04:34:44.687580+0000","last_peered":"2024-05-09T04:34:44.687580+0000","last_clean":"2024-05-09T04:34:44.687580+0000","last_became_active":"2024-05-09T04:34:43.691724+0000","last_became_peered":"2024-05-09T04:34:43.691724+0000","last_unstale":"2024-05-09T04:34:44.687580+0000","last_undegraded":"2024-05-09T04:34:44.687580+0000","last_fullsized":"2024-05-09T04:34:44.687580+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:03:31.499077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.684757+0000","last_change":"2024-05-09T04:34:42.681132+0000","last_active":"2024-05-09T04:34:44.684757+0000","last_peered":"2024-05-09T04:34:44.684757+0000","last_clean":"2024-05-09T04:34:44.684757+0000","last_became_active":"2024-05-09T04:34:42.680821+0000","last_became_peered":"2024-05-09T04:34:42.680821+0000","last_unstale":"2024-05-09T04:34:44.684757+0000","last_undegraded":"2024-05-09T04:34:44.684757+0000","last_fullsized":"2024-05-09T04:34:44.684757+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:37:24.627720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.694163+0000","last_change":"2024-05-09T04:34:43.690719+0000","last_active":"2024-05-09T04:34:44.694163+0000","last_peered":"2024-05-09T04:34:44.694163+0000","last_clean":"2024-05-09T04:34:44.694163+0000","last_became_active":"2024-05-09T04:34:43.689745+0000","last_became_peered":"2024-05-09T04:34:43.689745+0000","last_unstale":"2024-05-09T04:34:44.694163+0000","last_undegraded":"2024-05-09T04:34:44.694163+0000","last_fullsized":"2024-05-09T04:34:44.694163+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:59:34.370275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.689255+0000","last_change":"2024-05-09T04:34:42.689959+0000","last_active":"2024-05-09T04:34:44.689255+0000","last_peered":"2024-05-09T04:34:44.689255+0000","last_clean":"2024-05-09T04:34:44.689255+0000","last_became_active":"2024-05-09T04:34:42.689628+0000","last_became_peered":"2024-05-09T04:34:42.689628+0000","last_unstale":"2024-05-09T04:34:44.689255+0000","last_undegraded":"2024-05-09T04:34:44.689255+0000","last_fullsized":"2024-05-09T04:34:44.689255+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:04:49.709155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.711371+0000","last_change":"2024-05-09T04:34:43.707580+0000","last_active":"2024-05-09T04:34:43.711371+0000","last_peered":"2024-05-09T04:34:43.711371+0000","last_clean":"2024-05-09T04:34:43.711371+0000","last_became_active":"2024-05-09T04:34:43.707303+0000","last_became_peered":"2024-05-09T04:34:43.707303+0000","last_unstale":"2024-05-09T04:34:43.711371+0000","last_undegraded":"2024-05-09T04:34:43.711371+0000","last_fullsized":"2024-05-09T04:34:43.711371+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:59:34.370275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.685791+0000","last_change":"2024-05-09T04:34:42.688714+0000","last_active":"2024-05-09T04:34:44.685791+0000","last_peered":"2024-05-09T04:34:44.685791+0000","last_clean":"2024-05-09T04:34:44.685791+0000","last_became_active":"2024-05-09T04:34:42.687729+0000","last_became_peered":"2024-05-09T04:34:42.687729+0000","last_unstale":"2024-05-09T04:34:44.685791+0000","last_undegraded":"2024-05-09T04:34:44.685791+0000","last_fullsized":"2024-05-09T04:34:44.685791+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:03:30.493050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.685880+0000","last_change":"2024-05-09T04:34:43.692687+0000","last_active":"2024-05-09T04:34:44.685880+0000","last_peered":"2024-05-09T04:34:44.685880+0000","last_clean":"2024-05-09T04:34:44.685880+0000","last_became_active":"2024-05-09T04:34:43.692325+0000","last_became_peered":"2024-05-09T04:34:43.692325+0000","last_unstale":"2024-05-09T04:34:44.685880+0000","last_undegraded":"2024-05-09T04:34:44.685880+0000","last_fullsized":"2024-05-09T04:34:44.685880+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:50:14.994456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.691491+0000","last_change":"2024-05-09T04:34:42.683380+0000","last_active":"2024-05-09T04:34:44.691491+0000","last_peered":"2024-05-09T04:34:44.691491+0000","last_clean":"2024-05-09T04:34:44.691491+0000","last_became_active":"2024-05-09T04:34:42.681564+0000","last_became_peered":"2024-05-09T04:34:42.681564+0000","last_unstale":"2024-05-09T04:34:44.691491+0000","last_undegraded":"2024-05-09T04:34:44.691491+0000","last_fullsized":"2024-05-09T04:34:44.691491+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:51:30.947092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.685324+0000","last_change":"2024-05-09T04:34:43.709223+0000","last_active":"2024-05-09T04:34:44.685324+0000","last_peered":"2024-05-09T04:34:44.685324+0000","last_clean":"2024-05-09T04:34:44.685324+0000","last_became_active":"2024-05-09T04:34:43.708861+0000","last_became_peered":"2024-05-09T04:34:43.708861+0000","last_unstale":"2024-05-09T04:34:44.685324+0000","last_undegraded":"2024-05-09T04:34:44.685324+0000","last_fullsized":"2024-05-09T04:34:44.685324+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:54:33.072668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"31'3","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.711006+0000","last_change":"2024-05-09T04:34:42.683525+0000","last_active":"2024-05-09T04:34:43.711006+0000","last_peered":"2024-05-09T04:34:43.711006+0000","last_clean":"2024-05-09T04:34:43.711006+0000","last_became_active":"2024-05-09T04:34:42.683151+0000","last_became_peered":"2024-05-09T04:34:42.683151+0000","last_unstale":"2024-05-09T04:34:43.711006+0000","last_undegraded":"2024-05-09T04:34:43.711006+0000","last_fullsized":"2024-05-09T04:34:43.711006+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:47:37.225318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.699775+0000","last_change":"2024-05-09T04:34:43.693128+0000","last_active":"2024-05-09T04:34:43.699775+0000","last_peered":"2024-05-09T04:34:43.699775+0000","last_clean":"2024-05-09T04:34:43.699775+0000","last_became_active":"2024-05-09T04:34:43.692576+0000","last_became_peered":"2024-05-09T04:34:43.692576+0000","last_unstale":"2024-05-09T04:34:43.699775+0000","last_undegraded":"2024-05-09T04:34:43.699775+0000","last_fullsized":"2024-05-09T04:34:43.699775+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:08:58.949795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"31'6","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.685393+0000","last_change":"2024-05-09T04:34:42.686738+0000","last_active":"2024-05-09T04:34:44.685393+0000","last_peered":"2024-05-09T04:34:44.685393+0000","last_clean":"2024-05-09T04:34:44.685393+0000","last_became_active":"2024-05-09T04:34:42.685624+0000","last_became_peered":"2024-05-09T04:34:42.685624+0000","last_unstale":"2024-05-09T04:34:44.685393+0000","last_undegraded":"2024-05-09T04:34:44.685393+0000","last_fullsized":"2024-05-09T04:34:44.685393+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:04:33.372621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.694112+0000","last_change":"2024-05-09T04:34:43.689872+0000","last_active":"2024-05-09T04:34:44.694112+0000","last_peered":"2024-05-09T04:34:44.694112+0000","last_clean":"2024-05-09T04:34:44.694112+0000","last_became_active":"2024-05-09T04:34:43.689565+0000","last_became_peered":"2024-05-09T04:34:43.689565+0000","last_unstale":"2024-05-09T04:34:44.694112+0000","last_undegraded":"2024-05-09T04:34:44.694112+0000","last_fullsized":"2024-05-09T04:34:44.694112+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:11:23.998791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"31'2","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.688316+0000","last_change":"2024-05-09T04:34:42.691436+0000","last_active":"2024-05-09T04:34:44.688316+0000","last_peered":"2024-05-09T04:34:44.688316+0000","last_clean":"2024-05-09T04:34:44.688316+0000","last_became_active":"2024-05-09T04:34:42.691128+0000","last_became_peered":"2024-05-09T04:34:42.691128+0000","last_unstale":"2024-05-09T04:34:44.688316+0000","last_undegraded":"2024-05-09T04:34:44.688316+0000","last_fullsized":"2024-05-09T04:34:44.688316+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:51:57.033937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.685468+0000","last_change":"2024-05-09T04:34:43.701843+0000","last_active":"2024-05-09T04:34:44.685468+0000","last_peered":"2024-05-09T04:34:44.685468+0000","last_clean":"2024-05-09T04:34:44.685468+0000","last_became_active":"2024-05-09T04:34:43.701553+0000","last_became_peered":"2024-05-09T04:34:43.701553+0000","last_unstale":"2024-05-09T04:34:44.685468+0000","last_undegraded":"2024-05-09T04:34:44.685468+0000","last_fullsized":"2024-05-09T04:34:44.685468+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:57:06.820576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.694165+0000","last_change":"2024-05-09T04:34:42.691963+0000","last_active":"2024-05-09T04:34:44.694165+0000","last_peered":"2024-05-09T04:34:44.694165+0000","last_clean":"2024-05-09T04:34:44.694165+0000","last_became_active":"2024-05-09T04:34:42.691689+0000","last_became_peered":"2024-05-09T04:34:42.691689+0000","last_unstale":"2024-05-09T04:34:44.694165+0000","last_undegraded":"2024-05-09T04:34:44.694165+0000","last_fullsized":"2024-05-09T04:34:44.694165+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:47:37.225318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.711799+0000","last_change":"2024-05-09T04:34:43.706306+0000","last_active":"2024-05-09T04:34:43.711799+0000","last_peered":"2024-05-09T04:34:43.711799+0000","last_clean":"2024-05-09T04:34:43.711799+0000","last_became_active":"2024-05-09T04:34:43.705631+0000","last_became_peered":"2024-05-09T04:34:43.705631+0000","last_unstale":"2024-05-09T04:34:43.711799+0000","last_undegraded":"2024-05-09T04:34:43.711799+0000","last_fullsized":"2024-05-09T04:34:43.711799+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:13:44.029814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.684629+0000","last_change":"2024-05-09T04:34:42.687735+0000","last_active":"2024-05-09T04:34:44.684629+0000","last_peered":"2024-05-09T04:34:44.684629+0000","last_clean":"2024-05-09T04:34:44.684629+0000","last_became_active":"2024-05-09T04:34:42.687447+0000","last_became_peered":"2024-05-09T04:34:42.687447+0000","last_unstale":"2024-05-09T04:34:44.684629+0000","last_undegraded":"2024-05-09T04:34:44.684629+0000","last_fullsized":"2024-05-09T04:34:44.684629+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:13:43.023788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.706308+0000","last_change":"2024-05-09T04:34:43.700026+0000","last_active":"2024-05-09T04:34:43.706308+0000","last_peered":"2024-05-09T04:34:43.706308+0000","last_clean":"2024-05-09T04:34:43.706308+0000","last_became_active":"2024-05-09T04:34:43.699713+0000","last_became_peered":"2024-05-09T04:34:43.699713+0000","last_unstale":"2024-05-09T04:34:43.706308+0000","last_undegraded":"2024-05-09T04:34:43.706308+0000","last_fullsized":"2024-05-09T04:34:43.706308+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:07:39.701863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"31'8","reported_seq":22,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.684636+0000","last_change":"2024-05-09T04:34:42.689511+0000","last_active":"2024-05-09T04:34:44.684636+0000","last_peered":"2024-05-09T04:34:44.684636+0000","last_clean":"2024-05-09T04:34:44.684636+0000","last_became_active":"2024-05-09T04:34:42.688929+0000","last_became_peered":"2024-05-09T04:34:42.688929+0000","last_unstale":"2024-05-09T04:34:44.684636+0000","last_undegraded":"2024-05-09T04:34:44.684636+0000","last_fullsized":"2024-05-09T04:34:44.684636+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:24:02.108688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.688863+0000","last_change":"2024-05-09T04:34:43.695050+0000","last_active":"2024-05-09T04:34:44.688863+0000","last_peered":"2024-05-09T04:34:44.688863+0000","last_clean":"2024-05-09T04:34:44.688863+0000","last_became_active":"2024-05-09T04:34:43.694756+0000","last_became_peered":"2024-05-09T04:34:43.694756+0000","last_unstale":"2024-05-09T04:34:44.688863+0000","last_undegraded":"2024-05-09T04:34:44.688863+0000","last_fullsized":"2024-05-09T04:34:44.688863+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:59:34.370275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"31'1","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.694989+0000","last_change":"2024-05-09T04:34:42.684827+0000","last_active":"2024-05-09T04:34:44.694989+0000","last_peered":"2024-05-09T04:34:44.694989+0000","last_clean":"2024-05-09T04:34:44.694989+0000","last_became_active":"2024-05-09T04:34:42.684506+0000","last_became_peered":"2024-05-09T04:34:42.684506+0000","last_unstale":"2024-05-09T04:34:44.694989+0000","last_undegraded":"2024-05-09T04:34:44.694989+0000","last_fullsized":"2024-05-09T04:34:44.694989+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:12:31.271696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.711007+0000","last_change":"2024-05-09T04:34:43.699790+0000","last_active":"2024-05-09T04:34:43.711007+0000","last_peered":"2024-05-09T04:34:43.711007+0000","last_clean":"2024-05-09T04:34:43.711007+0000","last_became_active":"2024-05-09T04:34:43.699474+0000","last_became_peered":"2024-05-09T04:34:43.699474+0000","last_unstale":"2024-05-09T04:34:43.711007+0000","last_undegraded":"2024-05-09T04:34:43.711007+0000","last_fullsized":"2024-05-09T04:34:43.711007+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:48:53.850522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"31'2","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.688853+0000","last_change":"2024-05-09T04:34:42.692093+0000","last_active":"2024-05-09T04:34:44.688853+0000","last_peered":"2024-05-09T04:34:44.688853+0000","last_clean":"2024-05-09T04:34:44.688853+0000","last_became_active":"2024-05-09T04:34:42.691523+0000","last_became_peered":"2024-05-09T04:34:42.691523+0000","last_unstale":"2024-05-09T04:34:44.688853+0000","last_undegraded":"2024-05-09T04:34:44.688853+0000","last_fullsized":"2024-05-09T04:34:44.688853+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:11:52.548077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.694644+0000","last_change":"2024-05-09T04:34:43.691404+0000","last_active":"2024-05-09T04:34:44.694644+0000","last_peered":"2024-05-09T04:34:44.694644+0000","last_clean":"2024-05-09T04:34:44.694644+0000","last_became_active":"2024-05-09T04:34:43.691036+0000","last_became_peered":"2024-05-09T04:34:43.691036+0000","last_unstale":"2024-05-09T04:34:44.694644+0000","last_undegraded":"2024-05-09T04:34:44.694644+0000","last_fullsized":"2024-05-09T04:34:44.694644+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:21:15.637406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"31'1","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.688639+0000","last_change":"2024-05-09T04:34:42.680457+0000","last_active":"2024-05-09T04:34:44.688639+0000","last_peered":"2024-05-09T04:34:44.688639+0000","last_clean":"2024-05-09T04:34:44.688639+0000","last_became_active":"2024-05-09T04:34:42.679663+0000","last_became_peered":"2024-05-09T04:34:42.679663+0000","last_unstale":"2024-05-09T04:34:44.688639+0000","last_undegraded":"2024-05-09T04:34:44.688639+0000","last_fullsized":"2024-05-09T04:34:44.688639+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:47:31.029274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.687040+0000","last_change":"2024-05-09T04:34:43.704738+0000","last_active":"2024-05-09T04:34:44.687040+0000","last_peered":"2024-05-09T04:34:44.687040+0000","last_clean":"2024-05-09T04:34:44.687040+0000","last_became_active":"2024-05-09T04:34:43.704422+0000","last_became_peered":"2024-05-09T04:34:43.704422+0000","last_unstale":"2024-05-09T04:34:44.687040+0000","last_undegraded":"2024-05-09T04:34:44.687040+0000","last_fullsized":"2024-05-09T04:34:44.687040+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:33:16.575080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.688513+0000","last_change":"2024-05-09T04:34:42.683053+0000","last_active":"2024-05-09T04:34:44.688513+0000","last_peered":"2024-05-09T04:34:44.688513+0000","last_clean":"2024-05-09T04:34:44.688513+0000","last_became_active":"2024-05-09T04:34:42.682678+0000","last_became_peered":"2024-05-09T04:34:42.682678+0000","last_unstale":"2024-05-09T04:34:44.688513+0000","last_undegraded":"2024-05-09T04:34:44.688513+0000","last_fullsized":"2024-05-09T04:34:44.688513+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:10:00.199341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.683874+0000","last_change":"2024-05-09T04:34:43.691730+0000","last_active":"2024-05-09T04:34:44.683874+0000","last_peered":"2024-05-09T04:34:44.683874+0000","last_clean":"2024-05-09T04:34:44.683874+0000","last_became_active":"2024-05-09T04:34:43.691411+0000","last_became_peered":"2024-05-09T04:34:43.691411+0000","last_unstale":"2024-05-09T04:34:44.683874+0000","last_undegraded":"2024-05-09T04:34:44.683874+0000","last_fullsized":"2024-05-09T04:34:44.683874+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:10:01.205367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"31'2","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.691432+0000","last_change":"2024-05-09T04:34:42.678125+0000","last_active":"2024-05-09T04:34:44.691432+0000","last_peered":"2024-05-09T04:34:44.691432+0000","last_clean":"2024-05-09T04:34:44.691432+0000","last_became_active":"2024-05-09T04:34:42.677754+0000","last_became_peered":"2024-05-09T04:34:42.677754+0000","last_unstale":"2024-05-09T04:34:44.691432+0000","last_undegraded":"2024-05-09T04:34:44.691432+0000","last_fullsized":"2024-05-09T04:34:44.691432+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:07:38.695837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.684616+0000","last_change":"2024-05-09T04:34:43.704328+0000","last_active":"2024-05-09T04:34:44.684616+0000","last_peered":"2024-05-09T04:34:44.684616+0000","last_clean":"2024-05-09T04:34:44.684616+0000","last_became_active":"2024-05-09T04:34:43.698460+0000","last_became_peered":"2024-05-09T04:34:43.698460+0000","last_unstale":"2024-05-09T04:34:44.684616+0000","last_undegraded":"2024-05-09T04:34:44.684616+0000","last_fullsized":"2024-05-09T04:34:44.684616+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:12:32.277722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.681608+0000","last_change":"2024-05-09T04:34:42.694370+0000","last_active":"2024-05-09T04:34:43.681608+0000","last_peered":"2024-05-09T04:34:43.681608+0000","last_clean":"2024-05-09T04:34:43.681608+0000","last_became_active":"2024-05-09T04:34:42.693975+0000","last_became_peered":"2024-05-09T04:34:42.693975+0000","last_unstale":"2024-05-09T04:34:43.681608+0000","last_undegraded":"2024-05-09T04:34:43.681608+0000","last_fullsized":"2024-05-09T04:34:43.681608+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:53:40.510377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.684147+0000","last_change":"2024-05-09T04:34:43.697017+0000","last_active":"2024-05-09T04:34:44.684147+0000","last_peered":"2024-05-09T04:34:44.684147+0000","last_clean":"2024-05-09T04:34:44.684147+0000","last_became_active":"2024-05-09T04:34:43.696762+0000","last_became_peered":"2024-05-09T04:34:43.696762+0000","last_unstale":"2024-05-09T04:34:44.684147+0000","last_undegraded":"2024-05-09T04:34:44.684147+0000","last_fullsized":"2024-05-09T04:34:44.684147+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:11:53.554104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.687356+0000","last_change":"2024-05-09T04:34:42.685942+0000","last_active":"2024-05-09T04:34:44.687356+0000","last_peered":"2024-05-09T04:34:44.687356+0000","last_clean":"2024-05-09T04:34:44.687356+0000","last_became_active":"2024-05-09T04:34:42.685657+0000","last_became_peered":"2024-05-09T04:34:42.685657+0000","last_unstale":"2024-05-09T04:34:44.687356+0000","last_undegraded":"2024-05-09T04:34:44.687356+0000","last_fullsized":"2024-05-09T04:34:44.687356+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:42:56.870686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.707858+0000","last_change":"2024-05-09T04:34:43.701922+0000","last_active":"2024-05-09T04:34:43.707858+0000","last_peered":"2024-05-09T04:34:43.707858+0000","last_clean":"2024-05-09T04:34:43.707858+0000","last_became_active":"2024-05-09T04:34:43.701517+0000","last_became_peered":"2024-05-09T04:34:43.701517+0000","last_unstale":"2024-05-09T04:34:43.707858+0000","last_undegraded":"2024-05-09T04:34:43.707858+0000","last_fullsized":"2024-05-09T04:34:43.707858+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:47:38.231344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.685638+0000","last_change":"2024-05-09T04:34:42.682776+0000","last_active":"2024-05-09T04:34:43.685638+0000","last_peered":"2024-05-09T04:34:43.685638+0000","last_clean":"2024-05-09T04:34:43.685638+0000","last_became_active":"2024-05-09T04:34:42.682496+0000","last_became_peered":"2024-05-09T04:34:42.682496+0000","last_unstale":"2024-05-09T04:34:43.685638+0000","last_undegraded":"2024-05-09T04:34:43.685638+0000","last_fullsized":"2024-05-09T04:34:43.685638+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:33:15.569054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.683993+0000","last_change":"2024-05-09T04:34:43.695781+0000","last_active":"2024-05-09T04:34:44.683993+0000","last_peered":"2024-05-09T04:34:44.683993+0000","last_clean":"2024-05-09T04:34:44.683993+0000","last_became_active":"2024-05-09T04:34:43.695475+0000","last_became_peered":"2024-05-09T04:34:43.695475+0000","last_unstale":"2024-05-09T04:34:44.683993+0000","last_undegraded":"2024-05-09T04:34:44.683993+0000","last_fullsized":"2024-05-09T04:34:44.683993+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:51:31.953118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"31'2","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.694990+0000","last_change":"2024-05-09T04:34:42.675499+0000","last_active":"2024-05-09T04:34:44.694990+0000","last_peered":"2024-05-09T04:34:44.694990+0000","last_clean":"2024-05-09T04:34:44.694990+0000","last_became_active":"2024-05-09T04:34:42.672972+0000","last_became_peered":"2024-05-09T04:34:42.672972+0000","last_unstale":"2024-05-09T04:34:44.694990+0000","last_undegraded":"2024-05-09T04:34:44.694990+0000","last_fullsized":"2024-05-09T04:34:44.694990+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:33:15.569054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.710198+0000","last_change":"2024-05-09T04:34:43.696910+0000","last_active":"2024-05-09T04:34:43.710198+0000","last_peered":"2024-05-09T04:34:43.710198+0000","last_clean":"2024-05-09T04:34:43.710198+0000","last_became_active":"2024-05-09T04:34:43.694234+0000","last_became_peered":"2024-05-09T04:34:43.694234+0000","last_unstale":"2024-05-09T04:34:43.710198+0000","last_undegraded":"2024-05-09T04:34:43.710198+0000","last_fullsized":"2024-05-09T04:34:43.710198+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:37:25.633746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"32'1","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.693466+0000","last_change":"2024-05-09T04:34:42.674612+0000","last_active":"2024-05-09T04:34:44.693466+0000","last_peered":"2024-05-09T04:34:44.693466+0000","last_clean":"2024-05-09T04:34:44.693466+0000","last_became_active":"2024-05-09T04:34:42.674289+0000","last_became_peered":"2024-05-09T04:34:42.674289+0000","last_unstale":"2024-05-09T04:34:44.693466+0000","last_undegraded":"2024-05-09T04:34:44.693466+0000","last_fullsized":"2024-05-09T04:34:44.693466+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:11:52.548077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.691295+0000","last_change":"2024-05-09T04:34:43.688662+0000","last_active":"2024-05-09T04:34:44.691295+0000","last_peered":"2024-05-09T04:34:44.691295+0000","last_clean":"2024-05-09T04:34:44.691295+0000","last_became_active":"2024-05-09T04:34:43.688144+0000","last_became_peered":"2024-05-09T04:34:43.688144+0000","last_unstale":"2024-05-09T04:34:44.691295+0000","last_undegraded":"2024-05-09T04:34:44.691295+0000","last_fullsized":"2024-05-09T04:34:44.691295+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:13:44.029814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.694582+0000","last_change":"2024-05-09T04:34:42.675062+0000","last_active":"2024-05-09T04:34:44.694582+0000","last_peered":"2024-05-09T04:34:44.694582+0000","last_clean":"2024-05-09T04:34:44.694582+0000","last_became_active":"2024-05-09T04:34:42.674612+0000","last_became_peered":"2024-05-09T04:34:42.674612+0000","last_unstale":"2024-05-09T04:34:44.694582+0000","last_undegraded":"2024-05-09T04:34:44.694582+0000","last_fullsized":"2024-05-09T04:34:44.694582+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:52:20.074863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.694974+0000","last_change":"2024-05-09T04:34:43.690599+0000","last_active":"2024-05-09T04:34:43.694974+0000","last_peered":"2024-05-09T04:34:43.694974+0000","last_clean":"2024-05-09T04:34:43.694974+0000","last_became_active":"2024-05-09T04:34:43.690279+0000","last_became_peered":"2024-05-09T04:34:43.690279+0000","last_unstale":"2024-05-09T04:34:43.694974+0000","last_undegraded":"2024-05-09T04:34:43.694974+0000","last_fullsized":"2024-05-09T04:34:43.694974+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:47:32.035300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.691522+0000","last_change":"2024-05-09T04:34:42.678410+0000","last_active":"2024-05-09T04:34:44.691522+0000","last_peered":"2024-05-09T04:34:44.691522+0000","last_clean":"2024-05-09T04:34:44.691522+0000","last_became_active":"2024-05-09T04:34:42.678103+0000","last_became_peered":"2024-05-09T04:34:42.678103+0000","last_unstale":"2024-05-09T04:34:44.691522+0000","last_undegraded":"2024-05-09T04:34:44.691522+0000","last_fullsized":"2024-05-09T04:34:44.691522+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:10:00.199341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.691400+0000","last_change":"2024-05-09T04:34:43.699825+0000","last_active":"2024-05-09T04:34:44.691400+0000","last_peered":"2024-05-09T04:34:44.691400+0000","last_clean":"2024-05-09T04:34:44.691400+0000","last_became_active":"2024-05-09T04:34:43.694720+0000","last_became_peered":"2024-05-09T04:34:43.694720+0000","last_unstale":"2024-05-09T04:34:44.691400+0000","last_undegraded":"2024-05-09T04:34:44.691400+0000","last_fullsized":"2024-05-09T04:34:44.691400+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:37:25.633746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.694247+0000","last_change":"2024-05-09T04:34:42.684510+0000","last_active":"2024-05-09T04:34:44.694247+0000","last_peered":"2024-05-09T04:34:44.694247+0000","last_clean":"2024-05-09T04:34:44.694247+0000","last_became_active":"2024-05-09T04:34:42.684164+0000","last_became_peered":"2024-05-09T04:34:42.684164+0000","last_unstale":"2024-05-09T04:34:44.694247+0000","last_undegraded":"2024-05-09T04:34:44.694247+0000","last_fullsized":"2024-05-09T04:34:44.694247+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:07:38.695837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.684536+0000","last_change":"2024-05-09T04:34:43.687222+0000","last_active":"2024-05-09T04:34:44.684536+0000","last_peered":"2024-05-09T04:34:44.684536+0000","last_clean":"2024-05-09T04:34:44.684536+0000","last_became_active":"2024-05-09T04:34:43.684963+0000","last_became_peered":"2024-05-09T04:34:43.684963+0000","last_unstale":"2024-05-09T04:34:44.684536+0000","last_undegraded":"2024-05-09T04:34:44.684536+0000","last_fullsized":"2024-05-09T04:34:44.684536+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:29:30.265815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.687653+0000","last_change":"2024-05-09T04:34:42.680340+0000","last_active":"2024-05-09T04:34:44.687653+0000","last_peered":"2024-05-09T04:34:44.687653+0000","last_clean":"2024-05-09T04:34:44.687653+0000","last_became_active":"2024-05-09T04:34:42.680044+0000","last_became_peered":"2024-05-09T04:34:42.680044+0000","last_unstale":"2024-05-09T04:34:44.687653+0000","last_undegraded":"2024-05-09T04:34:44.687653+0000","last_fullsized":"2024-05-09T04:34:44.687653+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:53:40.510377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.687371+0000","last_change":"2024-05-09T04:34:43.687573+0000","last_active":"2024-05-09T04:34:44.687371+0000","last_peered":"2024-05-09T04:34:44.687371+0000","last_clean":"2024-05-09T04:34:44.687371+0000","last_became_active":"2024-05-09T04:34:43.684669+0000","last_became_peered":"2024-05-09T04:34:43.684669+0000","last_unstale":"2024-05-09T04:34:44.687371+0000","last_undegraded":"2024-05-09T04:34:44.687371+0000","last_fullsized":"2024-05-09T04:34:44.687371+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:37:25.633746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-05-09T04:34:44.684573+0000","last_change":"2024-05-09T04:34:42.687616+0000","last_active":"2024-05-09T04:34:44.684573+0000","last_peered":"2024-05-09T04:34:44.684573+0000","last_clean":"2024-05-09T04:34:44.684573+0000","last_became_active":"2024-05-09T04:34:42.687338+0000","last_became_peered":"2024-05-09T04:34:42.687338+0000","last_unstale":"2024-05-09T04:34:44.684573+0000","last_undegraded":"2024-05-09T04:34:44.684573+0000","last_fullsized":"2024-05-09T04:34:44.684573+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:08:57.943769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.712900+0000","last_change":"2024-05-09T04:34:43.708639+0000","last_active":"2024-05-09T04:34:43.712900+0000","last_peered":"2024-05-09T04:34:43.712900+0000","last_clean":"2024-05-09T04:34:43.712900+0000","last_became_active":"2024-05-09T04:34:43.708340+0000","last_became_peered":"2024-05-09T04:34:43.708340+0000","last_unstale":"2024-05-09T04:34:43.712900+0000","last_undegraded":"2024-05-09T04:34:43.712900+0000","last_fullsized":"2024-05-09T04:34:43.712900+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:42.635847+0000","last_clean_scrub_stamp":"2024-05-09T04:34:42.635847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:04:34.378647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.684736+0000","last_change":"2024-05-09T04:34:42.695559+0000","last_active":"2024-05-09T04:34:43.684736+0000","last_peered":"2024-05-09T04:34:43.684736+0000","last_clean":"2024-05-09T04:34:43.684736+0000","last_became_active":"2024-05-09T04:34:42.695234+0000","last_became_peered":"2024-05-09T04:34:42.695234+0000","last_unstale":"2024-05-09T04:34:43.684736+0000","last_undegraded":"2024-05-09T04:34:43.684736+0000","last_fullsized":"2024-05-09T04:34:43.684736+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:34:41.629821+0000","last_clean_scrub_stamp":"2024-05-09T04:34:41.629821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:10:00.199341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":130,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-05-09T04:34:43.683672+0000","last_change":"2024-05-09T04:33:41.092063+0000","last_active":"2024-05-09T04:34:43.683672+0000","last_peered":"2024-05-09T04:34:43.683672+0000","last_clean":"2024-05-09T04:34:43.683672+0000","last_became_active":"2024-05-09T04:33:41.091194+0000","last_became_peered":"2024-05-09T04:33:41.091194+0000","last_unstale":"2024-05-09T04:34:43.683672+0000","last_undegraded":"2024-05-09T04:34:43.683672+0000","last_fullsized":"2024-05-09T04:34:43.683672+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:40.045339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:40.045339+0000","last_clean_scrub_stamp":"2024-05-09T04:33:40.045339+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:04.908456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":946176,"data_stored":724827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":356352,"data_stored":134985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":356352,"data_stored":135041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":356352,"data_stored":134473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":364544,"data_stored":136003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":352256,"data_stored":134439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":364544,"data_stored":136003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":942080,"data_stored":724737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T04:34:48.604 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-09T04:34:48.604 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-09T04:34:48.855 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.855+0000 7f41f2c00640 1 Processor -- start 2024-05-09T04:34:48.855 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.855+0000 7f41f2c00640 1 -- start start 2024-05-09T04:34:48.857 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.857+0000 7f41f2c00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f41ec0af8c0 0x7f41ec0d0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:48.857 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.857+0000 7f41f2c00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f41ec15d180 0x7f41ec06f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:48.857 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.857+0000 7f41f2c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f41ec070150 0x7f41ec0afca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:48.857 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.857+0000 7f41f2c00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f41ec0add40 con 0x7f41ec15d180 2024-05-09T04:34:48.857 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.857+0000 7f41f2c00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f41ec0b01e0 con 0x7f41ec0af8c0 2024-05-09T04:34:48.857 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.857+0000 7f41f2c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f41ec0d0260 con 0x7f41ec070150 2024-05-09T04:34:48.857 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.857+0000 7f41f0e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f41ec15d180 0x7f41ec06f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:48.858 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.857+0000 7f41f0e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f41ec15d180 0x7f41ec06f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:59358/0 (socket says 172.21.15.145:59358) 2024-05-09T04:34:48.858 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.857+0000 7f41f0e00640 1 -- 172.21.15.145:0/1417321766 learned_addr learned my addr 172.21.15.145:0/1417321766 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:48.858 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.857+0000 7f41f1800640 1 --2- 172.21.15.145:0/1417321766 >> v2:172.21.15.192:3300/0 conn(0x7f41ec0af8c0 0x7f41ec0d0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:48.858 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.857+0000 7f41f2200640 1 --2- 172.21.15.145:0/1417321766 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f41ec070150 0x7f41ec0afca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:48.858 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.857+0000 7f41f0e00640 1 -- 172.21.15.145:0/1417321766 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f41ec070150 msgr2=0x7f41ec0afca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:48.858 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.858+0000 7f41f0e00640 1 --2- 172.21.15.145:0/1417321766 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f41ec070150 0x7f41ec0afca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:48.858 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.858+0000 7f41f0e00640 1 -- 172.21.15.145:0/1417321766 >> v2:172.21.15.192:3300/0 conn(0x7f41ec0af8c0 msgr2=0x7f41ec0d0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:48.858 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.858+0000 7f41f0e00640 1 --2- 172.21.15.145:0/1417321766 >> v2:172.21.15.192:3300/0 conn(0x7f41ec0af8c0 0x7f41ec0d0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:48.858 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.858+0000 7f41f0e00640 1 -- 172.21.15.145:0/1417321766 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41ec0adf90 con 0x7f41ec15d180 2024-05-09T04:34:48.860 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.859+0000 7f41f0e00640 1 --2- 172.21.15.145:0/1417321766 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f41ec15d180 0x7f41ec06f2c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f41ec0b0830 tx=0x7f41d802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c470149522f18c0e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:48.860 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.859+0000 7f41ebe00640 1 -- 172.21.15.145:0/1417321766 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d8037070 con 0x7f41ec15d180 2024-05-09T04:34:48.860 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.859+0000 7f41ebe00640 1 -- 172.21.15.145:0/1417321766 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41d8032440 con 0x7f41ec15d180 2024-05-09T04:34:48.860 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.859+0000 7f41ebe00640 1 -- 172.21.15.145:0/1417321766 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d8032810 con 0x7f41ec15d180 2024-05-09T04:34:48.860 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.860+0000 7f41f2c00640 1 -- 172.21.15.145:0/1417321766 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f41ec15d180 msgr2=0x7f41ec06f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:48.860 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.860+0000 7f41f2c00640 1 --2- 172.21.15.145:0/1417321766 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f41ec15d180 0x7f41ec06f2c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f41ec0b0830 tx=0x7f41d802a1e0 comp rx=0 tx=0).stop 2024-05-09T04:34:48.860 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.860+0000 7f41f2c00640 1 -- 172.21.15.145:0/1417321766 shutdown_connections 2024-05-09T04:34:48.860 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.860+0000 7f41f2c00640 1 --2- 172.21.15.145:0/1417321766 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f41ec070150 0x7f41ec0afca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:48.860 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.860+0000 7f41f2c00640 1 --2- 172.21.15.145:0/1417321766 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f41ec15d180 0x7f41ec06f2c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:48.860 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.860+0000 7f41f2c00640 1 --2- 172.21.15.145:0/1417321766 >> v2:172.21.15.192:3300/0 conn(0x7f41ec0af8c0 0x7f41ec0d0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:48.860 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.860+0000 7f41f2c00640 1 -- 172.21.15.145:0/1417321766 >> 172.21.15.145:0/1417321766 conn(0x7f41ec0b29f0 msgr2=0x7f41ec0b2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:48.860 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.860+0000 7f41f2c00640 1 -- 172.21.15.145:0/1417321766 shutdown_connections 2024-05-09T04:34:48.860 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.860+0000 7f41f2c00640 1 -- 172.21.15.145:0/1417321766 wait complete. 2024-05-09T04:34:48.861 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.861+0000 7f41f2c00640 1 Processor -- start 2024-05-09T04:34:48.861 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.862+0000 7f41f2c00640 1 -- start start 2024-05-09T04:34:48.863 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.862+0000 7f41f2c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f41ec070150 0x7f41ec15e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:48.863 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.862+0000 7f41f2c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f41ec0af8c0 0x7f41ec14fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:48.863 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.863+0000 7f41f2c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41ec15d180 0x7f41ec155d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:48.863 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.863+0000 7f41f2c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f41ec0ae270 con 0x7f41ec0af8c0 2024-05-09T04:34:48.863 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.863+0000 7f41f2c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f41ec0cfef0 con 0x7f41ec15d180 2024-05-09T04:34:48.863 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.863+0000 7f41f2c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f41ec072190 con 0x7f41ec070150 2024-05-09T04:34:48.863 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.863+0000 7f41f0e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f41ec0af8c0 0x7f41ec14fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:48.863 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.863+0000 7f41f0e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f41ec0af8c0 0x7f41ec14fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:59364/0 (socket says 172.21.15.145:59364) 2024-05-09T04:34:48.863 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.863+0000 7f41f0e00640 1 -- 172.21.15.145:0/424461400 learned_addr learned my addr 172.21.15.145:0/424461400 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:48.863 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.863+0000 7f41f0e00640 1 -- 172.21.15.145:0/424461400 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f41ec070150 msgr2=0x7f41ec15e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:48.863 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.863+0000 7f41f0e00640 1 --2- 172.21.15.145:0/424461400 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f41ec070150 0x7f41ec15e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:48.864 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.863+0000 7f41f0e00640 1 -- 172.21.15.145:0/424461400 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41ec15d180 msgr2=0x7f41ec155d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:48.864 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.863+0000 7f41f0e00640 1 --2- 172.21.15.145:0/424461400 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41ec15d180 0x7f41ec155d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:48.864 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.863+0000 7f41f0e00640 1 -- 172.21.15.145:0/424461400 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41ec154ca0 con 0x7f41ec0af8c0 2024-05-09T04:34:48.865 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.864+0000 7f41f0e00640 1 --2- 172.21.15.145:0/424461400 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f41ec0af8c0 0x7f41ec14fc70 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f41d80097c0 tx=0x7f41d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:48.865 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.864+0000 7f41e9600640 1 -- 172.21.15.145:0/424461400 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d8041020 con 0x7f41ec0af8c0 2024-05-09T04:34:48.865 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.864+0000 7f41e9600640 1 -- 172.21.15.145:0/424461400 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41d80306c0 con 0x7f41ec0af8c0 2024-05-09T04:34:48.865 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.864+0000 7f41f2c00640 1 -- 172.21.15.145:0/424461400 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41ec154f80 con 0x7f41ec0af8c0 2024-05-09T04:34:48.865 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.864+0000 7f41e9600640 1 -- 172.21.15.145:0/424461400 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d8037040 con 0x7f41ec0af8c0 2024-05-09T04:34:48.865 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.864+0000 7f41f2c00640 1 -- 172.21.15.145:0/424461400 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41ec155540 con 0x7f41ec0af8c0 2024-05-09T04:34:48.865 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.865+0000 7f41f2c00640 1 -- 172.21.15.145:0/424461400 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41ec0aede0 con 0x7f41ec0af8c0 2024-05-09T04:34:48.866 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.866+0000 7f41e9600640 1 -- 172.21.15.145:0/424461400 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f41d8044020 con 0x7f41ec0af8c0 2024-05-09T04:34:48.867 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.867+0000 7f41e9600640 1 --2- 172.21.15.145:0/424461400 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f41cc0762d0 0x7f41cc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:48.867 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.867+0000 7f41e9600640 1 -- 172.21.15.145:0/424461400 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f41d80b7de0 con 0x7f41ec0af8c0 2024-05-09T04:34:48.867 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.867+0000 7f41f1800640 1 --2- 172.21.15.145:0/424461400 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f41cc0762d0 0x7f41cc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:48.867 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.868+0000 7f41f1800640 1 --2- 172.21.15.145:0/424461400 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f41cc0762d0 0x7f41cc078790 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f41e00098f0 tx=0x7f41e0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:48.871 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:48.871+0000 7f41e9600640 1 -- 172.21.15.145:0/424461400 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f41ec0aede0 con 0x7f41ec0af8c0 2024-05-09T04:34:49.157 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.157+0000 7f41f2c00640 1 -- 172.21.15.145:0/424461400 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f41ec076890 con 0x7f41ec0af8c0 2024-05-09T04:34:49.162 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.162+0000 7f41e9600640 1 -- 172.21.15.145:0/424461400 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) ==== 190+0+147 (secure 0 0 0) 0x7f41d804b030 con 0x7f41ec0af8c0 2024-05-09T04:34:49.162 INFO:teuthology.orchestra.run.smithi145.stdout:[client.0] 2024-05-09T04:34:49.162 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQDmUTxmfB7QDRAAs0TGpMOXJSerMXVrobw+hA== 2024-05-09T04:34:49.162 INFO:teuthology.orchestra.run.smithi145.stdout: caps mds = "allow" 2024-05-09T04:34:49.163 INFO:teuthology.orchestra.run.smithi145.stdout: caps mon = "allow r" 2024-05-09T04:34:49.163 INFO:teuthology.orchestra.run.smithi145.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-09T04:34:49.163 INFO:teuthology.orchestra.run.smithi145.stderr:updated caps for client.0 2024-05-09T04:34:49.165 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.165+0000 7f41f2c00640 1 -- 172.21.15.145:0/424461400 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f41cc0762d0 msgr2=0x7f41cc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:49.165 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.165+0000 7f41f2c00640 1 --2- 172.21.15.145:0/424461400 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f41cc0762d0 0x7f41cc078790 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f41e00098f0 tx=0x7f41e0008040 comp rx=0 tx=0).stop 2024-05-09T04:34:49.165 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.165+0000 7f41f2c00640 1 -- 172.21.15.145:0/424461400 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f41ec0af8c0 msgr2=0x7f41ec14fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:49.165 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.165+0000 7f41f2c00640 1 --2- 172.21.15.145:0/424461400 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f41ec0af8c0 0x7f41ec14fc70 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f41d80097c0 tx=0x7f41d80046c0 comp rx=0 tx=0).stop 2024-05-09T04:34:49.166 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.165+0000 7f41f2c00640 1 -- 172.21.15.145:0/424461400 shutdown_connections 2024-05-09T04:34:49.166 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.165+0000 7f41f2c00640 1 --2- 172.21.15.145:0/424461400 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f41cc0762d0 0x7f41cc078790 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:49.166 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.165+0000 7f41f2c00640 1 --2- 172.21.15.145:0/424461400 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41ec15d180 0x7f41ec155d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:49.166 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.165+0000 7f41f2c00640 1 --2- 172.21.15.145:0/424461400 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f41ec0af8c0 0x7f41ec14fc70 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:49.166 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.165+0000 7f41f2c00640 1 --2- 172.21.15.145:0/424461400 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f41ec070150 0x7f41ec15e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:49.166 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.165+0000 7f41f2c00640 1 -- 172.21.15.145:0/424461400 >> 172.21.15.145:0/424461400 conn(0x7f41ec0b29f0 msgr2=0x7f41ec0d0b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:49.166 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.165+0000 7f41f2c00640 1 -- 172.21.15.145:0/424461400 shutdown_connections 2024-05-09T04:34:49.166 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.165+0000 7f41f2c00640 1 -- 172.21.15.145:0/424461400 wait complete. 2024-05-09T04:34:49.184 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T04:34:49.383 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.383+0000 7ff391e00640 1 Processor -- start 2024-05-09T04:34:49.383 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.383+0000 7ff391e00640 1 -- start start 2024-05-09T04:34:49.385 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.385+0000 7ff391e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff38c15bae0 0x7ff38c0721b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:49.385 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.385+0000 7ff391e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff38c15bec0 0x7ff38c15f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:49.385 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.385+0000 7ff391e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff38c15fde0 0x7ff38c0cfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:49.386 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.385+0000 7ff391e00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7ff38c160840 con 0x7ff38c15bec0 2024-05-09T04:34:49.386 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.385+0000 7ff391e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7ff38c0b4190 con 0x7ff38c15bae0 2024-05-09T04:34:49.386 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.385+0000 7ff391e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff38c160280 con 0x7ff38c15fde0 2024-05-09T04:34:49.386 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.385+0000 7ff38be00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff38c15bec0 0x7ff38c15f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:49.386 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.385+0000 7ff38be00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff38c15bec0 0x7ff38c15f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:59374/0 (socket says 172.21.15.145:59374) 2024-05-09T04:34:49.386 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.385+0000 7ff38be00640 1 -- 172.21.15.145:0/3288012167 learned_addr learned my addr 172.21.15.145:0/3288012167 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:49.386 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.385+0000 7ff390a00640 1 --2- 172.21.15.145:0/3288012167 >> v2:172.21.15.192:3300/0 conn(0x7ff38c15bae0 0x7ff38c0721b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:49.386 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.385+0000 7ff391400640 1 --2- 172.21.15.145:0/3288012167 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff38c15fde0 0x7ff38c0cfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:49.386 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.386+0000 7ff38be00640 1 -- 172.21.15.145:0/3288012167 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff38c15fde0 msgr2=0x7ff38c0cfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:49.386 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.386+0000 7ff38be00640 1 --2- 172.21.15.145:0/3288012167 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff38c15fde0 0x7ff38c0cfe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:49.386 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.386+0000 7ff38be00640 1 -- 172.21.15.145:0/3288012167 >> v2:172.21.15.192:3300/0 conn(0x7ff38c15bae0 msgr2=0x7ff38c0721b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:49.386 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.386+0000 7ff38be00640 1 --2- 172.21.15.145:0/3288012167 >> v2:172.21.15.192:3300/0 conn(0x7ff38c15bae0 0x7ff38c0721b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:49.386 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.386+0000 7ff38be00640 1 -- 172.21.15.145:0/3288012167 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff38c160560 con 0x7ff38c15bec0 2024-05-09T04:34:49.388 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.387+0000 7ff38be00640 1 --2- 172.21.15.145:0/3288012167 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff38c15bec0 0x7ff38c15f8a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff38c06eb80 tx=0x7ff37802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37aeb89e00faf416 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:49.388 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.387+0000 7ff38b400640 1 -- 172.21.15.145:0/3288012167 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff378037070 con 0x7ff38c15bec0 2024-05-09T04:34:49.388 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.387+0000 7ff38b400640 1 -- 172.21.15.145:0/3288012167 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff378032440 con 0x7ff38c15bec0 2024-05-09T04:34:49.388 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.387+0000 7ff38b400640 1 -- 172.21.15.145:0/3288012167 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff378032810 con 0x7ff38c15bec0 2024-05-09T04:34:49.388 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.388+0000 7ff391e00640 1 -- 172.21.15.145:0/3288012167 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff38c15bec0 msgr2=0x7ff38c15f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:49.388 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.388+0000 7ff391e00640 1 --2- 172.21.15.145:0/3288012167 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff38c15bec0 0x7ff38c15f8a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff38c06eb80 tx=0x7ff37802a140 comp rx=0 tx=0).stop 2024-05-09T04:34:49.388 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.388+0000 7ff391e00640 1 -- 172.21.15.145:0/3288012167 shutdown_connections 2024-05-09T04:34:49.388 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.388+0000 7ff391e00640 1 --2- 172.21.15.145:0/3288012167 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff38c15fde0 0x7ff38c0cfe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:49.388 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.388+0000 7ff391e00640 1 --2- 172.21.15.145:0/3288012167 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff38c15bec0 0x7ff38c15f8a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:49.388 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.388+0000 7ff391e00640 1 --2- 172.21.15.145:0/3288012167 >> v2:172.21.15.192:3300/0 conn(0x7ff38c15bae0 0x7ff38c0721b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:49.388 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.388+0000 7ff391e00640 1 -- 172.21.15.145:0/3288012167 >> 172.21.15.145:0/3288012167 conn(0x7ff38c06c8a0 msgr2=0x7ff38c06ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:49.388 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.388+0000 7ff391e00640 1 -- 172.21.15.145:0/3288012167 shutdown_connections 2024-05-09T04:34:49.388 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.388+0000 7ff391e00640 1 -- 172.21.15.145:0/3288012167 wait complete. 2024-05-09T04:34:49.390 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.390+0000 7ff391e00640 1 Processor -- start 2024-05-09T04:34:49.390 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.390+0000 7ff391e00640 1 -- start start 2024-05-09T04:34:49.392 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.391+0000 7ff391e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff38c15bae0 0x7ff38c14ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:49.392 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.391+0000 7ff391e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff38c15bec0 0x7ff38c14f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:49.392 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.392+0000 7ff391e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff38c15fde0 0x7ff38c0b26c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:49.392 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.392+0000 7ff391e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff38c0aded0 con 0x7ff38c15fde0 2024-05-09T04:34:49.393 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.392+0000 7ff391e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff38c0726f0 con 0x7ff38c15bae0 2024-05-09T04:34:49.393 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.392+0000 7ff391e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff38c0ae050 con 0x7ff38c15bec0 2024-05-09T04:34:49.393 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.392+0000 7ff391400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff38c15fde0 0x7ff38c0b26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:49.393 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.392+0000 7ff391400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff38c15fde0 0x7ff38c0b26c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:59382/0 (socket says 172.21.15.145:59382) 2024-05-09T04:34:49.393 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.392+0000 7ff391400640 1 -- 172.21.15.145:0/781559535 learned_addr learned my addr 172.21.15.145:0/781559535 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:49.393 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.392+0000 7ff390a00640 1 --2- 172.21.15.145:0/781559535 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff38c15bae0 0x7ff38c14ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:49.393 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.392+0000 7ff38be00640 1 --2- 172.21.15.145:0/781559535 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff38c15bec0 0x7ff38c14f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:49.393 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.392+0000 7ff391400640 1 -- 172.21.15.145:0/781559535 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff38c15bec0 msgr2=0x7ff38c14f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:49.393 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.392+0000 7ff391400640 1 --2- 172.21.15.145:0/781559535 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff38c15bec0 0x7ff38c14f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:49.393 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.392+0000 7ff391400640 1 -- 172.21.15.145:0/781559535 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff38c15bae0 msgr2=0x7ff38c14ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:49.393 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.392+0000 7ff391400640 1 --2- 172.21.15.145:0/781559535 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff38c15bae0 0x7ff38c14ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:49.393 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.392+0000 7ff391400640 1 -- 172.21.15.145:0/781559535 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff38c15dae0 con 0x7ff38c15fde0 2024-05-09T04:34:49.393 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.392+0000 7ff38be00640 1 --2- 172.21.15.145:0/781559535 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff38c15bec0 0x7ff38c14f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:34:49.393 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.393+0000 7ff391400640 1 --2- 172.21.15.145:0/781559535 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff38c15fde0 0x7ff38c0b26c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff384007e60 tx=0x7ff38400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:49.395 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.393+0000 7ff388c00640 1 -- 172.21.15.145:0/781559535 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff384014420 con 0x7ff38c15fde0 2024-05-09T04:34:49.395 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.393+0000 7ff388c00640 1 -- 172.21.15.145:0/781559535 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff384019070 con 0x7ff38c15fde0 2024-05-09T04:34:49.395 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.393+0000 7ff388c00640 1 -- 172.21.15.145:0/781559535 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff384014820 con 0x7ff38c15fde0 2024-05-09T04:34:49.395 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.393+0000 7ff391e00640 1 -- 172.21.15.145:0/781559535 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff38c15ddc0 con 0x7ff38c15fde0 2024-05-09T04:34:49.395 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.393+0000 7ff391e00640 1 -- 172.21.15.145:0/781559535 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff38c15d340 con 0x7ff38c15fde0 2024-05-09T04:34:49.395 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.394+0000 7ff391e00640 1 -- 172.21.15.145:0/781559535 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff38c06a310 con 0x7ff38c15fde0 2024-05-09T04:34:49.395 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.396+0000 7ff388c00640 1 -- 172.21.15.145:0/781559535 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff3840149c0 con 0x7ff38c15fde0 2024-05-09T04:34:49.396 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.397+0000 7ff388c00640 1 --2- 172.21.15.145:0/781559535 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff36c0762d0 0x7ff36c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:49.396 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.397+0000 7ff388c00640 1 -- 172.21.15.145:0/781559535 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7ff384097e00 con 0x7ff38c15fde0 2024-05-09T04:34:49.396 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.397+0000 7ff390a00640 1 --2- 172.21.15.145:0/781559535 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff36c0762d0 0x7ff36c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:49.397 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.398+0000 7ff390a00640 1 --2- 172.21.15.145:0/781559535 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff36c0762d0 0x7ff36c078790 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff380006fd0 tx=0x7ff380008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:49.401 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.401+0000 7ff388c00640 1 -- 172.21.15.145:0/781559535 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff384061100 con 0x7ff38c15fde0 2024-05-09T04:34:49.645 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.645+0000 7ff391e00640 1 -- 172.21.15.145:0/781559535 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff38c14d300 con 0x7ff38c15fde0 2024-05-09T04:34:49.646 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.646+0000 7ff388c00640 1 -- 172.21.15.145:0/781559535 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5187 (secure 0 0 0) 0x7ff384065400 con 0x7ff38c15fde0 2024-05-09T04:34:49.646 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:34:49.646 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4499,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6834/3227728059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3227728059},{"type":"v1","addr":"172.21.15.192:6834","nonce":3227728059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4502,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1603013787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6833","nonce":1603013787},{"type":"v1","addr":"172.21.15.192:6835","nonce":1603013787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4672,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/68778418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":68778418},{"type":"v1","addr":"172.21.15.145:6837","nonce":68778418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T04:34:43.133970+0000","modified":"2024-05-09T04:34:46.181657+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4761},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4761":{"gid":4761,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.145:6835/4260798843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4260798843},{"type":"v1","addr":"172.21.15.145:6835","nonce":4260798843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4761,"qdb_cluster":[4761]},"id":2}]} 2024-05-09T04:34:49.647 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 17 2024-05-09T04:34:49.648 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.649+0000 7ff391e00640 1 -- 172.21.15.145:0/781559535 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff36c0762d0 msgr2=0x7ff36c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:49.649 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.649+0000 7ff391e00640 1 --2- 172.21.15.145:0/781559535 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff36c0762d0 0x7ff36c078790 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff380006fd0 tx=0x7ff380008040 comp rx=0 tx=0).stop 2024-05-09T04:34:49.649 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.649+0000 7ff391e00640 1 -- 172.21.15.145:0/781559535 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff38c15fde0 msgr2=0x7ff38c0b26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:49.649 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.649+0000 7ff391e00640 1 --2- 172.21.15.145:0/781559535 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff38c15fde0 0x7ff38c0b26c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff384007e60 tx=0x7ff38400e4b0 comp rx=0 tx=0).stop 2024-05-09T04:34:49.650 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.649+0000 7ff391e00640 1 -- 172.21.15.145:0/781559535 shutdown_connections 2024-05-09T04:34:49.650 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.649+0000 7ff391e00640 1 --2- 172.21.15.145:0/781559535 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff36c0762d0 0x7ff36c078790 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:49.650 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.649+0000 7ff391e00640 1 --2- 172.21.15.145:0/781559535 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff38c15fde0 0x7ff38c0b26c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:49.650 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.649+0000 7ff391e00640 1 --2- 172.21.15.145:0/781559535 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff38c15bec0 0x7ff38c14f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:49.650 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.649+0000 7ff391e00640 1 --2- 172.21.15.145:0/781559535 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff38c15bae0 0x7ff38c14ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:49.650 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.649+0000 7ff391e00640 1 -- 172.21.15.145:0/781559535 >> 172.21.15.145:0/781559535 conn(0x7ff38c06c8a0 msgr2=0x7ff38c06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:49.650 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.649+0000 7ff391e00640 1 -- 172.21.15.145:0/781559535 shutdown_connections 2024-05-09T04:34:49.650 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:49.649+0000 7ff391e00640 1 -- 172.21.15.145:0/781559535 wait complete. 2024-05-09T04:34:50.070 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-09T04:34:43.133970+0000', 'modified': '2024-05-09T04:34:46.181657+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4761}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4761': {'gid': 4761, 'name': 'c', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.145:6835/4260798843', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6834', 'nonce': 4260798843}, {'type': 'v1', 'addr': '172.21.15.145:6835', 'nonce': 4260798843}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4761, 'qdb_cluster': [4761]} 2024-05-09T04:34:50.070 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-09T04:34:50.070 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-09T04:34:50.276 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.276+0000 7fb526200640 1 Processor -- start 2024-05-09T04:34:50.276 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.276+0000 7fb526200640 1 -- start start 2024-05-09T04:34:50.278 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.278+0000 7fb526200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fb5200add40 0x7fb5200ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:50.278 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.278+0000 7fb526200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb52006fa50 0x7fb52006e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:50.278 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.278+0000 7fb526200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5200cf300 con 0x7fb52006c8a0 2024-05-09T04:34:50.278 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.278+0000 7fb526200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fb52006e1f0 con 0x7fb5200add40 2024-05-09T04:34:50.278 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.278+0000 7fb526200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fb5200cf5e0 con 0x7fb52006fa50 2024-05-09T04:34:50.278 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.278+0000 7fb524e00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7fb52006c8a0 0x7fb52006cc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:49054/0 (socket says 172.21.15.145:49054) 2024-05-09T04:34:50.278 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.278+0000 7fb524e00640 1 -- 172.21.15.145:0/2751157997 learned_addr learned my addr 172.21.15.145:0/2751157997 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:34:50.278 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.278+0000 7fb517e00640 1 --2- 172.21.15.145:0/2751157997 >> v2:172.21.15.145:3300/0 conn(0x7fb5200add40 0x7fb5200ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:50.280 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.279+0000 7fb525800640 1 --2- 172.21.15.145:0/2751157997 >> v2:172.21.15.192:3300/0 conn(0x7fb52006fa50 0x7fb52006e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:50.280 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.279+0000 7fb51fe00640 1 -- 172.21.15.145:0/2751157997 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1523534698 0 0) 0x7fb5200cf300 con 0x7fb52006c8a0 2024-05-09T04:34:50.280 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.279+0000 7fb51fe00640 1 -- 172.21.15.145:0/2751157997 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb504003520 con 0x7fb52006c8a0 2024-05-09T04:34:50.280 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.279+0000 7fb517e00640 1 -- 172.21.15.145:0/2751157997 >> v2:172.21.15.192:3300/0 conn(0x7fb52006fa50 msgr2=0x7fb52006e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:50.280 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.279+0000 7fb517e00640 1 --2- 172.21.15.145:0/2751157997 >> v2:172.21.15.192:3300/0 conn(0x7fb52006fa50 0x7fb52006e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:50.280 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.279+0000 7fb517e00640 1 -- 172.21.15.145:0/2751157997 >> v1:172.21.15.145:6789/0 conn(0x7fb52006c8a0 legacy=0x7fb52006cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:50.280 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.279+0000 7fb517e00640 1 -- 172.21.15.145:0/2751157997 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5200cf8c0 con 0x7fb5200add40 2024-05-09T04:34:50.280 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.280+0000 7fb517e00640 1 --2- 172.21.15.145:0/2751157997 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb5200add40 0x7fb5200ceab0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb5200b28f0 tx=0x7fb50802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a08433da8cc27e80 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:50.280 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.280+0000 7fb51fe00640 1 -- 172.21.15.145:0/2751157997 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb508037070 con 0x7fb5200add40 2024-05-09T04:34:50.280 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.280+0000 7fb51fe00640 1 -- 172.21.15.145:0/2751157997 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb508032440 con 0x7fb5200add40 2024-05-09T04:34:50.280 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.280+0000 7fb51fe00640 1 -- 172.21.15.145:0/2751157997 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb50802ae60 con 0x7fb5200add40 2024-05-09T04:34:50.281 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.281+0000 7fb526200640 1 -- 172.21.15.145:0/2751157997 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb5200add40 msgr2=0x7fb5200ceab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:50.281 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.281+0000 7fb526200640 1 --2- 172.21.15.145:0/2751157997 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb5200add40 0x7fb5200ceab0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb5200b28f0 tx=0x7fb50802a110 comp rx=0 tx=0).stop 2024-05-09T04:34:50.281 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.281+0000 7fb526200640 1 -- 172.21.15.145:0/2751157997 shutdown_connections 2024-05-09T04:34:50.281 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.281+0000 7fb526200640 1 --2- 172.21.15.145:0/2751157997 >> v2:172.21.15.192:3300/0 conn(0x7fb52006fa50 0x7fb52006e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:50.281 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.281+0000 7fb526200640 1 --2- 172.21.15.145:0/2751157997 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb5200add40 0x7fb5200ceab0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:50.281 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.281+0000 7fb526200640 1 -- 172.21.15.145:0/2751157997 >> 172.21.15.145:0/2751157997 conn(0x7fb5200b2d80 msgr2=0x7fb5200b3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:50.282 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.281+0000 7fb526200640 1 -- 172.21.15.145:0/2751157997 shutdown_connections 2024-05-09T04:34:50.282 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.281+0000 7fb526200640 1 -- 172.21.15.145:0/2751157997 wait complete. 2024-05-09T04:34:50.282 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.282+0000 7fb526200640 1 Processor -- start 2024-05-09T04:34:50.282 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.283+0000 7fb526200640 1 -- start start 2024-05-09T04:34:50.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.283+0000 7fb526200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb52006c8a0 0x7fb5200b1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:50.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.284+0000 7fb526200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb52006fa50 0x7fb5201582a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:50.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.284+0000 7fb526200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5200add40 0x7fb5201587e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:50.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.284+0000 7fb526200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb52015c100 con 0x7fb52006fa50 2024-05-09T04:34:50.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.284+0000 7fb526200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb5200ceff0 con 0x7fb52006c8a0 2024-05-09T04:34:50.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.284+0000 7fb526200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb52015c280 con 0x7fb5200add40 2024-05-09T04:34:50.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.284+0000 7fb517e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb52006fa50 0x7fb5201582a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:50.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.284+0000 7fb517e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb52006fa50 0x7fb5201582a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:59392/0 (socket says 172.21.15.145:59392) 2024-05-09T04:34:50.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.284+0000 7fb517e00640 1 -- 172.21.15.145:0/2183381583 learned_addr learned my addr 172.21.15.145:0/2183381583 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:50.285 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.284+0000 7fb517e00640 1 -- 172.21.15.145:0/2183381583 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5200add40 msgr2=0x7fb5201587e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:50.285 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.284+0000 7fb517e00640 1 --2- 172.21.15.145:0/2183381583 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5200add40 0x7fb5201587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:50.285 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.284+0000 7fb517e00640 1 -- 172.21.15.145:0/2183381583 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb52006c8a0 msgr2=0x7fb5200b1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:50.285 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.284+0000 7fb517e00640 1 --2- 172.21.15.145:0/2183381583 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb52006c8a0 0x7fb5200b1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:50.285 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.284+0000 7fb517e00640 1 -- 172.21.15.145:0/2183381583 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb52009e100 con 0x7fb52006fa50 2024-05-09T04:34:50.285 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.284+0000 7fb517e00640 1 --2- 172.21.15.145:0/2183381583 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb52006fa50 0x7fb5201582a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb508009890 tx=0x7fb5080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:50.285 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.285+0000 7fb51d600640 1 -- 172.21.15.145:0/2183381583 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb508037040 con 0x7fb52006fa50 2024-05-09T04:34:50.286 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.285+0000 7fb51d600640 1 -- 172.21.15.145:0/2183381583 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb508042070 con 0x7fb52006fa50 2024-05-09T04:34:50.286 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.285+0000 7fb526200640 1 -- 172.21.15.145:0/2183381583 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb52009ba10 con 0x7fb52006fa50 2024-05-09T04:34:50.286 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.285+0000 7fb51d600640 1 -- 172.21.15.145:0/2183381583 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb508030900 con 0x7fb52006fa50 2024-05-09T04:34:50.286 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.285+0000 7fb526200640 1 -- 172.21.15.145:0/2183381583 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb52009bfd0 con 0x7fb52006fa50 2024-05-09T04:34:50.288 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.288+0000 7fb51d600640 1 -- 172.21.15.145:0/2183381583 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb508030aa0 con 0x7fb52006fa50 2024-05-09T04:34:50.289 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.289+0000 7fb51d600640 1 --2- 172.21.15.145:0/2183381583 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fb4fc0762d0 0x7fb4fc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:50.289 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.289+0000 7fb52a615740 1 Processor -- start 2024-05-09T04:34:50.289 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.289+0000 7fb51d600640 1 -- 172.21.15.145:0/2183381583 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fb5080b7fa0 con 0x7fb52006fa50 2024-05-09T04:34:50.289 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.289+0000 7fb52a615740 1 -- start start 2024-05-09T04:34:50.289 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.289+0000 7fb524e00640 1 --2- 172.21.15.145:0/2183381583 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fb4fc0762d0 0x7fb4fc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:50.291 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.290+0000 7fb524e00640 1 --2- 172.21.15.145:0/2183381583 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fb4fc0762d0 0x7fb4fc078790 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb510001220 tx=0x7fb5100078f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:50.291 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.290+0000 7fb52a615740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5556617fea10 0x5556617fedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:50.291 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.290+0000 7fb52a615740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556617f98a0 0x5556617fd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:50.291 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.290+0000 7fb52a615740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5556617fdb50 0x5556617fdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:50.291 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.290+0000 7fb52a615740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x555661762790 con 0x5556617fea10 2024-05-09T04:34:50.291 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.290+0000 7fb52a615740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x555661707f20 con 0x5556617f98a0 2024-05-09T04:34:50.292 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.290+0000 7fb52a615740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x55566175e750 con 0x5556617fdb50 2024-05-09T04:34:50.292 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.291+0000 7fb524e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5556617fea10 0x5556617fedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:50.292 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.291+0000 7fb524e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5556617fea10 0x5556617fedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:59396/0 (socket says 172.21.15.145:59396) 2024-05-09T04:34:50.292 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.291+0000 7fb524e00640 1 -- 172.21.15.145:0/2106636880 learned_addr learned my addr 172.21.15.145:0/2106636880 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:50.292 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.291+0000 7fb517e00640 1 --2- 172.21.15.145:0/2106636880 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556617f98a0 0x5556617fd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:50.292 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.291+0000 7fb525800640 1 --2- 172.21.15.145:0/2106636880 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5556617fdb50 0x5556617fdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:50.292 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.291+0000 7fb524e00640 1 -- 172.21.15.145:0/2106636880 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5556617fdb50 msgr2=0x5556617fdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:50.292 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.291+0000 7fb524e00640 1 --2- 172.21.15.145:0/2106636880 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5556617fdb50 0x5556617fdf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:50.292 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.291+0000 7fb524e00640 1 -- 172.21.15.145:0/2106636880 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556617f98a0 msgr2=0x5556617fd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:50.292 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.291+0000 7fb524e00640 1 --2- 172.21.15.145:0/2106636880 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556617f98a0 0x5556617fd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:50.292 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.291+0000 7fb524e00640 1 -- 172.21.15.145:0/2106636880 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556614742d0 con 0x5556617fea10 2024-05-09T04:34:50.292 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.291+0000 7fb517e00640 1 --2- 172.21.15.145:0/2106636880 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556617f98a0 0x5556617fd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:34:50.292 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.291+0000 7fb524e00640 1 --2- 172.21.15.145:0/2106636880 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5556617fea10 0x5556617fedf0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb510005cd0 tx=0x7fb51000f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=286cb796515f801f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:50.292 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.292+0000 7fb515600640 1 -- 172.21.15.145:0/2106636880 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb510016110 con 0x5556617fea10 2024-05-09T04:34:50.292 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.292+0000 7fb515600640 1 -- 172.21.15.145:0/2106636880 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb510007560 con 0x5556617fea10 2024-05-09T04:34:50.292 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.292+0000 7fb515600640 1 -- 172.21.15.145:0/2106636880 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5100162b0 con 0x5556617fea10 2024-05-09T04:34:50.296 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.292+0000 7fb52a615740 1 -- 172.21.15.145:0/2106636880 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5556617fea10 msgr2=0x5556617fedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:50.296 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.292+0000 7fb52a615740 1 --2- 172.21.15.145:0/2106636880 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5556617fea10 0x5556617fedf0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb510005cd0 tx=0x7fb51000f000 comp rx=0 tx=0).stop 2024-05-09T04:34:50.296 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.292+0000 7fb52a615740 1 -- 172.21.15.145:0/2106636880 shutdown_connections 2024-05-09T04:34:50.296 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.292+0000 7fb52a615740 1 --2- 172.21.15.145:0/2106636880 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5556617fdb50 0x5556617fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:50.296 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.292+0000 7fb52a615740 1 --2- 172.21.15.145:0/2106636880 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556617f98a0 0x5556617fd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:50.297 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.292+0000 7fb52a615740 1 --2- 172.21.15.145:0/2106636880 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5556617fea10 0x5556617fedf0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:50.297 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.292+0000 7fb52a615740 1 -- 172.21.15.145:0/2106636880 >> 172.21.15.145:0/2106636880 conn(0x5556617fbb10 msgr2=0x5556618445d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:50.297 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.292+0000 7fb52a615740 1 -- 172.21.15.145:0/2106636880 shutdown_connections 2024-05-09T04:34:50.297 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.292+0000 7fb52a615740 1 -- 172.21.15.145:0/2106636880 wait complete. 2024-05-09T04:34:50.297 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.293+0000 7fb52a615740 1 -- start start 2024-05-09T04:34:50.297 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.293+0000 7fb52a615740 1 Processor -- start 2024-05-09T04:34:50.297 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.294+0000 7fb52a615740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556617fdb50 0x55566181ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:50.297 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.294+0000 7fb52a615740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5556617fea10 0x55566184a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:50.297 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.294+0000 7fb52a615740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55566181f360 0x55566184ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:50.297 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.294+0000 7fb52a615740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55566182a5a0 con 0x55566181f360 2024-05-09T04:34:50.297 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.294+0000 7fb52a615740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55566171b9c0 con 0x5556617fdb50 2024-05-09T04:34:50.297 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.294+0000 7fb52a615740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x555661829a20 con 0x5556617fea10 2024-05-09T04:34:50.297 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.294+0000 7fb52a615740 10 client.0 _ll_register_callbacks cb 0x5556616e8120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:34:50.297 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.294+0000 7fb525800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55566181f360 0x55566184ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:50.297 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.295+0000 7fb525800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55566181f360 0x55566184ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:59402/0 (socket says 172.21.15.145:59402) 2024-05-09T04:34:50.297 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.295+0000 7fb525800640 1 -- 172.21.15.145:0/4003463232 learned_addr learned my addr 172.21.15.145:0/4003463232 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:50.298 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.295+0000 7fb524e00640 1 --2- 172.21.15.145:0/4003463232 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556617fdb50 0x55566181ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:50.298 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.295+0000 7fb517e00640 1 --2- 172.21.15.145:0/4003463232 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5556617fea10 0x55566184a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:50.298 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.295+0000 7fb525800640 1 -- 172.21.15.145:0/4003463232 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5556617fea10 msgr2=0x55566184a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:50.298 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.295+0000 7fb525800640 1 --2- 172.21.15.145:0/4003463232 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5556617fea10 0x55566184a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:50.298 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.295+0000 7fb525800640 1 -- 172.21.15.145:0/4003463232 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556617fdb50 msgr2=0x55566181ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:50.298 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.295+0000 7fb525800640 1 --2- 172.21.15.145:0/4003463232 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556617fdb50 0x55566181ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:50.298 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.295+0000 7fb525800640 1 -- 172.21.15.145:0/4003463232 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55566153e6b0 con 0x55566181f360 2024-05-09T04:34:50.298 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.295+0000 7fb525800640 1 --2- 172.21.15.145:0/4003463232 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55566181f360 0x55566184ac70 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb5180079c0 tx=0x7fb518007e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e031eb3059cb43fa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:50.298 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.295+0000 7fb50e000640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-05-09T04:34:50.298 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.295+0000 7fb50e000640 1 -- 172.21.15.145:0/4003463232 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb518018020 con 0x55566181f360 2024-05-09T04:34:50.298 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.296+0000 7fb52a615740 1 -- 172.21.15.145:0/4003463232 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb510007560 con 0x55566181f360 2024-05-09T04:34:50.298 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.296+0000 7fb50e000640 1 -- 172.21.15.145:0/4003463232 <== mon.0 v2:172.21.15.145:3300/0 2 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fb51800fc00 con 0x55566181f360 2024-05-09T04:34:50.298 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.296+0000 7fb50e000640 1 -- 172.21.15.145:0/4003463232 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fb518022050 con 0x55566181f360 2024-05-09T04:34:50.298 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.296+0000 7fb52a615740 10 client.4926 fetch_fsmap learned FSMap version 17 2024-05-09T04:34:50.298 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.296+0000 7fb52a615740 1 -- 172.21.15.145:0/4003463232 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x5556618465e0 con 0x55566181f360 2024-05-09T04:34:50.299 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.296+0000 7fb50e000640 1 -- 172.21.15.145:0/4003463232 <== mon.0 v2:172.21.15.145:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb510007560 con 0x55566181f360 2024-05-09T04:34:50.299 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.296+0000 7fb50e000640 1 -- 172.21.15.145:0/4003463232 <== mon.0 v2:172.21.15.145:3300/0 5 ==== fsmap(e 17) ==== 3494+0+0 (secure 0 0 0) 0x7fb518025020 con 0x55566181f360 2024-05-09T04:34:50.299 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.296+0000 7fb52a615740 10 client.4926 fetch_fsmap finished waiting for FSMap version 17 2024-05-09T04:34:50.299 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.296+0000 7fb52a615740 10 client.4926 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-09T04:34:50.299 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.296+0000 7fb52a615740 20 client.4926 populate_metadata read hostname 'smithi145' 2024-05-09T04:34:50.299 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.296+0000 7fb52a615740 1 --2- 172.21.15.145:0/4003463232 >> [v2:172.21.15.145:6834/4260798843,v1:172.21.15.145:6835/4260798843] conn(0x555661860aa0 0x555661862e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:50.299 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.296+0000 7fb52a615740 4 client.4926 mds_command: new command op to 4761 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:34:50.299 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.296+0000 7fb52a615740 1 -- 172.21.15.145:0/4003463232 --> [v2:172.21.15.145:6834/4260798843,v1:172.21.15.145:6835/4260798843] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556617703a0 con 0x555661860aa0 2024-05-09T04:34:50.299 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.297+0000 7fb524e00640 1 --2- 172.21.15.145:0/4003463232 >> [v2:172.21.15.145:6834/4260798843,v1:172.21.15.145:6835/4260798843] conn(0x555661860aa0 0x555661862e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:50.300 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.297+0000 7fb524e00640 1 --2- 172.21.15.145:0/4003463232 >> [v2:172.21.15.145:6834/4260798843,v1:172.21.15.145:6835/4260798843] conn(0x555661860aa0 0x555661862e90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8afd9eb0f586cc47 server_cookie=bf11f347eab0fbfe in_seq=0 out_seq=0 2024-05-09T04:34:50.300 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.297+0000 7fb50e000640 10 client.4926 ms_handle_connect on v2:172.21.15.145:6834/4260798843 2024-05-09T04:34:50.300 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.298+0000 7fb50e000640 1 -- 172.21.15.145:0/4003463232 <== mds.0 v2:172.21.15.145:6834/4260798843 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5556617703a0 con 0x555661860aa0 2024-05-09T04:34:50.300 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.298+0000 7fb50e000640 10 client.4926 handle_command_reply: tid=0 2024-05-09T04:34:50.300 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.298+0000 7fb52a615740 1 client.4926 shutdown 2024-05-09T04:34:50.301 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.298+0000 7fb52a615740 1 -- 172.21.15.145:0/4003463232 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55566181f360 msgr2=0x55566184ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:50.301 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.298+0000 7fb52a615740 1 --2- 172.21.15.145:0/4003463232 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55566181f360 0x55566184ac70 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb5180079c0 tx=0x7fb518007e80 comp rx=0 tx=0).stop 2024-05-09T04:34:50.301 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.298+0000 7fb52a615740 1 -- 172.21.15.145:0/4003463232 shutdown_connections 2024-05-09T04:34:50.301 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.298+0000 7fb52a615740 1 --2- 172.21.15.145:0/4003463232 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55566181f360 0x55566184ac70 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:50.301 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.298+0000 7fb52a615740 1 --2- 172.21.15.145:0/4003463232 >> [v2:172.21.15.145:6834/4260798843,v1:172.21.15.145:6835/4260798843] conn(0x555661860aa0 0x555661862e90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:50.301 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.298+0000 7fb52a615740 1 --2- 172.21.15.145:0/4003463232 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5556617fea10 0x55566184a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:50.301 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.298+0000 7fb52a615740 1 --2- 172.21.15.145:0/4003463232 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556617fdb50 0x55566181ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:50.301 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.298+0000 7fb52a615740 1 -- 172.21.15.145:0/4003463232 >> 172.21.15.145:0/4003463232 conn(0x5556617f98a0 msgr2=0x5556617f8e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:50.301 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.298+0000 7fb50e000640 0 client.4926 ms_handle_reset on v2:172.21.15.145:6834/4260798843 2024-05-09T04:34:50.301 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.298+0000 7fb52a615740 1 -- 172.21.15.145:0/4003463232 shutdown_connections 2024-05-09T04:34:50.301 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.298+0000 7fb52a615740 1 -- 172.21.15.145:0/4003463232 wait complete. 2024-05-09T04:34:50.301 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.299+0000 7fb52a615740 20 client.4926 trim_cache size 0 max 16384 2024-05-09T04:34:50.318 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.319+0000 7fb52a615740 1 Processor -- start 2024-05-09T04:34:50.318 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.319+0000 7fb52a615740 1 -- start start 2024-05-09T04:34:50.319 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.319+0000 7fb52a615740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5556617fdb50 0x5556618b8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:50.319 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.319+0000 7fb52a615740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556617fea10 0x55566185e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:50.319 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.319+0000 7fb52a615740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55566181f360 0x55566185e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:50.320 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.319+0000 7fb52a615740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb520200470 con 0x5556617fdb50 2024-05-09T04:34:50.320 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.319+0000 7fb52a615740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb52015cf50 con 0x5556617fea10 2024-05-09T04:34:50.320 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.319+0000 7fb52a615740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb52015d5a0 con 0x55566181f360 2024-05-09T04:34:50.320 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.320+0000 7fb524e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5556617fdb50 0x5556618b8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:50.320 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.320+0000 7fb524e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5556617fdb50 0x5556618b8280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:59418/0 (socket says 172.21.15.145:59418) 2024-05-09T04:34:50.320 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.320+0000 7fb524e00640 1 -- 172.21.15.145:0/2098626740 learned_addr learned my addr 172.21.15.145:0/2098626740 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:50.322 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.320+0000 7fb525800640 1 --2- 172.21.15.145:0/2098626740 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55566181f360 0x55566185e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:50.322 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.320+0000 7fb517e00640 1 --2- 172.21.15.145:0/2098626740 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556617fea10 0x55566185e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:50.322 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.320+0000 7fb524e00640 1 -- 172.21.15.145:0/2098626740 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55566181f360 msgr2=0x55566185e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:50.322 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.320+0000 7fb524e00640 1 --2- 172.21.15.145:0/2098626740 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55566181f360 0x55566185e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:50.322 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.320+0000 7fb524e00640 1 -- 172.21.15.145:0/2098626740 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556617fea10 msgr2=0x55566185e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:50.322 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.320+0000 7fb524e00640 1 --2- 172.21.15.145:0/2098626740 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556617fea10 0x55566185e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:50.322 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.320+0000 7fb524e00640 1 -- 172.21.15.145:0/2098626740 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556617fd650 con 0x5556617fdb50 2024-05-09T04:34:50.322 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.320+0000 7fb517e00640 1 --2- 172.21.15.145:0/2098626740 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556617fea10 0x55566185e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:34:50.322 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.320+0000 7fb524e00640 1 --2- 172.21.15.145:0/2098626740 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5556617fdb50 0x5556618b8280 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb510001eb0 tx=0x7fb510007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b096ace02619d1d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:50.322 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.321+0000 7fb514c00640 1 -- 172.21.15.145:0/2098626740 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5100160a0 con 0x5556617fdb50 2024-05-09T04:34:50.322 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.321+0000 7fb514c00640 1 -- 172.21.15.145:0/2098626740 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb510016240 con 0x5556617fdb50 2024-05-09T04:34:50.322 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.321+0000 7fb514c00640 1 -- 172.21.15.145:0/2098626740 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb510016b70 con 0x5556617fdb50 2024-05-09T04:34:50.322 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.321+0000 7fb52a615740 1 -- 172.21.15.145:0/2098626740 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5556617fdb50 msgr2=0x5556618b8280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:50.322 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.321+0000 7fb52a615740 1 --2- 172.21.15.145:0/2098626740 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5556617fdb50 0x5556618b8280 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb510001eb0 tx=0x7fb510007000 comp rx=0 tx=0).stop 2024-05-09T04:34:50.323 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.321+0000 7fb52a615740 1 -- 172.21.15.145:0/2098626740 shutdown_connections 2024-05-09T04:34:50.323 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.321+0000 7fb52a615740 1 --2- 172.21.15.145:0/2098626740 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55566181f360 0x55566185e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:50.323 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.321+0000 7fb52a615740 1 --2- 172.21.15.145:0/2098626740 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5556617fea10 0x55566185e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:50.323 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.321+0000 7fb52a615740 1 --2- 172.21.15.145:0/2098626740 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5556617fdb50 0x5556618b8280 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:50.323 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.321+0000 7fb52a615740 1 -- 172.21.15.145:0/2098626740 >> 172.21.15.145:0/2098626740 conn(0x5556617f98a0 msgr2=0x5556618002f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:50.323 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.321+0000 7fb52a615740 1 -- 172.21.15.145:0/2098626740 shutdown_connections 2024-05-09T04:34:50.323 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.322+0000 7fb52a615740 1 -- 172.21.15.145:0/2098626740 wait complete. 2024-05-09T04:34:50.323 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.322+0000 7fb52a615740 1 -- start start 2024-05-09T04:34:50.327 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.322+0000 7fb52a615740 1 Processor -- start 2024-05-09T04:34:50.327 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.323+0000 7fb52a615740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5556617fdb50 0x5556618c6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:50.327 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.323+0000 7fb52a615740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5556617fea10 0x555661841c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:50.327 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.323+0000 7fb52a615740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55566181f360 0x5556618215c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:50.327 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.323+0000 7fb52a615740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x555661849fb0 con 0x5556617fea10 2024-05-09T04:34:50.327 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.323+0000 7fb52a615740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55566184ccd0 con 0x55566181f360 2024-05-09T04:34:50.327 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.323+0000 7fb52a615740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x555661845ed0 con 0x5556617fdb50 2024-05-09T04:34:50.327 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.323+0000 7fb52a615740 10 client.0 _ll_register_callbacks cb 0x7fb5180097f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:34:50.327 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.324+0000 7fb517e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5556617fea10 0x555661841c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:50.327 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.324+0000 7fb517e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5556617fea10 0x555661841c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:59424/0 (socket says 172.21.15.145:59424) 2024-05-09T04:34:50.327 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.324+0000 7fb517e00640 1 -- 172.21.15.145:0/1049601977 learned_addr learned my addr 172.21.15.145:0/1049601977 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:50.327 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.324+0000 7fb524e00640 1 --2- 172.21.15.145:0/1049601977 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5556617fdb50 0x5556618c6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:50.327 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.324+0000 7fb517e00640 1 -- 172.21.15.145:0/1049601977 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5556617fdb50 msgr2=0x5556618c6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:50.328 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.324+0000 7fb517e00640 1 --2- 172.21.15.145:0/1049601977 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5556617fdb50 0x5556618c6f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:50.328 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.324+0000 7fb525800640 1 --2- 172.21.15.145:0/1049601977 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55566181f360 0x5556618215c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:50.328 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.324+0000 7fb517e00640 1 -- 172.21.15.145:0/1049601977 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55566181f360 msgr2=0x5556618215c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:50.328 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.324+0000 7fb517e00640 1 --2- 172.21.15.145:0/1049601977 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55566181f360 0x5556618215c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:50.328 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.324+0000 7fb517e00640 1 -- 172.21.15.145:0/1049601977 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555661820fd0 con 0x5556617fea10 2024-05-09T04:34:50.328 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.324+0000 7fb517e00640 1 --2- 172.21.15.145:0/1049601977 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5556617fea10 0x555661841c00 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb508006ce0 tx=0x7fb508080a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5678da095683428b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:50.328 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.324+0000 7fb50e000640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-05-09T04:34:50.328 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.324+0000 7fb50e000640 1 -- 172.21.15.145:0/1049601977 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb50809a020 con 0x5556617fea10 2024-05-09T04:34:50.328 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.324+0000 7fb50e000640 1 -- 172.21.15.145:0/1049601977 <== mon.0 v2:172.21.15.145:3300/0 2 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fb508089d30 con 0x5556617fea10 2024-05-09T04:34:50.328 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.324+0000 7fb52a615740 1 -- 172.21.15.145:0/1049601977 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb510016240 con 0x5556617fea10 2024-05-09T04:34:50.328 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.325+0000 7fb50e000640 1 -- 172.21.15.145:0/1049601977 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fb508085360 con 0x5556617fea10 2024-05-09T04:34:50.328 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.325+0000 7fb52a615740 10 client.4932 fetch_fsmap learned FSMap version 17 2024-05-09T04:34:50.328 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.325+0000 7fb52a615740 1 -- 172.21.15.145:0/1049601977 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x5556618426c0 con 0x5556617fea10 2024-05-09T04:34:50.328 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.325+0000 7fb50e000640 1 -- 172.21.15.145:0/1049601977 <== mon.0 v2:172.21.15.145:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb510016240 con 0x5556617fea10 2024-05-09T04:34:50.328 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.325+0000 7fb50e000640 1 -- 172.21.15.145:0/1049601977 <== mon.0 v2:172.21.15.145:3300/0 5 ==== fsmap(e 17) ==== 3494+0+0 (secure 0 0 0) 0x7fb50803a070 con 0x5556617fea10 2024-05-09T04:34:50.328 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.325+0000 7fb52a615740 10 client.4932 fetch_fsmap finished waiting for FSMap version 17 2024-05-09T04:34:50.328 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.325+0000 7fb52a615740 10 client.4932 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-09T04:34:50.328 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.325+0000 7fb52a615740 20 client.4932 populate_metadata read hostname 'smithi145' 2024-05-09T04:34:50.329 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.325+0000 7fb52a615740 1 --2- 172.21.15.145:0/1049601977 >> [v2:172.21.15.145:6834/4260798843,v1:172.21.15.145:6835/4260798843] conn(0x555661821b00 0x55566181d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:50.329 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.325+0000 7fb52a615740 4 client.4932 mds_command: new command op to 4761 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T04:34:50.329 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.325+0000 7fb52a615740 1 -- 172.21.15.145:0/1049601977 --> [v2:172.21.15.145:6834/4260798843,v1:172.21.15.145:6835/4260798843] -- command(tid 0: {"prefix": "status"}) -- 0x55566176a210 con 0x555661821b00 2024-05-09T04:34:50.329 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.325+0000 7fb524e00640 1 --2- 172.21.15.145:0/1049601977 >> [v2:172.21.15.145:6834/4260798843,v1:172.21.15.145:6835/4260798843] conn(0x555661821b00 0x55566181d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:50.329 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.326+0000 7fb524e00640 1 --2- 172.21.15.145:0/1049601977 >> [v2:172.21.15.145:6834/4260798843,v1:172.21.15.145:6835/4260798843] conn(0x555661821b00 0x55566181d4d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8222073287873d20 server_cookie=c161ef8d83e7fe41 in_seq=0 out_seq=0 2024-05-09T04:34:50.329 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.326+0000 7fb50e000640 10 client.4932 ms_handle_connect on v2:172.21.15.145:6834/4260798843 2024-05-09T04:34:50.329 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.326+0000 7fb50e000640 1 -- 172.21.15.145:0/1049601977 <== mds.0 v2:172.21.15.145:6834/4260798843 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55566176a210 con 0x555661821b00 2024-05-09T04:34:50.329 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.326+0000 7fb50e000640 10 client.4932 handle_command_reply: tid=0 2024-05-09T04:34:50.329 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.326+0000 7fb52a615740 1 client.4932 shutdown 2024-05-09T04:34:50.329 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.326+0000 7fb52a615740 1 -- 172.21.15.145:0/1049601977 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5556617fea10 msgr2=0x555661841c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:50.329 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.326+0000 7fb52a615740 1 --2- 172.21.15.145:0/1049601977 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5556617fea10 0x555661841c00 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb508006ce0 tx=0x7fb508080a20 comp rx=0 tx=0).stop 2024-05-09T04:34:50.329 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.326+0000 7fb52a615740 1 -- 172.21.15.145:0/1049601977 shutdown_connections 2024-05-09T04:34:50.329 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.326+0000 7fb52a615740 1 --2- 172.21.15.145:0/1049601977 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55566181f360 0x5556618215c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:50.329 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.326+0000 7fb52a615740 1 --2- 172.21.15.145:0/1049601977 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5556617fea10 0x555661841c00 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:50.329 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.326+0000 7fb52a615740 1 --2- 172.21.15.145:0/1049601977 >> [v2:172.21.15.145:6834/4260798843,v1:172.21.15.145:6835/4260798843] conn(0x555661821b00 0x55566181d4d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:50.329 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.326+0000 7fb52a615740 1 --2- 172.21.15.145:0/1049601977 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5556617fdb50 0x5556618c6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:50.330 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.326+0000 7fb52a615740 1 -- 172.21.15.145:0/1049601977 >> 172.21.15.145:0/1049601977 conn(0x5556617f98a0 msgr2=0x55566185e350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:50.330 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.326+0000 7fb50e000640 0 client.4932 ms_handle_reset on v2:172.21.15.145:6834/4260798843 2024-05-09T04:34:50.330 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.327+0000 7fb52a615740 1 -- 172.21.15.145:0/1049601977 shutdown_connections 2024-05-09T04:34:50.330 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.327+0000 7fb52a615740 1 -- 172.21.15.145:0/1049601977 wait complete. 2024-05-09T04:34:50.330 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.327+0000 7fb52a615740 20 client.4932 trim_cache size 0 max 16384 2024-05-09T04:34:50.330 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-05-09T04:34:50.330 INFO:teuthology.orchestra.run.smithi145.stdout: "cluster_fsid": "957ee6c1-073b-44cf-9b93-9491b3fdfc1d", 2024-05-09T04:34:50.330 INFO:teuthology.orchestra.run.smithi145.stdout: "whoami": 0, 2024-05-09T04:34:50.330 INFO:teuthology.orchestra.run.smithi145.stdout: "id": 4761, 2024-05-09T04:34:50.330 INFO:teuthology.orchestra.run.smithi145.stdout: "want_state": "up:active", 2024-05-09T04:34:50.330 INFO:teuthology.orchestra.run.smithi145.stdout: "state": "up:active", 2024-05-09T04:34:50.331 INFO:teuthology.orchestra.run.smithi145.stdout: "fs_name": "cephfs", 2024-05-09T04:34:50.331 INFO:teuthology.orchestra.run.smithi145.stdout: "rank_uptime": 6.6540664510000003, 2024-05-09T04:34:50.331 INFO:teuthology.orchestra.run.smithi145.stdout: "mdsmap_epoch": 17, 2024-05-09T04:34:50.331 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_epoch": 32, 2024-05-09T04:34:50.331 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_epoch_barrier": 32, 2024-05-09T04:34:50.331 INFO:teuthology.orchestra.run.smithi145.stdout: "uptime": 14.636188658 2024-05-09T04:34:50.331 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-05-09T04:34:50.331 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.327+0000 7fb526200640 1 -- 172.21.15.145:0/2183381583 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fb4fc0762d0 msgr2=0x7fb4fc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:50.331 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.327+0000 7fb526200640 1 --2- 172.21.15.145:0/2183381583 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fb4fc0762d0 0x7fb4fc078790 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb510001220 tx=0x7fb5100078f0 comp rx=0 tx=0).stop 2024-05-09T04:34:50.332 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.327+0000 7fb526200640 1 -- 172.21.15.145:0/2183381583 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb52006fa50 msgr2=0x7fb5201582a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:50.332 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.327+0000 7fb526200640 1 --2- 172.21.15.145:0/2183381583 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb52006fa50 0x7fb5201582a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb508009890 tx=0x7fb5080046c0 comp rx=0 tx=0).stop 2024-05-09T04:34:50.332 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.327+0000 7fb526200640 1 -- 172.21.15.145:0/2183381583 shutdown_connections 2024-05-09T04:34:50.332 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.327+0000 7fb526200640 1 --2- 172.21.15.145:0/2183381583 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fb4fc0762d0 0x7fb4fc078790 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:50.332 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.327+0000 7fb526200640 1 --2- 172.21.15.145:0/2183381583 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5200add40 0x7fb5201587e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:50.332 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.327+0000 7fb526200640 1 --2- 172.21.15.145:0/2183381583 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb52006fa50 0x7fb5201582a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:50.332 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.327+0000 7fb526200640 1 --2- 172.21.15.145:0/2183381583 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb52006c8a0 0x7fb5200b1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:50.332 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.327+0000 7fb526200640 1 -- 172.21.15.145:0/2183381583 >> 172.21.15.145:0/2183381583 conn(0x7fb5200b2d80 msgr2=0x7fb5200729b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:50.332 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.328+0000 7fb526200640 1 -- 172.21.15.145:0/2183381583 shutdown_connections 2024-05-09T04:34:50.332 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.328+0000 7fb526200640 1 -- 172.21.15.145:0/2183381583 wait complete. 2024-05-09T04:34:50.344 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-09T04:34:50.528 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.528+0000 7ff36f200640 1 Processor -- start 2024-05-09T04:34:50.528 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.528+0000 7ff36f200640 1 -- start start 2024-05-09T04:34:50.530 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.530+0000 7ff36f200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff3701572c0 0x7ff37015deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:50.530 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.530+0000 7ff36f200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff3700ae710 0x7ff370161220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:50.530 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.530+0000 7ff36f200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3700aecd0 con 0x7ff3700cff30 2024-05-09T04:34:50.531 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.530+0000 7ff36f200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7ff3700b6290 con 0x7ff3700ae710 2024-05-09T04:34:50.531 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.530+0000 7ff36f200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff3700aefb0 con 0x7ff3701572c0 2024-05-09T04:34:50.531 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.530+0000 7ff367e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff3700ae710 0x7ff370161220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:50.531 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.530+0000 7ff36e800640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7ff3700cff30 0x7ff3700b4d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:49066/0 (socket says 172.21.15.145:49066) 2024-05-09T04:34:50.531 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.530+0000 7ff36e800640 1 -- 172.21.15.145:0/2344006065 learned_addr learned my addr 172.21.15.145:0/2344006065 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:34:50.531 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.531+0000 7ff367e00640 1 -- 172.21.15.145:0/2344006065 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff3701572c0 msgr2=0x7ff37015deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:50.531 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.531+0000 7ff367e00640 1 --2- 172.21.15.145:0/2344006065 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff3701572c0 0x7ff37015deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:50.531 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.531+0000 7ff367e00640 1 -- 172.21.15.145:0/2344006065 >> v1:172.21.15.145:6789/0 conn(0x7ff3700cff30 legacy=0x7ff3700b4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:50.531 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.531+0000 7ff367e00640 1 -- 172.21.15.145:0/2344006065 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3700af290 con 0x7ff3700ae710 2024-05-09T04:34:50.532 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.532+0000 7ff367e00640 1 --2- 172.21.15.145:0/2344006065 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3700ae710 0x7ff370161220 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff3580097a0 tx=0x7ff35802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=443b586a1767193a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:50.532 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.532+0000 7ff36d400640 1 -- 172.21.15.145:0/2344006065 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff358037070 con 0x7ff3700ae710 2024-05-09T04:34:50.533 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.532+0000 7ff36d400640 1 -- 172.21.15.145:0/2344006065 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff358032440 con 0x7ff3700ae710 2024-05-09T04:34:50.533 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.532+0000 7ff36d400640 1 -- 172.21.15.145:0/2344006065 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff358032810 con 0x7ff3700ae710 2024-05-09T04:34:50.533 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.532+0000 7ff36f200640 1 -- 172.21.15.145:0/2344006065 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3700ae710 msgr2=0x7ff370161220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:50.533 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.532+0000 7ff36f200640 1 --2- 172.21.15.145:0/2344006065 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3700ae710 0x7ff370161220 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff3580097a0 tx=0x7ff35802a180 comp rx=0 tx=0).stop 2024-05-09T04:34:50.533 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.533+0000 7ff36f200640 1 -- 172.21.15.145:0/2344006065 shutdown_connections 2024-05-09T04:34:50.533 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.533+0000 7ff36f200640 1 --2- 172.21.15.145:0/2344006065 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3700ae710 0x7ff370161220 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:50.533 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.533+0000 7ff36f200640 1 --2- 172.21.15.145:0/2344006065 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff3701572c0 0x7ff37015deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:50.533 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.533+0000 7ff36f200640 1 -- 172.21.15.145:0/2344006065 >> 172.21.15.145:0/2344006065 conn(0x7ff37006c8a0 msgr2=0x7ff37006ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:50.533 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.533+0000 7ff36f200640 1 -- 172.21.15.145:0/2344006065 shutdown_connections 2024-05-09T04:34:50.533 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.533+0000 7ff36f200640 1 -- 172.21.15.145:0/2344006065 wait complete. 2024-05-09T04:34:50.534 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.534+0000 7ff36f200640 1 Processor -- start 2024-05-09T04:34:50.534 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.534+0000 7ff36f200640 1 -- start start 2024-05-09T04:34:50.536 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.535+0000 7ff36f200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3700ae710 0x7ff37009e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:50.536 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.535+0000 7ff36f200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff3700cff30 0x7ff37009ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:50.536 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.535+0000 7ff36f200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3701572c0 0x7ff3701456f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:50.536 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.535+0000 7ff36f200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff370161760 con 0x7ff3701572c0 2024-05-09T04:34:50.536 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.535+0000 7ff36f200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff370160d00 con 0x7ff3700ae710 2024-05-09T04:34:50.536 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.535+0000 7ff36f200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff3701618e0 con 0x7ff3700cff30 2024-05-09T04:34:50.536 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.536+0000 7ff36e800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3701572c0 0x7ff3701456f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:50.536 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.536+0000 7ff36e800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3701572c0 0x7ff3701456f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:59432/0 (socket says 172.21.15.145:59432) 2024-05-09T04:34:50.536 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.536+0000 7ff36e800640 1 -- 172.21.15.145:0/940179230 learned_addr learned my addr 172.21.15.145:0/940179230 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:50.536 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.536+0000 7ff36de00640 1 --2- 172.21.15.145:0/940179230 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3700ae710 0x7ff37009e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:50.536 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.536+0000 7ff367e00640 1 --2- 172.21.15.145:0/940179230 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff3700cff30 0x7ff37009ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:50.536 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.536+0000 7ff36e800640 1 -- 172.21.15.145:0/940179230 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff3700cff30 msgr2=0x7ff37009ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:50.537 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.536+0000 7ff36e800640 1 --2- 172.21.15.145:0/940179230 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff3700cff30 0x7ff37009ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:50.537 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.536+0000 7ff36e800640 1 -- 172.21.15.145:0/940179230 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3700ae710 msgr2=0x7ff37009e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:50.537 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.536+0000 7ff36e800640 1 --2- 172.21.15.145:0/940179230 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3700ae710 0x7ff37009e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:50.537 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.536+0000 7ff36e800640 1 -- 172.21.15.145:0/940179230 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff37009cfa0 con 0x7ff3701572c0 2024-05-09T04:34:50.538 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.537+0000 7ff36e800640 1 --2- 172.21.15.145:0/940179230 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3701572c0 0x7ff3701456f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff36800cb50 tx=0x7ff36800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:50.538 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.537+0000 7ff366000640 1 -- 172.21.15.145:0/940179230 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff368017070 con 0x7ff3701572c0 2024-05-09T04:34:50.538 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.537+0000 7ff366000640 1 -- 172.21.15.145:0/940179230 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff36800ace0 con 0x7ff3701572c0 2024-05-09T04:34:50.538 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.537+0000 7ff366000640 1 -- 172.21.15.145:0/940179230 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff368012600 con 0x7ff3701572c0 2024-05-09T04:34:50.538 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.537+0000 7ff36f200640 1 -- 172.21.15.145:0/940179230 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff37009d270 con 0x7ff3701572c0 2024-05-09T04:34:50.538 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.537+0000 7ff36f200640 1 -- 172.21.15.145:0/940179230 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff37009d830 con 0x7ff3701572c0 2024-05-09T04:34:50.538 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.538+0000 7ff36f200640 1 -- 172.21.15.145:0/940179230 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff37009da70 con 0x7ff3701572c0 2024-05-09T04:34:50.539 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.539+0000 7ff366000640 1 -- 172.21.15.145:0/940179230 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff36801a070 con 0x7ff3701572c0 2024-05-09T04:34:50.541 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.541+0000 7ff366000640 1 --2- 172.21.15.145:0/940179230 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff34c076300 0x7ff34c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:50.541 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.541+0000 7ff366000640 1 -- 172.21.15.145:0/940179230 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7ff368097260 con 0x7ff3701572c0 2024-05-09T04:34:50.541 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.541+0000 7ff36de00640 1 --2- 172.21.15.145:0/940179230 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff34c076300 0x7ff34c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:50.542 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.542+0000 7ff36de00640 1 --2- 172.21.15.145:0/940179230 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff34c076300 0x7ff34c0787c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff36000b810 tx=0x7ff360008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:50.545 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.546+0000 7ff366000640 1 -- 172.21.15.145:0/940179230 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff37009da70 con 0x7ff3701572c0 2024-05-09T04:34:50.797 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.797+0000 7ff36f200640 1 -- 172.21.15.145:0/940179230 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff37006a310 con 0x7ff3701572c0 2024-05-09T04:34:50.798 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.798+0000 7ff366000640 1 -- 172.21.15.145:0/940179230 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7ff3680604d0 con 0x7ff3701572c0 2024-05-09T04:34:50.798 INFO:teuthology.orchestra.run.smithi145.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-09T04:34:50.801 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.802+0000 7ff36f200640 1 -- 172.21.15.145:0/940179230 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff34c076300 msgr2=0x7ff34c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:50.801 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.802+0000 7ff36f200640 1 --2- 172.21.15.145:0/940179230 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff34c076300 0x7ff34c0787c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff36000b810 tx=0x7ff360008040 comp rx=0 tx=0).stop 2024-05-09T04:34:50.801 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.802+0000 7ff36f200640 1 -- 172.21.15.145:0/940179230 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3701572c0 msgr2=0x7ff3701456f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:50.802 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.802+0000 7ff36f200640 1 --2- 172.21.15.145:0/940179230 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3701572c0 0x7ff3701456f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff36800cb50 tx=0x7ff36800a430 comp rx=0 tx=0).stop 2024-05-09T04:34:50.802 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.802+0000 7ff36f200640 1 -- 172.21.15.145:0/940179230 shutdown_connections 2024-05-09T04:34:50.802 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.802+0000 7ff36f200640 1 --2- 172.21.15.145:0/940179230 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff34c076300 0x7ff34c0787c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:50.802 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.802+0000 7ff36f200640 1 --2- 172.21.15.145:0/940179230 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3701572c0 0x7ff3701456f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:50.802 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.802+0000 7ff36f200640 1 --2- 172.21.15.145:0/940179230 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff3700cff30 0x7ff37009ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:50.802 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.802+0000 7ff36f200640 1 --2- 172.21.15.145:0/940179230 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3700ae710 0x7ff37009e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:50.802 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.802+0000 7ff36f200640 1 -- 172.21.15.145:0/940179230 >> 172.21.15.145:0/940179230 conn(0x7ff37006c8a0 msgr2=0x7ff37015e480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:50.802 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.802+0000 7ff36f200640 1 -- 172.21.15.145:0/940179230 shutdown_connections 2024-05-09T04:34:50.802 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:50.802+0000 7ff36f200640 1 -- 172.21.15.145:0/940179230 wait complete. 2024-05-09T04:34:50.818 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-09T04:34:50.818 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-09T04:34:50.818 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi145.front.sepia.ceph.com 2024-05-09T04:34:50.818 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-09T04:34:50.818 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-09T04:34:50.819 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-09T04:34:50.819 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-09T04:34:50.819 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-09T04:34:50.819 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-09T04:34:50.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:50.819 DEBUG:teuthology.orchestra.run.smithi145:> ip addr 2024-05-09T04:34:50.867 INFO:teuthology.orchestra.run.smithi145.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T04:34:50.867 INFO:teuthology.orchestra.run.smithi145.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T04:34:50.867 INFO:teuthology.orchestra.run.smithi145.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T04:34:50.867 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-05-09T04:34:50.867 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 ::1/128 scope host 2024-05-09T04:34:50.867 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-05-09T04:34:50.867 INFO:teuthology.orchestra.run.smithi145.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T04:34:50.867 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:d9:38:66 brd ff:ff:ff:ff:ff:ff 2024-05-09T04:34:50.867 INFO:teuthology.orchestra.run.smithi145.stdout: altname enp7s0f0 2024-05-09T04:34:50.868 INFO:teuthology.orchestra.run.smithi145.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T04:34:50.868 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:d9:38:67 brd ff:ff:ff:ff:ff:ff 2024-05-09T04:34:50.868 INFO:teuthology.orchestra.run.smithi145.stdout: altname enp7s0f1 2024-05-09T04:34:50.868 INFO:teuthology.orchestra.run.smithi145.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T04:34:50.868 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:88:77:90 brd ff:ff:ff:ff:ff:ff 2024-05-09T04:34:50.868 INFO:teuthology.orchestra.run.smithi145.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-09T04:34:50.868 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:88:77:91 brd ff:ff:ff:ff:ff:ff 2024-05-09T04:34:50.868 INFO:teuthology.orchestra.run.smithi145.stdout: inet 172.21.15.145/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-09T04:34:50.868 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft 42570sec preferred_lft 42570sec 2024-05-09T04:34:50.869 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::ec4:7aff:fe88:7791/64 scope link 2024-05-09T04:34:50.869 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-05-09T04:34:50.869 INFO:teuthology.orchestra.run.smithi145.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-09T04:34:50.869 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0e:0f:1c:f7:96:a9 brd ff:ff:ff:ff:ff:ff 2024-05-09T04:34:50.869 INFO:teuthology.orchestra.run.smithi145.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-09T04:34:50.869 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-05-09T04:34:50.869 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::e0a8:1bff:fe9d:6975/64 scope link 2024-05-09T04:34:50.869 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-05-09T04:34:50.869 INFO:teuthology.orchestra.run.smithi145.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-09T04:34:50.869 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0e:0f:1c:f7:96:a9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T04:34:50.869 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::c0f:1cff:fef7:96a9/64 scope link 2024-05-09T04:34:50.869 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-05-09T04:34:50.869 INFO:teuthology.orchestra.run.smithi145.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-09T04:34:50.869 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether b2:86:d9:a3:05:22 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T04:34:50.869 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::b086:d9ff:fea3:522/64 scope link 2024-05-09T04:34:50.870 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-05-09T04:34:50.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:50.871 DEBUG:teuthology.orchestra.run.smithi145:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-09T04:34:50.945 INFO:teuthology.orchestra.run.smithi145.stdout:1 2024-05-09T04:34:50.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:50.949 DEBUG:teuthology.orchestra.run.smithi145:> route 2024-05-09T04:34:51.002 INFO:teuthology.orchestra.run.smithi145.stdout:Kernel IP routing table 2024-05-09T04:34:51.003 INFO:teuthology.orchestra.run.smithi145.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-09T04:34:51.003 INFO:teuthology.orchestra.run.smithi145.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-09T04:34:51.003 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-09T04:34:51.003 INFO:teuthology.orchestra.run.smithi145.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-09T04:34:51.003 DEBUG:teuthology.orchestra.run.smithi145:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T04:34:51.004 DEBUG:teuthology.orchestra.run.smithi145:> set -e 2024-05-09T04:34:51.004 DEBUG:teuthology.orchestra.run.smithi145:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-09T04:34:51.004 DEBUG:teuthology.orchestra.run.smithi145:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-09T04:34:51.004 DEBUG:teuthology.orchestra.run.smithi145:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-09T04:34:51.004 DEBUG:teuthology.orchestra.run.smithi145:> ') 2024-05-09T04:34:51.106 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T04:34:51.204 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T04:34:51.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:51.215 DEBUG:teuthology.orchestra.run.smithi145:> ip netns list 2024-05-09T04:34:51.269 INFO:teuthology.orchestra.run.smithi145.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-09T04:34:51.270 INFO:teuthology.orchestra.run.smithi145.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-09T04:34:51.271 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-09T04:34:51.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:51.271 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip link set brx.0 up 2024-05-09T04:34:51.344 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-09T04:34:51.345 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T04:34:51.345 DEBUG:teuthology.orchestra.run.smithi145:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-09T04:34:51.390 INFO:teuthology.orchestra.run.smithi145.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-09T04:34:51.393 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T04:34:51.393 DEBUG:teuthology.orchestra.run.smithi145:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-09T04:34:51.485 DEBUG:teuthology.orchestra.run.smithi145:> sudo modprobe fuse 2024-05-09T04:34:51.570 DEBUG:teuthology.orchestra.run.smithi145:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T04:34:51.627 INFO:teuthology.orchestra.run.smithi145.stdout:/proc 2024-05-09T04:34:51.627 INFO:teuthology.orchestra.run.smithi145.stdout:/sys 2024-05-09T04:34:51.627 INFO:teuthology.orchestra.run.smithi145.stdout:/dev 2024-05-09T04:34:51.627 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/kernel/security 2024-05-09T04:34:51.627 INFO:teuthology.orchestra.run.smithi145.stdout:/dev/shm 2024-05-09T04:34:51.627 INFO:teuthology.orchestra.run.smithi145.stdout:/dev/pts 2024-05-09T04:34:51.627 INFO:teuthology.orchestra.run.smithi145.stdout:/run 2024-05-09T04:34:51.627 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/fs/cgroup 2024-05-09T04:34:51.627 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/fs/pstore 2024-05-09T04:34:51.627 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/fs/bpf 2024-05-09T04:34:51.628 INFO:teuthology.orchestra.run.smithi145.stdout:/ 2024-05-09T04:34:51.628 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/fs/selinux 2024-05-09T04:34:51.628 INFO:teuthology.orchestra.run.smithi145.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T04:34:51.628 INFO:teuthology.orchestra.run.smithi145.stdout:/dev/mqueue 2024-05-09T04:34:51.628 INFO:teuthology.orchestra.run.smithi145.stdout:/dev/hugepages 2024-05-09T04:34:51.628 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/kernel/debug 2024-05-09T04:34:51.628 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/kernel/tracing 2024-05-09T04:34:51.628 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/fs/fuse/connections 2024-05-09T04:34:51.628 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/kernel/config 2024-05-09T04:34:51.628 INFO:teuthology.orchestra.run.smithi145.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T04:34:51.628 INFO:teuthology.orchestra.run.smithi145.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T04:34:51.628 INFO:teuthology.orchestra.run.smithi145.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T04:34:51.628 INFO:teuthology.orchestra.run.smithi145.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T04:34:51.628 INFO:teuthology.orchestra.run.smithi145.stdout:/run/user/1000 2024-05-09T04:34:51.629 INFO:teuthology.orchestra.run.smithi145.stdout:/var/lib/ceph 2024-05-09T04:34:51.629 INFO:teuthology.orchestra.run.smithi145.stdout:/var/lib/ceph/osd/ceph-0 2024-05-09T04:34:51.629 INFO:teuthology.orchestra.run.smithi145.stdout:/var/lib/ceph/osd/ceph-1 2024-05-09T04:34:51.629 INFO:teuthology.orchestra.run.smithi145.stdout:/var/lib/ceph/osd/ceph-2 2024-05-09T04:34:51.629 INFO:teuthology.orchestra.run.smithi145.stdout:/var/lib/ceph/osd/ceph-3 2024-05-09T04:34:51.629 INFO:teuthology.orchestra.run.smithi145.stdout:/run/netns 2024-05-09T04:34:51.629 INFO:teuthology.orchestra.run.smithi145.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T04:34:51.629 INFO:teuthology.orchestra.run.smithi145.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T04:34:51.629 INFO:teuthology.orchestra.run.smithi145.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T04:34:51.629 INFO:teuthology.orchestra.run.smithi145.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T04:34:51.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:51.630 DEBUG:teuthology.orchestra.run.smithi145:> ls /sys/fs/fuse/connections 2024-05-09T04:34:51.683 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-09T04:34:51.683 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-09T04:34:51.688 DEBUG:teuthology.orchestra.run.smithi145:> sudo modprobe fuse 2024-05-09T04:34:51.757 DEBUG:teuthology.orchestra.run.smithi145:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T04:34:51.800 INFO:teuthology.orchestra.run.smithi145.stdout:/proc 2024-05-09T04:34:51.800 INFO:teuthology.orchestra.run.smithi145.stdout:/sys 2024-05-09T04:34:51.800 INFO:teuthology.orchestra.run.smithi145.stdout:/dev 2024-05-09T04:34:51.800 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/kernel/security 2024-05-09T04:34:51.800 INFO:teuthology.orchestra.run.smithi145.stdout:/dev/shm 2024-05-09T04:34:51.800 INFO:teuthology.orchestra.run.smithi145.stdout:/dev/pts 2024-05-09T04:34:51.800 INFO:teuthology.orchestra.run.smithi145.stdout:/run 2024-05-09T04:34:51.800 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/fs/cgroup 2024-05-09T04:34:51.800 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/fs/pstore 2024-05-09T04:34:51.800 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/fs/bpf 2024-05-09T04:34:51.800 INFO:teuthology.orchestra.run.smithi145.stdout:/ 2024-05-09T04:34:51.800 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/fs/selinux 2024-05-09T04:34:51.801 INFO:teuthology.orchestra.run.smithi145.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T04:34:51.801 INFO:teuthology.orchestra.run.smithi145.stdout:/dev/mqueue 2024-05-09T04:34:51.801 INFO:teuthology.orchestra.run.smithi145.stdout:/dev/hugepages 2024-05-09T04:34:51.801 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/kernel/debug 2024-05-09T04:34:51.801 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/kernel/tracing 2024-05-09T04:34:51.801 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/fs/fuse/connections 2024-05-09T04:34:51.801 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/kernel/config 2024-05-09T04:34:51.801 INFO:teuthology.orchestra.run.smithi145.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T04:34:51.801 INFO:teuthology.orchestra.run.smithi145.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T04:34:51.801 INFO:teuthology.orchestra.run.smithi145.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T04:34:51.801 INFO:teuthology.orchestra.run.smithi145.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T04:34:51.801 INFO:teuthology.orchestra.run.smithi145.stdout:/run/user/1000 2024-05-09T04:34:51.801 INFO:teuthology.orchestra.run.smithi145.stdout:/var/lib/ceph 2024-05-09T04:34:51.801 INFO:teuthology.orchestra.run.smithi145.stdout:/var/lib/ceph/osd/ceph-0 2024-05-09T04:34:51.801 INFO:teuthology.orchestra.run.smithi145.stdout:/var/lib/ceph/osd/ceph-1 2024-05-09T04:34:51.801 INFO:teuthology.orchestra.run.smithi145.stdout:/var/lib/ceph/osd/ceph-2 2024-05-09T04:34:51.802 INFO:teuthology.orchestra.run.smithi145.stdout:/var/lib/ceph/osd/ceph-3 2024-05-09T04:34:51.802 INFO:teuthology.orchestra.run.smithi145.stdout:/run/netns 2024-05-09T04:34:51.802 INFO:teuthology.orchestra.run.smithi145.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T04:34:51.802 INFO:teuthology.orchestra.run.smithi145.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T04:34:51.802 INFO:teuthology.orchestra.run.smithi145.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T04:34:51.802 INFO:teuthology.orchestra.run.smithi145.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T04:34:51.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:51.803 DEBUG:teuthology.orchestra.run.smithi145:> ls /sys/fs/fuse/connections 2024-05-09T04:34:51.858 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi145.stderr:2024-05-09T04:34:51.859+0000 7ff4b040d480 -1 init, newargv = 0x55b84d1992d0 newargc=15 2024-05-09T04:34:51.858 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi145.stderr:ceph-fuse[46320]: starting ceph client 2024-05-09T04:34:51.866 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi145.stderr:ceph-fuse[46320]: starting fuse 2024-05-09T04:34:51.888 INFO:teuthology.orchestra.run.smithi145.stdout:40 2024-05-09T04:34:51.889 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2024-05-09T04:34:51.889 DEBUG:teuthology.orchestra.run.smithi145:> sudo stdin-killer -- python3 -c ' 2024-05-09T04:34:51.889 DEBUG:teuthology.orchestra.run.smithi145:> import glob 2024-05-09T04:34:51.889 DEBUG:teuthology.orchestra.run.smithi145:> import re 2024-05-09T04:34:51.889 DEBUG:teuthology.orchestra.run.smithi145:> import os 2024-05-09T04:34:51.889 DEBUG:teuthology.orchestra.run.smithi145:> import subprocess 2024-05-09T04:34:51.889 DEBUG:teuthology.orchestra.run.smithi145:> 2024-05-09T04:34:51.889 DEBUG:teuthology.orchestra.run.smithi145:> def _find_admin_socket(client_name): 2024-05-09T04:34:51.889 DEBUG:teuthology.orchestra.run.smithi145:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-09T04:34:51.889 DEBUG:teuthology.orchestra.run.smithi145:> files = glob.glob(asok_path) 2024-05-09T04:34:51.890 DEBUG:teuthology.orchestra.run.smithi145:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-09T04:34:51.890 DEBUG:teuthology.orchestra.run.smithi145:> 2024-05-09T04:34:51.890 DEBUG:teuthology.orchestra.run.smithi145:> # Given a non-glob path, it better be there 2024-05-09T04:34:51.890 DEBUG:teuthology.orchestra.run.smithi145:> if "*" not in asok_path: 2024-05-09T04:34:51.890 DEBUG:teuthology.orchestra.run.smithi145:> assert(len(files) == 1) 2024-05-09T04:34:51.890 DEBUG:teuthology.orchestra.run.smithi145:> return files[0] 2024-05-09T04:34:51.890 DEBUG:teuthology.orchestra.run.smithi145:> 2024-05-09T04:34:51.890 DEBUG:teuthology.orchestra.run.smithi145:> for f in files: 2024-05-09T04:34:51.890 DEBUG:teuthology.orchestra.run.smithi145:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-09T04:34:51.890 DEBUG:teuthology.orchestra.run.smithi145:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-09T04:34:51.890 DEBUG:teuthology.orchestra.run.smithi145:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-09T04:34:51.890 DEBUG:teuthology.orchestra.run.smithi145:> contents = proc_f.read() 2024-05-09T04:34:51.890 DEBUG:teuthology.orchestra.run.smithi145:> if mountpoint in contents: 2024-05-09T04:34:51.890 DEBUG:teuthology.orchestra.run.smithi145:> return f 2024-05-09T04:34:51.890 DEBUG:teuthology.orchestra.run.smithi145:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-09T04:34:51.891 DEBUG:teuthology.orchestra.run.smithi145:> 2024-05-09T04:34:51.891 DEBUG:teuthology.orchestra.run.smithi145:> print(_find_admin_socket("client.0")) 2024-05-09T04:34:51.891 DEBUG:teuthology.orchestra.run.smithi145:> ' 2024-05-09T04:34:52.081 INFO:teuthology.orchestra.run.smithi145.stdout:/var/run/ceph/ceph-client.0.46320.asok 2024-05-09T04:34:52.085 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T04:34:52.098 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.46320.asok 2024-05-09T04:34:52.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:52.098 DEBUG:teuthology.orchestra.run.smithi145:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.46320.asok status 2024-05-09T04:34:52.332 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-05-09T04:34:52.332 INFO:teuthology.orchestra.run.smithi145.stdout: "metadata": { 2024-05-09T04:34:52.332 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_sha1": "165a15e51d08073b5317c443415b0d8a7141b1ab", 2024-05-09T04:34:52.332 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_version": "ceph version 19.0.0-3568-g165a15e5 (165a15e51d08073b5317c443415b0d8a7141b1ab) squid (dev)", 2024-05-09T04:34:52.332 INFO:teuthology.orchestra.run.smithi145.stdout: "entity_id": "0", 2024-05-09T04:34:52.332 INFO:teuthology.orchestra.run.smithi145.stdout: "hostname": "smithi145", 2024-05-09T04:34:52.332 INFO:teuthology.orchestra.run.smithi145.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-09T04:34:52.332 INFO:teuthology.orchestra.run.smithi145.stdout: "pid": "46320", 2024-05-09T04:34:52.333 INFO:teuthology.orchestra.run.smithi145.stdout: "root": "/" 2024-05-09T04:34:52.333 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:34:52.333 INFO:teuthology.orchestra.run.smithi145.stdout: "dentry_count": 0, 2024-05-09T04:34:52.333 INFO:teuthology.orchestra.run.smithi145.stdout: "dentry_pinned_count": 0, 2024-05-09T04:34:52.333 INFO:teuthology.orchestra.run.smithi145.stdout: "id": 4947, 2024-05-09T04:34:52.333 INFO:teuthology.orchestra.run.smithi145.stdout: "inst": { 2024-05-09T04:34:52.333 INFO:teuthology.orchestra.run.smithi145.stdout: "name": { 2024-05-09T04:34:52.333 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "client", 2024-05-09T04:34:52.333 INFO:teuthology.orchestra.run.smithi145.stdout: "num": 4947 2024-05-09T04:34:52.333 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:34:52.333 INFO:teuthology.orchestra.run.smithi145.stdout: "addr": { 2024-05-09T04:34:52.333 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "v1", 2024-05-09T04:34:52.333 INFO:teuthology.orchestra.run.smithi145.stdout: "addr": "192.168.0.1:0", 2024-05-09T04:34:52.333 INFO:teuthology.orchestra.run.smithi145.stdout: "nonce": 597588053 2024-05-09T04:34:52.333 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-05-09T04:34:52.334 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:34:52.334 INFO:teuthology.orchestra.run.smithi145.stdout: "addr": { 2024-05-09T04:34:52.334 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "v1", 2024-05-09T04:34:52.334 INFO:teuthology.orchestra.run.smithi145.stdout: "addr": "192.168.0.1:0", 2024-05-09T04:34:52.334 INFO:teuthology.orchestra.run.smithi145.stdout: "nonce": 597588053 2024-05-09T04:34:52.334 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:34:52.334 INFO:teuthology.orchestra.run.smithi145.stdout: "inst_str": "client.4947 192.168.0.1:0/597588053", 2024-05-09T04:34:52.334 INFO:teuthology.orchestra.run.smithi145.stdout: "addr_str": "192.168.0.1:0/597588053", 2024-05-09T04:34:52.334 INFO:teuthology.orchestra.run.smithi145.stdout: "inode_count": 1, 2024-05-09T04:34:52.334 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_epoch": 17, 2024-05-09T04:34:52.334 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_epoch": 32, 2024-05-09T04:34:52.334 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_epoch_barrier": 0, 2024-05-09T04:34:52.334 INFO:teuthology.orchestra.run.smithi145.stdout: "blocklisted": false, 2024-05-09T04:34:52.334 INFO:teuthology.orchestra.run.smithi145.stdout: "fs_name": "cephfs" 2024-05-09T04:34:52.335 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-05-09T04:34:52.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:52.348 DEBUG:teuthology.orchestra.run.smithi145:> stat --file-system '--printf=%T 2024-05-09T04:34:52.348 DEBUG:teuthology.orchestra.run.smithi145:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-09T04:34:52.400 INFO:teuthology.orchestra.run.smithi145.stdout:fuseblk 2024-05-09T04:34:52.400 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-09T04:34:52.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:52.401 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-09T04:34:52.478 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_symlink_recovery false 2024-05-09T04:34:52.678 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.679+0000 7ff3c4200640 1 Processor -- start 2024-05-09T04:34:52.678 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.679+0000 7ff3c4200640 1 -- start start 2024-05-09T04:34:52.679 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.680+0000 7ff3c4200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff3bc1400b0 0x7ff3bc0cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:52.679 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.680+0000 7ff3c4200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff3bc142f30 0x7ff3bc06ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:52.679 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.681+0000 7ff3c4200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3bc06e560 con 0x7ff3bc143450 2024-05-09T04:34:52.679 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.681+0000 7ff3c4200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7ff3bc140490 con 0x7ff3bc1400b0 2024-05-09T04:34:52.679 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.681+0000 7ff3c4200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff3bc0d0290 con 0x7ff3bc142f30 2024-05-09T04:34:52.680 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.681+0000 7ff3c1800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff3bc1400b0 0x7ff3bc0cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:52.680 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.681+0000 7ff3c1800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff3bc1400b0 0x7ff3bc0cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:41128/0 (socket says 172.21.15.145:41128) 2024-05-09T04:34:52.680 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.681+0000 7ff3c1800640 1 -- 172.21.15.145:0/1999208067 learned_addr learned my addr 172.21.15.145:0/1999208067 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:52.681 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.681+0000 7ff3c2200640 1 --2- 172.21.15.145:0/1999208067 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff3bc142f30 0x7ff3bc06ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:52.681 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.681+0000 7ff3c1800640 1 -- 172.21.15.145:0/1999208067 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff3bc142f30 msgr2=0x7ff3bc06ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:52.681 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.681+0000 7ff3c1800640 1 --2- 172.21.15.145:0/1999208067 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff3bc142f30 0x7ff3bc06ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:52.681 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.681+0000 7ff3c1800640 1 -- 172.21.15.145:0/1999208067 >> v1:172.21.15.192:6789/0 conn(0x7ff3bc143450 legacy=0x7ff3bc142820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:52.681 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.682+0000 7ff3c1800640 1 -- 172.21.15.145:0/1999208067 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3bc0d0570 con 0x7ff3bc1400b0 2024-05-09T04:34:52.681 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.682+0000 7ff3c2200640 1 --2- 172.21.15.145:0/1999208067 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff3bc142f30 0x7ff3bc06ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:34:52.681 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.683+0000 7ff3c1800640 1 --2- 172.21.15.145:0/1999208067 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3bc1400b0 0x7ff3bc0cf7d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0009870 tx=0x7ff3b002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60b46566c40a6e91 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:52.682 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.683+0000 7ff3bbe00640 1 -- 172.21.15.145:0/1999208067 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b0037070 con 0x7ff3bc1400b0 2024-05-09T04:34:52.682 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.683+0000 7ff3bbe00640 1 -- 172.21.15.145:0/1999208067 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3b0032440 con 0x7ff3bc1400b0 2024-05-09T04:34:52.682 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.683+0000 7ff3bbe00640 1 -- 172.21.15.145:0/1999208067 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b0032810 con 0x7ff3bc1400b0 2024-05-09T04:34:52.682 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.683+0000 7ff3c4200640 1 -- 172.21.15.145:0/1999208067 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3bc1400b0 msgr2=0x7ff3bc0cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:52.683 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.683+0000 7ff3c4200640 1 --2- 172.21.15.145:0/1999208067 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3bc1400b0 0x7ff3bc0cf7d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0009870 tx=0x7ff3b002a1c0 comp rx=0 tx=0).stop 2024-05-09T04:34:52.683 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.684+0000 7ff3c4200640 1 -- 172.21.15.145:0/1999208067 shutdown_connections 2024-05-09T04:34:52.683 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.684+0000 7ff3c4200640 1 --2- 172.21.15.145:0/1999208067 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff3bc142f30 0x7ff3bc06ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:52.683 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.684+0000 7ff3c4200640 1 --2- 172.21.15.145:0/1999208067 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3bc1400b0 0x7ff3bc0cf7d0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:52.683 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.684+0000 7ff3c4200640 1 -- 172.21.15.145:0/1999208067 >> 172.21.15.145:0/1999208067 conn(0x7ff3bc06cab0 msgr2=0x7ff3bc0d0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:52.683 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.684+0000 7ff3c4200640 1 -- 172.21.15.145:0/1999208067 shutdown_connections 2024-05-09T04:34:52.683 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.684+0000 7ff3c4200640 1 -- 172.21.15.145:0/1999208067 wait complete. 2024-05-09T04:34:52.684 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.685+0000 7ff3c4200640 1 Processor -- start 2024-05-09T04:34:52.684 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.686+0000 7ff3c4200640 1 -- start start 2024-05-09T04:34:52.686 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.686+0000 7ff3c4200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff3bc1400b0 0x7ff3bc156e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:52.686 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.686+0000 7ff3c4200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3bc142f30 0x7ff3bc0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:52.686 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.686+0000 7ff3c4200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3bc143450 0x7ff3bc0b5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:52.686 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.686+0000 7ff3c4200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff3bc0cfd10 con 0x7ff3bc143450 2024-05-09T04:34:52.686 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.686+0000 7ff3c4200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff3bc0d0b30 con 0x7ff3bc142f30 2024-05-09T04:34:52.686 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.686+0000 7ff3c4200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff3bc0cfe90 con 0x7ff3bc1400b0 2024-05-09T04:34:52.686 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.687+0000 7ff3c2200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3bc143450 0x7ff3bc0b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:52.686 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.687+0000 7ff3c2200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3bc143450 0x7ff3bc0b5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:41140/0 (socket says 172.21.15.145:41140) 2024-05-09T04:34:52.686 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.687+0000 7ff3c2200640 1 -- 172.21.15.145:0/3307211869 learned_addr learned my addr 172.21.15.145:0/3307211869 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:52.686 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.687+0000 7ff3c1800640 1 --2- 172.21.15.145:0/3307211869 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff3bc1400b0 0x7ff3bc156e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:52.687 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.687+0000 7ff3c0e00640 1 --2- 172.21.15.145:0/3307211869 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3bc142f30 0x7ff3bc0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:52.687 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.687+0000 7ff3c2200640 1 -- 172.21.15.145:0/3307211869 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff3bc1400b0 msgr2=0x7ff3bc156e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:52.687 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.687+0000 7ff3c2200640 1 --2- 172.21.15.145:0/3307211869 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff3bc1400b0 0x7ff3bc156e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:52.687 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.687+0000 7ff3c2200640 1 -- 172.21.15.145:0/3307211869 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3bc142f30 msgr2=0x7ff3bc0b4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:52.687 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.687+0000 7ff3c2200640 1 --2- 172.21.15.145:0/3307211869 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3bc142f30 0x7ff3bc0b4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:52.687 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.687+0000 7ff3c2200640 1 -- 172.21.15.145:0/3307211869 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3bc15b5e0 con 0x7ff3bc143450 2024-05-09T04:34:52.688 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.688+0000 7ff3c2200640 1 --2- 172.21.15.145:0/3307211869 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3bc143450 0x7ff3bc0b5390 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff3b400cef0 tx=0x7ff3b400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:52.688 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.688+0000 7ff3b9600640 1 -- 172.21.15.145:0/3307211869 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b401a070 con 0x7ff3bc143450 2024-05-09T04:34:52.688 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.688+0000 7ff3b9600640 1 -- 172.21.15.145:0/3307211869 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3b400d860 con 0x7ff3bc143450 2024-05-09T04:34:52.688 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.688+0000 7ff3c4200640 1 -- 172.21.15.145:0/3307211869 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3bc15b8c0 con 0x7ff3bc143450 2024-05-09T04:34:52.688 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.688+0000 7ff3b9600640 1 -- 172.21.15.145:0/3307211869 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b400dc50 con 0x7ff3bc143450 2024-05-09T04:34:52.688 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.688+0000 7ff3c4200640 1 -- 172.21.15.145:0/3307211869 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3bc15be80 con 0x7ff3bc143450 2024-05-09T04:34:52.688 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.689+0000 7ff3c4200640 1 -- 172.21.15.145:0/3307211869 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3bc06a310 con 0x7ff3bc143450 2024-05-09T04:34:52.689 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.690+0000 7ff3b9600640 1 -- 172.21.15.145:0/3307211869 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff3b401d070 con 0x7ff3bc143450 2024-05-09T04:34:52.690 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.692+0000 7ff3b9600640 1 --2- 172.21.15.145:0/3307211869 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff39c076300 0x7ff39c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:52.691 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.692+0000 7ff3b9600640 1 -- 172.21.15.145:0/3307211869 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7ff3b4097ed0 con 0x7ff3bc143450 2024-05-09T04:34:52.691 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.692+0000 7ff3c1800640 1 --2- 172.21.15.145:0/3307211869 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff39c076300 0x7ff39c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:52.691 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.693+0000 7ff3c1800640 1 --2- 172.21.15.145:0/3307211869 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff39c076300 0x7ff39c0787c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0034000 tx=0x7ff3b002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:52.695 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.697+0000 7ff3b9600640 1 -- 172.21.15.145:0/3307211869 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff3b40618e0 con 0x7ff3bc143450 2024-05-09T04:34:52.902 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.903+0000 7ff3c4200640 1 -- 172.21.15.145:0/3307211869 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command([{prefix=config set, name=mds_symlink_recovery}] v 0) -- 0x7ff3bc0af930 con 0x7ff3bc143450 2024-05-09T04:34:52.909 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.911+0000 7ff3b9600640 1 -- 172.21.15.145:0/3307211869 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_symlink_recovery}]=0 v11)=0 v11) ==== 122+0+0 (secure 0 0 0) 0x7ff3b40611e0 con 0x7ff3bc143450 2024-05-09T04:34:52.913 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.914+0000 7ff3c4200640 1 -- 172.21.15.145:0/3307211869 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff39c076300 msgr2=0x7ff39c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:52.913 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.914+0000 7ff3c4200640 1 --2- 172.21.15.145:0/3307211869 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff39c076300 0x7ff39c0787c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0034000 tx=0x7ff3b002e000 comp rx=0 tx=0).stop 2024-05-09T04:34:52.913 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.914+0000 7ff3c4200640 1 -- 172.21.15.145:0/3307211869 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3bc143450 msgr2=0x7ff3bc0b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:52.913 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.914+0000 7ff3c4200640 1 --2- 172.21.15.145:0/3307211869 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3bc143450 0x7ff3bc0b5390 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff3b400cef0 tx=0x7ff3b400a710 comp rx=0 tx=0).stop 2024-05-09T04:34:52.913 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.914+0000 7ff3c4200640 1 -- 172.21.15.145:0/3307211869 shutdown_connections 2024-05-09T04:34:52.913 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.914+0000 7ff3c4200640 1 --2- 172.21.15.145:0/3307211869 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff39c076300 0x7ff39c0787c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:52.913 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.914+0000 7ff3c4200640 1 --2- 172.21.15.145:0/3307211869 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff3bc143450 0x7ff3bc0b5390 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:52.913 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.914+0000 7ff3c4200640 1 --2- 172.21.15.145:0/3307211869 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3bc142f30 0x7ff3bc0b4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:52.913 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.914+0000 7ff3c4200640 1 --2- 172.21.15.145:0/3307211869 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff3bc1400b0 0x7ff3bc156e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:52.913 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.914+0000 7ff3c4200640 1 -- 172.21.15.145:0/3307211869 >> 172.21.15.145:0/3307211869 conn(0x7ff3bc06cab0 msgr2=0x7ff3bc0b07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:52.914 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.914+0000 7ff3c4200640 1 -- 172.21.15.145:0/3307211869 shutdown_connections 2024-05-09T04:34:52.914 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:52.914+0000 7ff3c4200640 1 -- 172.21.15.145:0/3307211869 wait complete. 2024-05-09T04:34:52.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:52.932 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir sym_dir0) 2024-05-09T04:34:52.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:52.996 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch sym_dir0/file1) 2024-05-09T04:34:53.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:53.053 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest/mnt.0 && exec ln -s sym_dir0/file1 sym_dir0/symlink_file1) 2024-05-09T04:34:53.103 DEBUG:teuthology.orchestra.run.smithi145:> stdin-killer -- python3 -c ' 2024-05-09T04:34:53.103 DEBUG:teuthology.orchestra.run.smithi145:> import os 2024-05-09T04:34:53.103 DEBUG:teuthology.orchestra.run.smithi145:> import stat 2024-05-09T04:34:53.103 DEBUG:teuthology.orchestra.run.smithi145:> 2024-05-09T04:34:53.103 DEBUG:teuthology.orchestra.run.smithi145:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/sym_dir0/symlink_file1").st_ino) 2024-05-09T04:34:53.103 DEBUG:teuthology.orchestra.run.smithi145:> ' 2024-05-09T04:34:53.237 INFO:teuthology.orchestra.run.smithi145.stdout:1099511628278 2024-05-09T04:34:53.242 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T04:34:53.251 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T04:34:53.497 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.498+0000 7f080ae00640 1 Processor -- start 2024-05-09T04:34:53.497 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.498+0000 7f080ae00640 1 -- start start 2024-05-09T04:34:53.499 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.499+0000 7f080ae00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f0804142200 0x7f08040cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:53.499 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.499+0000 7f080ae00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f08040cfe10 0x7f08040726b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:53.499 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.499+0000 7f080ae00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f080413f340 con 0x7f080415fbc0 2024-05-09T04:34:53.499 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.500+0000 7f080ae00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f0804072220 con 0x7f0804142200 2024-05-09T04:34:53.499 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.500+0000 7f080ae00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f08040d02b0 con 0x7f08040cfe10 2024-05-09T04:34:53.499 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.500+0000 7f07fb400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f0804142200 0x7f08040cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:53.499 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.500+0000 7f07fb400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f0804142200 0x7f08040cf8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:41152/0 (socket says 172.21.15.145:41152) 2024-05-09T04:34:53.499 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.500+0000 7f07fb400640 1 -- 172.21.15.145:0/3138827774 learned_addr learned my addr 172.21.15.145:0/3138827774 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:53.499 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.500+0000 7f0808e00640 1 --2- 172.21.15.145:0/3138827774 >> v2:172.21.15.192:3300/0 conn(0x7f08040cfe10 0x7f08040726b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:53.499 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.500+0000 7f07fb400640 1 -- 172.21.15.145:0/3138827774 >> v2:172.21.15.192:3300/0 conn(0x7f08040cfe10 msgr2=0x7f08040726b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:53.500 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.500+0000 7f07fb400640 1 --2- 172.21.15.145:0/3138827774 >> v2:172.21.15.192:3300/0 conn(0x7f08040cfe10 0x7f08040726b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:53.500 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.500+0000 7f07fb400640 1 -- 172.21.15.145:0/3138827774 >> v1:172.21.15.192:6789/0 conn(0x7f080415fbc0 legacy=0x7f080415ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:53.500 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.500+0000 7f07fb400640 1 -- 172.21.15.145:0/3138827774 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08040d0590 con 0x7f0804142200 2024-05-09T04:34:53.500 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.501+0000 7f07fb400640 1 --2- 172.21.15.145:0/3138827774 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0804142200 0x7f08040cf8d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f08040b4020 tx=0x7f07ec02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f47517fd952374aa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:53.500 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.502+0000 7f0803400640 1 -- 172.21.15.145:0/3138827774 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ec037070 con 0x7f0804142200 2024-05-09T04:34:53.501 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.502+0000 7f0803400640 1 -- 172.21.15.145:0/3138827774 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07ec032440 con 0x7f0804142200 2024-05-09T04:34:53.501 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.502+0000 7f0803400640 1 -- 172.21.15.145:0/3138827774 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ec032810 con 0x7f0804142200 2024-05-09T04:34:53.501 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.502+0000 7f080ae00640 1 -- 172.21.15.145:0/3138827774 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0804142200 msgr2=0x7f08040cf8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:53.501 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.502+0000 7f080ae00640 1 --2- 172.21.15.145:0/3138827774 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0804142200 0x7f08040cf8d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f08040b4020 tx=0x7f07ec02a140 comp rx=0 tx=0).stop 2024-05-09T04:34:53.502 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.502+0000 7f080ae00640 1 -- 172.21.15.145:0/3138827774 shutdown_connections 2024-05-09T04:34:53.502 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.502+0000 7f080ae00640 1 --2- 172.21.15.145:0/3138827774 >> v2:172.21.15.192:3300/0 conn(0x7f08040cfe10 0x7f08040726b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:53.502 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.502+0000 7f080ae00640 1 --2- 172.21.15.145:0/3138827774 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0804142200 0x7f08040cf8d0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:53.502 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.502+0000 7f080ae00640 1 -- 172.21.15.145:0/3138827774 >> 172.21.15.145:0/3138827774 conn(0x7f080406c870 msgr2=0x7f080406cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:53.502 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.502+0000 7f080ae00640 1 -- 172.21.15.145:0/3138827774 shutdown_connections 2024-05-09T04:34:53.502 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.503+0000 7f080ae00640 1 -- 172.21.15.145:0/3138827774 wait complete. 2024-05-09T04:34:53.503 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.504+0000 7f080ae00640 1 Processor -- start 2024-05-09T04:34:53.503 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.504+0000 7f080ae00640 1 -- start start 2024-05-09T04:34:53.506 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.505+0000 7f080ae00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08040cfe10 0x7f08040afea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:53.506 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.505+0000 7f080ae00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0804142200 0x7f0804140b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:53.506 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.505+0000 7f080ae00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f080415fbc0 0x7f0804141050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:53.506 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.505+0000 7f080ae00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0804072bf0 con 0x7f080415fbc0 2024-05-09T04:34:53.506 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.505+0000 7f080ae00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f08040d0b50 con 0x7f0804142200 2024-05-09T04:34:53.506 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.505+0000 7f080ae00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0804072d70 con 0x7f08040cfe10 2024-05-09T04:34:53.506 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.506+0000 7f0808e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f080415fbc0 0x7f0804141050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:53.507 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.506+0000 7f0808e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f080415fbc0 0x7f0804141050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:41168/0 (socket says 172.21.15.145:41168) 2024-05-09T04:34:53.507 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.506+0000 7f0808e00640 1 -- 172.21.15.145:0/426816000 learned_addr learned my addr 172.21.15.145:0/426816000 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:53.507 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.506+0000 7f0803e00640 1 --2- 172.21.15.145:0/426816000 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08040cfe10 0x7f08040afea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:53.507 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.506+0000 7f07fb400640 1 --2- 172.21.15.145:0/426816000 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0804142200 0x7f0804140b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:53.507 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.506+0000 7f0808e00640 1 -- 172.21.15.145:0/426816000 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08040cfe10 msgr2=0x7f08040afea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:53.507 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.506+0000 7f0808e00640 1 --2- 172.21.15.145:0/426816000 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08040cfe10 0x7f08040afea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:53.507 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.506+0000 7f0808e00640 1 -- 172.21.15.145:0/426816000 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0804142200 msgr2=0x7f0804140b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:53.507 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.506+0000 7f0808e00640 1 --2- 172.21.15.145:0/426816000 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0804142200 0x7f0804140b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:53.507 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.506+0000 7f0808e00640 1 -- 172.21.15.145:0/426816000 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f080412cdf0 con 0x7f080415fbc0 2024-05-09T04:34:53.507 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.506+0000 7f0803e00640 1 --2- 172.21.15.145:0/426816000 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08040cfe10 0x7f08040afea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:34:53.507 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.506+0000 7f0808e00640 1 --2- 172.21.15.145:0/426816000 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f080415fbc0 0x7f0804141050 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f07fc00cd60 tx=0x7f07fc00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:53.507 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.507+0000 7f0800c00640 1 -- 172.21.15.145:0/426816000 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07fc003b90 con 0x7f080415fbc0 2024-05-09T04:34:53.507 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.507+0000 7f0800c00640 1 -- 172.21.15.145:0/426816000 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07fc003d50 con 0x7f080415fbc0 2024-05-09T04:34:53.507 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.507+0000 7f080ae00640 1 -- 172.21.15.145:0/426816000 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f080415d950 con 0x7f080415fbc0 2024-05-09T04:34:53.507 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.507+0000 7f0800c00640 1 -- 172.21.15.145:0/426816000 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07fc004260 con 0x7f080415fbc0 2024-05-09T04:34:53.507 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.507+0000 7f080ae00640 1 -- 172.21.15.145:0/426816000 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f080412c080 con 0x7f080415fbc0 2024-05-09T04:34:53.508 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.508+0000 7f080ae00640 1 -- 172.21.15.145:0/426816000 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f080406a310 con 0x7f080415fbc0 2024-05-09T04:34:53.508 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.509+0000 7f0800c00640 1 -- 172.21.15.145:0/426816000 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f07fc004400 con 0x7f080415fbc0 2024-05-09T04:34:53.509 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.510+0000 7f0800c00640 1 --2- 172.21.15.145:0/426816000 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f07e00762d0 0x7f07e0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:53.509 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.510+0000 7f0800c00640 1 -- 172.21.15.145:0/426816000 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f07fc098120 con 0x7f080415fbc0 2024-05-09T04:34:53.509 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.510+0000 7f0803e00640 1 --2- 172.21.15.145:0/426816000 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f07e00762d0 0x7f07e0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:53.509 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.511+0000 7f0803e00640 1 --2- 172.21.15.145:0/426816000 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f07e00762d0 0x7f07e0078790 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f07f400a810 tx=0x7f07f4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:53.513 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.514+0000 7f0800c00640 1 -- 172.21.15.145:0/426816000 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f07fc061390 con 0x7f080415fbc0 2024-05-09T04:34:53.763 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.764+0000 7f080ae00640 1 -- 172.21.15.145:0/426816000 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f08040742c0 con 0x7f080415fbc0 2024-05-09T04:34:53.764 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.766+0000 7f0800c00640 1 -- 172.21.15.145:0/426816000 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5187 (secure 0 0 0) 0x7f07fc065690 con 0x7f080415fbc0 2024-05-09T04:34:53.764 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:34:53.764 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4499,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6834/3227728059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3227728059},{"type":"v1","addr":"172.21.15.192:6834","nonce":3227728059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4502,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1603013787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6833","nonce":1603013787},{"type":"v1","addr":"172.21.15.192:6835","nonce":1603013787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4672,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/68778418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":68778418},{"type":"v1","addr":"172.21.15.145:6837","nonce":68778418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T04:34:43.133970+0000","modified":"2024-05-09T04:34:46.181657+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4761},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4761":{"gid":4761,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.145:6835/4260798843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4260798843},{"type":"v1","addr":"172.21.15.145:6835","nonce":4260798843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4761,"qdb_cluster":[4761]},"id":2}]} 2024-05-09T04:34:53.765 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 17 2024-05-09T04:34:53.767 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.768+0000 7f080ae00640 1 -- 172.21.15.145:0/426816000 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f07e00762d0 msgr2=0x7f07e0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:53.767 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.768+0000 7f080ae00640 1 --2- 172.21.15.145:0/426816000 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f07e00762d0 0x7f07e0078790 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f07f400a810 tx=0x7f07f4008040 comp rx=0 tx=0).stop 2024-05-09T04:34:53.768 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.768+0000 7f080ae00640 1 -- 172.21.15.145:0/426816000 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f080415fbc0 msgr2=0x7f0804141050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:53.768 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.768+0000 7f080ae00640 1 --2- 172.21.15.145:0/426816000 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f080415fbc0 0x7f0804141050 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f07fc00cd60 tx=0x7f07fc00a660 comp rx=0 tx=0).stop 2024-05-09T04:34:53.768 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.769+0000 7f080ae00640 1 -- 172.21.15.145:0/426816000 shutdown_connections 2024-05-09T04:34:53.768 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.769+0000 7f080ae00640 1 --2- 172.21.15.145:0/426816000 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f07e00762d0 0x7f07e0078790 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:53.768 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.769+0000 7f080ae00640 1 --2- 172.21.15.145:0/426816000 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f080415fbc0 0x7f0804141050 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:53.768 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.769+0000 7f080ae00640 1 --2- 172.21.15.145:0/426816000 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0804142200 0x7f0804140b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:53.768 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.769+0000 7f080ae00640 1 --2- 172.21.15.145:0/426816000 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08040cfe10 0x7f08040afea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:53.768 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.769+0000 7f080ae00640 1 -- 172.21.15.145:0/426816000 >> 172.21.15.145:0/426816000 conn(0x7f080406c870 msgr2=0x7f08040b0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:53.768 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.769+0000 7f080ae00640 1 -- 172.21.15.145:0/426816000 shutdown_connections 2024-05-09T04:34:53.768 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:53.769+0000 7f080ae00640 1 -- 172.21.15.145:0/426816000 wait complete. 2024-05-09T04:34:53.788 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-05-09T04:34:54.036 INFO:teuthology.orchestra.run.smithi145.stdout:{"message":"","return_code":0} 2024-05-09T04:34:54.036 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-09T04:34:54.037 DEBUG:teuthology.orchestra.run.smithi145:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f6.00000000 symlink 2024-05-09T04:34:54.143 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.144+0000 7f9ee9b37f40 1 Processor -- start 2024-05-09T04:34:54.143 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.144+0000 7f9ee9b37f40 1 -- start start 2024-05-09T04:34:54.144 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.145+0000 7f9ee9b37f40 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ecc0028c0 0x558551284420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:54.144 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.145+0000 7f9ee9b37f40 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558551288490 0x5585511e1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:54.144 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.145+0000 7f9ee9b37f40 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5585511e2eb0 0x5585511e3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:54.144 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.145+0000 7f9ee9b37f40 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5585511fc6e0 con 0x558551288490 2024-05-09T04:34:54.144 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.145+0000 7f9ee9b37f40 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x558551159f50 con 0x7f9ecc0028c0 2024-05-09T04:34:54.144 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.145+0000 7f9ee9b37f40 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x5585511fc980 con 0x5585511e2eb0 2024-05-09T04:34:54.144 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.145+0000 7f9ee6400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558551288490 0x5585511e1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:54.145 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.145+0000 7f9ee6400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558551288490 0x5585511e1330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:41186/0 (socket says 172.21.15.145:41186) 2024-05-09T04:34:54.145 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.145+0000 7f9ee6400640 1 -- 172.21.15.145:0/4230123852 learned_addr learned my addr 172.21.15.145:0/4230123852 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:54.145 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.145+0000 7f9ee6400640 1 -- 172.21.15.145:0/4230123852 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5585511e2eb0 msgr2=0x5585511e3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:54.145 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.145+0000 7f9ee6400640 1 --2- 172.21.15.145:0/4230123852 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5585511e2eb0 0x5585511e3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:54.145 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.145+0000 7f9ee6400640 1 -- 172.21.15.145:0/4230123852 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ecc0028c0 msgr2=0x558551284420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:54.145 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.145+0000 7f9ee6400640 1 --2- 172.21.15.145:0/4230123852 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ecc0028c0 0x558551284420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:54.145 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.145+0000 7f9ee6400640 1 -- 172.21.15.145:0/4230123852 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585510dcb30 con 0x558551288490 2024-05-09T04:34:54.145 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.146+0000 7f9ee6400640 1 --2- 172.21.15.145:0/4230123852 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558551288490 0x5585511e1330 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4025e00 tx=0x7f9ed4051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a786926849d44317 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:54.146 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.146+0000 7f9ee5000640 1 -- 172.21.15.145:0/4230123852 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed404f860 con 0x558551288490 2024-05-09T04:34:54.147 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.146+0000 7f9ee5000640 1 -- 172.21.15.145:0/4230123852 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ed404fa00 con 0x558551288490 2024-05-09T04:34:54.147 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.146+0000 7f9ee5000640 1 -- 172.21.15.145:0/4230123852 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed404fdf0 con 0x558551288490 2024-05-09T04:34:54.147 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.146+0000 7f9ee9b37f40 1 -- 172.21.15.145:0/4230123852 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558551288490 msgr2=0x5585511e1330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:54.147 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.146+0000 7f9ee9b37f40 1 --2- 172.21.15.145:0/4230123852 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558551288490 0x5585511e1330 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4025e00 tx=0x7f9ed4051000 comp rx=0 tx=0).stop 2024-05-09T04:34:54.147 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.147+0000 7f9ee9b37f40 1 -- 172.21.15.145:0/4230123852 shutdown_connections 2024-05-09T04:34:54.147 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.147+0000 7f9ee9b37f40 1 --2- 172.21.15.145:0/4230123852 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5585511e2eb0 0x5585511e3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:54.147 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.147+0000 7f9ee9b37f40 1 --2- 172.21.15.145:0/4230123852 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x558551288490 0x5585511e1330 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:54.147 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.147+0000 7f9ee9b37f40 1 --2- 172.21.15.145:0/4230123852 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ecc0028c0 0x558551284420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:54.147 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.147+0000 7f9ee9b37f40 1 -- 172.21.15.145:0/4230123852 >> 172.21.15.145:0/4230123852 conn(0x5585511fddf0 msgr2=0x558551288d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:54.147 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.147+0000 7f9ee9b37f40 1 -- 172.21.15.145:0/4230123852 shutdown_connections 2024-05-09T04:34:54.147 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.147+0000 7f9ee9b37f40 1 -- 172.21.15.145:0/4230123852 wait complete. 2024-05-09T04:34:54.147 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.147+0000 7f9ee9b37f40 1 Processor -- start 2024-05-09T04:34:54.147 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.148+0000 7f9ee9b37f40 1 -- start start 2024-05-09T04:34:54.149 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.149+0000 7f9ee9b37f40 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5585511e2eb0 0x55855127f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:54.149 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.149+0000 7f9ee9b37f40 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558551288490 0x55855127ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:54.149 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.149+0000 7f9ee9b37f40 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9ecc0028c0 0x5585511f91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:54.150 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.149+0000 7f9ee9b37f40 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x558551288220 con 0x7f9ecc0028c0 2024-05-09T04:34:54.150 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.149+0000 7f9ee9b37f40 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5585511e1870 con 0x558551288490 2024-05-09T04:34:54.150 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.149+0000 7f9ee9b37f40 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x55855128dc90 con 0x5585511e2eb0 2024-05-09T04:34:54.150 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.149+0000 7f9ee7800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9ecc0028c0 0x5585511f91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:54.150 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.149+0000 7f9ee7800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9ecc0028c0 0x5585511f91c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:41196/0 (socket says 172.21.15.145:41196) 2024-05-09T04:34:54.150 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.149+0000 7f9ee7800640 1 -- 172.21.15.145:0/1725925119 learned_addr learned my addr 172.21.15.145:0/1725925119 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:54.150 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.149+0000 7f9ee6e00640 1 --2- 172.21.15.145:0/1725925119 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5585511e2eb0 0x55855127f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:54.150 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.149+0000 7f9ee6400640 1 --2- 172.21.15.145:0/1725925119 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558551288490 0x55855127ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:54.150 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.149+0000 7f9ee7800640 1 -- 172.21.15.145:0/1725925119 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5585511e2eb0 msgr2=0x55855127f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:54.150 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.149+0000 7f9ee7800640 1 --2- 172.21.15.145:0/1725925119 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5585511e2eb0 0x55855127f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:54.150 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.149+0000 7f9ee7800640 1 -- 172.21.15.145:0/1725925119 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558551288490 msgr2=0x55855127ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:54.150 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.149+0000 7f9ee7800640 1 --2- 172.21.15.145:0/1725925119 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558551288490 0x55855127ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:54.150 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.149+0000 7f9ee7800640 1 -- 172.21.15.145:0/1725925119 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558551283b90 con 0x7f9ecc0028c0 2024-05-09T04:34:54.150 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.150+0000 7f9ee7800640 1 --2- 172.21.15.145:0/1725925119 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9ecc0028c0 0x5585511f91c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x5585511fdd80 tx=0x7f9ee0009e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:54.150 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.150+0000 7f9edf400640 1 -- 172.21.15.145:0/1725925119 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee0008e40 con 0x7f9ecc0028c0 2024-05-09T04:34:54.151 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.150+0000 7f9edf400640 1 -- 172.21.15.145:0/1725925119 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ee0019070 con 0x7f9ecc0028c0 2024-05-09T04:34:54.151 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.150+0000 7f9edf400640 1 -- 172.21.15.145:0/1725925119 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee0014640 con 0x7f9ecc0028c0 2024-05-09T04:34:54.151 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.150+0000 7f9ee9b37f40 1 -- 172.21.15.145:0/1725925119 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5585511f8400 con 0x7f9ecc0028c0 2024-05-09T04:34:54.151 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.150+0000 7f9ee9b37f40 1 -- 172.21.15.145:0/1725925119 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585511f89c0 con 0x7f9ecc0028c0 2024-05-09T04:34:54.151 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.152+0000 7f9edf400640 1 -- 172.21.15.145:0/1725925119 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9ee0013070 con 0x7f9ecc0028c0 2024-05-09T04:34:54.152 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.153+0000 7f9edf400640 1 --2- 172.21.15.145:0/1725925119 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f9ec8074cb0 0x7f9ec8077170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:54.152 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.153+0000 7f9edf400640 1 -- 172.21.15.145:0/1725925119 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f9ee0095c70 con 0x7f9ecc0028c0 2024-05-09T04:34:54.152 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.154+0000 7f9ee6e00640 1 --2- 172.21.15.145:0/1725925119 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f9ec8074cb0 0x7f9ec8077170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:54.152 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.154+0000 7f9ee9b37f40 1 --2- 172.21.15.145:0/1725925119 >> [v2:172.21.15.192:6816/3117700995,v1:172.21.15.192:6818/3117700995] conn(0x558551280340 0x558551284970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:54.153 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.154+0000 7f9ee9b37f40 1 -- 172.21.15.145:0/1725925119 --> [v2:172.21.15.192:6816/3117700995,v1:172.21.15.192:6818/3117700995] -- osd_op(unknown.0.0:1 5.2a 5:57fe7535:::100000001f6.00000000:head [getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x558551284eb0 con 0x558551280340 2024-05-09T04:34:54.153 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.154+0000 7f9ee6e00640 1 --2- 172.21.15.145:0/1725925119 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f9ec8074cb0 0x7f9ec8077170 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9ed80014e0 tx=0x7f9ed8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:54.153 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.154+0000 7f9ee6400640 1 --2- 172.21.15.145:0/1725925119 >> [v2:172.21.15.192:6816/3117700995,v1:172.21.15.192:6818/3117700995] conn(0x558551280340 0x558551284970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:54.154 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.156+0000 7f9ee6400640 1 --2- 172.21.15.145:0/1725925119 >> [v2:172.21.15.192:6816/3117700995,v1:172.21.15.192:6818/3117700995] conn(0x558551280340 0x558551284970 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:54.156 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.156+0000 7f9ee6400640 1 -- 172.21.15.145:0/1725925119 <== osd.4 v2:172.21.15.192:6816/3117700995 1 ==== osd_op_reply(1 100000001f6.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) ==== 164+0+0 (crc 0 0 0) 0x7f9ed404fc90 con 0x558551280340 2024-05-09T04:34:54.156 INFO:teuthology.orchestra.run.smithi145.stderr:error getting xattr cephfs_data/100000001f6.00000000/symlink: (61) No data available 2024-05-09T04:34:54.156 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.157+0000 7f9ee9b37f40 1 -- 172.21.15.145:0/1725925119 >> [v2:172.21.15.192:6816/3117700995,v1:172.21.15.192:6818/3117700995] conn(0x558551280340 msgr2=0x558551284970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:54.156 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.157+0000 7f9ee9b37f40 1 --2- 172.21.15.145:0/1725925119 >> [v2:172.21.15.192:6816/3117700995,v1:172.21.15.192:6818/3117700995] conn(0x558551280340 0x558551284970 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:54.156 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.157+0000 7f9ee9b37f40 1 -- 172.21.15.145:0/1725925119 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f9ec8074cb0 msgr2=0x7f9ec8077170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:54.156 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.157+0000 7f9ee9b37f40 1 --2- 172.21.15.145:0/1725925119 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f9ec8074cb0 0x7f9ec8077170 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9ed80014e0 tx=0x7f9ed8008040 comp rx=0 tx=0).stop 2024-05-09T04:34:54.156 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.157+0000 7f9ee9b37f40 1 -- 172.21.15.145:0/1725925119 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9ecc0028c0 msgr2=0x5585511f91c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:54.156 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.157+0000 7f9ee9b37f40 1 --2- 172.21.15.145:0/1725925119 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9ecc0028c0 0x5585511f91c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x5585511fdd80 tx=0x7f9ee0009e50 comp rx=0 tx=0).stop 2024-05-09T04:34:54.156 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.157+0000 7f9ee7800640 1 -- 172.21.15.145:0/1725925119 reap_dead start 2024-05-09T04:34:54.157 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.157+0000 7f9ee9b37f40 1 -- 172.21.15.145:0/1725925119 shutdown_connections 2024-05-09T04:34:54.157 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.157+0000 7f9ee9b37f40 1 -- 172.21.15.145:0/1725925119 >> 172.21.15.145:0/1725925119 conn(0x5585511fddf0 msgr2=0x558551288d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:54.157 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.157+0000 7f9ee9b37f40 1 -- 172.21.15.145:0/1725925119 shutdown_connections 2024-05-09T04:34:54.157 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.157+0000 7f9ee9b37f40 1 -- 172.21.15.145:0/1725925119 wait complete. 2024-05-09T04:34:54.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T04:34:54.160 ERROR:tasks.cephfs.filesystem:Command failed on smithi145 with status 1: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f6.00000000 symlink' 2024-05-09T04:34:54.160 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_symlink_recovery true 2024-05-09T04:34:54.349 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.349+0000 7fc6b7200640 1 Processor -- start 2024-05-09T04:34:54.349 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.350+0000 7fc6b7200640 1 -- start start 2024-05-09T04:34:54.350 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.351+0000 7fc6b7200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fc6b0142f30 0x7fc6b0167fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:54.350 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.351+0000 7fc6b7200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6b006e560 con 0x7fc6b0143450 2024-05-09T04:34:54.350 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.351+0000 7fc6b7200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6b0140490 con 0x7fc6b01400b0 2024-05-09T04:34:54.350 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.351+0000 7fc6b7200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fc6b00d0290 con 0x7fc6b0142f30 2024-05-09T04:34:54.351 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.352+0000 7fc6af400640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7fc6b0143450 0x7fc6b0142820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:58346/0 (socket says 172.21.15.145:58346) 2024-05-09T04:34:54.351 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.352+0000 7fc6af400640 1 -- 172.21.15.145:0/4186637199 learned_addr learned my addr 172.21.15.145:0/4186637199 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:34:54.351 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.352+0000 7fc6b5200640 1 --2- 172.21.15.145:0/4186637199 >> v2:172.21.15.145:3300/0 conn(0x7fc6b0142f30 0x7fc6b0167fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:54.351 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.352+0000 7fc6b5200640 1 -- 172.21.15.145:0/4186637199 >> v1:172.21.15.192:6789/0 conn(0x7fc6b01400b0 legacy=0x7fc6b00cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:54.351 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.352+0000 7fc6b5200640 1 -- 172.21.15.145:0/4186637199 >> v1:172.21.15.145:6789/0 conn(0x7fc6b0143450 legacy=0x7fc6b0142820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:54.351 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.352+0000 7fc6b5200640 1 -- 172.21.15.145:0/4186637199 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6b00d0570 con 0x7fc6b0142f30 2024-05-09T04:34:54.352 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.352+0000 7fc6aea00640 1 -- 172.21.15.145:0/4186637199 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 965237997 0 0) 0x7fc6b006e560 con 0x7fc6b0143450 2024-05-09T04:34:54.352 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.353+0000 7fc6b5200640 1 --2- 172.21.15.145:0/4186637199 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc6b0142f30 0x7fc6b0167fe0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8009d90 tx=0x7fc6a802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e685bb76119950b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:54.352 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.354+0000 7fc6aea00640 1 -- 172.21.15.145:0/4186637199 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a8007740 con 0x7fc6b0142f30 2024-05-09T04:34:54.352 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.354+0000 7fc6aea00640 1 -- 172.21.15.145:0/4186637199 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6a80078e0 con 0x7fc6b0142f30 2024-05-09T04:34:54.352 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.354+0000 7fc6aea00640 1 -- 172.21.15.145:0/4186637199 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a8007cb0 con 0x7fc6b0142f30 2024-05-09T04:34:54.353 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.354+0000 7fc6b7200640 1 -- 172.21.15.145:0/4186637199 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc6b0142f30 msgr2=0x7fc6b0167fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:54.353 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.354+0000 7fc6b7200640 1 --2- 172.21.15.145:0/4186637199 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc6b0142f30 0x7fc6b0167fe0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8009d90 tx=0x7fc6a802a660 comp rx=0 tx=0).stop 2024-05-09T04:34:54.353 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.354+0000 7fc6b7200640 1 -- 172.21.15.145:0/4186637199 shutdown_connections 2024-05-09T04:34:54.353 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.354+0000 7fc6b7200640 1 --2- 172.21.15.145:0/4186637199 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc6b0142f30 0x7fc6b0167fe0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:54.354 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.354+0000 7fc6b7200640 1 -- 172.21.15.145:0/4186637199 >> 172.21.15.145:0/4186637199 conn(0x7fc6b006cab0 msgr2=0x7fc6b00d0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:54.354 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.354+0000 7fc6b7200640 1 -- 172.21.15.145:0/4186637199 shutdown_connections 2024-05-09T04:34:54.354 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.354+0000 7fc6b7200640 1 -- 172.21.15.145:0/4186637199 wait complete. 2024-05-09T04:34:54.355 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.356+0000 7fc6b7200640 1 Processor -- start 2024-05-09T04:34:54.355 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.356+0000 7fc6b7200640 1 -- start start 2024-05-09T04:34:54.357 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.357+0000 7fc6b7200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc6b01400b0 0x7fc6b01f5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:54.357 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.357+0000 7fc6b7200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6b0142f30 0x7fc6b015a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:54.357 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.357+0000 7fc6b7200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc6b0143450 0x7fc6b01563d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:54.357 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.357+0000 7fc6b7200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b00cfee0 con 0x7fc6b01400b0 2024-05-09T04:34:54.357 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.357+0000 7fc6b7200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b00d0b30 con 0x7fc6b0142f30 2024-05-09T04:34:54.357 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.357+0000 7fc6b7200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc6b015a6b0 con 0x7fc6b0143450 2024-05-09T04:34:54.357 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.357+0000 7fc6afe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc6b01400b0 0x7fc6b01f5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:54.357 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.357+0000 7fc6afe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc6b01400b0 0x7fc6b01f5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:41218/0 (socket says 172.21.15.145:41218) 2024-05-09T04:34:54.357 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.358+0000 7fc6afe00640 1 -- 172.21.15.145:0/3000658454 learned_addr learned my addr 172.21.15.145:0/3000658454 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:54.357 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.358+0000 7fc6b5200640 1 --2- 172.21.15.145:0/3000658454 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc6b0143450 0x7fc6b01563d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:54.357 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.358+0000 7fc6af400640 1 --2- 172.21.15.145:0/3000658454 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6b0142f30 0x7fc6b015a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:54.358 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.358+0000 7fc6afe00640 1 -- 172.21.15.145:0/3000658454 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc6b0143450 msgr2=0x7fc6b01563d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:54.358 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.358+0000 7fc6afe00640 1 --2- 172.21.15.145:0/3000658454 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc6b0143450 0x7fc6b01563d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:54.358 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.358+0000 7fc6afe00640 1 -- 172.21.15.145:0/3000658454 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6b0142f30 msgr2=0x7fc6b015a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:54.358 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.358+0000 7fc6afe00640 1 --2- 172.21.15.145:0/3000658454 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6b0142f30 0x7fc6b015a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:54.358 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.358+0000 7fc6afe00640 1 -- 172.21.15.145:0/3000658454 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6b015b320 con 0x7fc6b01400b0 2024-05-09T04:34:54.358 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.358+0000 7fc6afe00640 1 --2- 172.21.15.145:0/3000658454 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc6b01400b0 0x7fc6b01f5a50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc6a000c530 tx=0x7fc6a000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:54.359 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.359+0000 7fc6a7e00640 1 -- 172.21.15.145:0/3000658454 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a00052d0 con 0x7fc6b01400b0 2024-05-09T04:34:54.359 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.359+0000 7fc6a7e00640 1 -- 172.21.15.145:0/3000658454 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6a0005470 con 0x7fc6b01400b0 2024-05-09T04:34:54.359 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.359+0000 7fc6a7e00640 1 -- 172.21.15.145:0/3000658454 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a0012870 con 0x7fc6b01400b0 2024-05-09T04:34:54.359 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.359+0000 7fc6b7200640 1 -- 172.21.15.145:0/3000658454 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6b015b600 con 0x7fc6b01400b0 2024-05-09T04:34:54.359 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.359+0000 7fc6b7200640 1 -- 172.21.15.145:0/3000658454 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6b0073c50 con 0x7fc6b01400b0 2024-05-09T04:34:54.359 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.360+0000 7fc6b7200640 1 -- 172.21.15.145:0/3000658454 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6b006a310 con 0x7fc6b01400b0 2024-05-09T04:34:54.360 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.361+0000 7fc6a7e00640 1 -- 172.21.15.145:0/3000658454 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc6a0012a10 con 0x7fc6b01400b0 2024-05-09T04:34:54.361 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.362+0000 7fc6a7e00640 1 --2- 172.21.15.145:0/3000658454 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fc68c0762d0 0x7fc68c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:54.361 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.362+0000 7fc6a7e00640 1 -- 172.21.15.145:0/3000658454 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fc6a0097180 con 0x7fc6b01400b0 2024-05-09T04:34:54.361 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.362+0000 7fc6af400640 1 --2- 172.21.15.145:0/3000658454 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fc68c0762d0 0x7fc68c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:54.361 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.363+0000 7fc6af400640 1 --2- 172.21.15.145:0/3000658454 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fc68c0762d0 0x7fc68c078790 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc6b015e2a0 tx=0x7fc698007d00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:54.365 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.367+0000 7fc6a7e00640 1 -- 172.21.15.145:0/3000658454 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc6a0060b70 con 0x7fc6b01400b0 2024-05-09T04:34:54.571 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.572+0000 7fc6b7200640 1 -- 172.21.15.145:0/3000658454 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command([{prefix=config set, name=mds_symlink_recovery}] v 0) -- 0x7fc6b00d1be0 con 0x7fc6b01400b0 2024-05-09T04:34:54.577 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.578+0000 7fc6a7e00640 1 -- 172.21.15.145:0/3000658454 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_symlink_recovery}]=0 v12)=0 v12) ==== 121+0+0 (secure 0 0 0) 0x7fc6a0060470 con 0x7fc6b01400b0 2024-05-09T04:34:54.580 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.581+0000 7fc6b7200640 1 -- 172.21.15.145:0/3000658454 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fc68c0762d0 msgr2=0x7fc68c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:54.580 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.581+0000 7fc6b7200640 1 --2- 172.21.15.145:0/3000658454 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fc68c0762d0 0x7fc68c078790 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc6b015e2a0 tx=0x7fc698007d00 comp rx=0 tx=0).stop 2024-05-09T04:34:54.580 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.581+0000 7fc6b7200640 1 -- 172.21.15.145:0/3000658454 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc6b01400b0 msgr2=0x7fc6b01f5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:54.580 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.581+0000 7fc6b7200640 1 --2- 172.21.15.145:0/3000658454 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc6b01400b0 0x7fc6b01f5a50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc6a000c530 tx=0x7fc6a000c9f0 comp rx=0 tx=0).stop 2024-05-09T04:34:54.581 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.581+0000 7fc6b7200640 1 -- 172.21.15.145:0/3000658454 shutdown_connections 2024-05-09T04:34:54.581 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.581+0000 7fc6b7200640 1 --2- 172.21.15.145:0/3000658454 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fc68c0762d0 0x7fc68c078790 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:54.581 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.581+0000 7fc6b7200640 1 --2- 172.21.15.145:0/3000658454 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc6b0143450 0x7fc6b01563d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:54.581 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.581+0000 7fc6b7200640 1 --2- 172.21.15.145:0/3000658454 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6b0142f30 0x7fc6b015a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:54.581 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.581+0000 7fc6b7200640 1 --2- 172.21.15.145:0/3000658454 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc6b01400b0 0x7fc6b01f5a50 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:54.581 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.581+0000 7fc6b7200640 1 -- 172.21.15.145:0/3000658454 >> 172.21.15.145:0/3000658454 conn(0x7fc6b006cab0 msgr2=0x7fc6b00b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:54.581 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.581+0000 7fc6b7200640 1 -- 172.21.15.145:0/3000658454 shutdown_connections 2024-05-09T04:34:54.581 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54.581+0000 7fc6b7200640 1 -- 172.21.15.145:0/3000658454 wait complete. 2024-05-09T04:34:54.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:54.599 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir sym_dir) 2024-05-09T04:34:54.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:54.655 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch sym_dir/file1) 2024-05-09T04:34:54.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:54.719 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest/mnt.0 && exec ln -s ./file1 sym_dir/symlink_file1) 2024-05-09T04:34:54.777 DEBUG:teuthology.orchestra.run.smithi145:> stdin-killer -- python3 -c ' 2024-05-09T04:34:54.777 DEBUG:teuthology.orchestra.run.smithi145:> import os 2024-05-09T04:34:54.777 DEBUG:teuthology.orchestra.run.smithi145:> import stat 2024-05-09T04:34:54.777 DEBUG:teuthology.orchestra.run.smithi145:> 2024-05-09T04:34:54.777 DEBUG:teuthology.orchestra.run.smithi145:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/sym_dir/symlink_file1").st_ino) 2024-05-09T04:34:54.777 DEBUG:teuthology.orchestra.run.smithi145:> ' 2024-05-09T04:34:54.916 INFO:teuthology.orchestra.run.smithi145.stdout:1099511628281 2024-05-09T04:34:54.921 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T04:34:54.930 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T04:34:55.176 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.177+0000 7f6cbd000640 1 Processor -- start 2024-05-09T04:34:55.176 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.177+0000 7f6cbd000640 1 -- start start 2024-05-09T04:34:55.177 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.178+0000 7f6cbd000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f6cb815fbc0 0x7f6cb8142200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:55.177 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.178+0000 7f6cbd000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f6cb815ffa0 0x7f6cb80cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:55.177 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.179+0000 7f6cbd000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cb813f340 con 0x7f6cb80cfe10 2024-05-09T04:34:55.177 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.179+0000 7f6cbd000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f6cb8072220 con 0x7f6cb815fbc0 2024-05-09T04:34:55.177 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.179+0000 7f6cbd000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f6cb80d02b0 con 0x7f6cb815ffa0 2024-05-09T04:34:55.178 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.179+0000 7f6cb5e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f6cb815fbc0 0x7f6cb8142200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:55.178 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.179+0000 7f6cb6800640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f6cb80cfe10 0x7f6cb815c5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:58348/0 (socket says 172.21.15.145:58348) 2024-05-09T04:34:55.178 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.179+0000 7f6cb6800640 1 -- 172.21.15.145:0/1066079836 learned_addr learned my addr 172.21.15.145:0/1066079836 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:34:55.178 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.179+0000 7f6cb5e00640 1 -- 172.21.15.145:0/1066079836 >> v2:172.21.15.192:3300/0 conn(0x7f6cb815ffa0 msgr2=0x7f6cb80cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:55.178 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.179+0000 7f6cb5e00640 1 --2- 172.21.15.145:0/1066079836 >> v2:172.21.15.192:3300/0 conn(0x7f6cb815ffa0 0x7f6cb80cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:55.179 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.179+0000 7f6cb5e00640 1 -- 172.21.15.145:0/1066079836 >> v1:172.21.15.145:6789/0 conn(0x7f6cb80cfe10 legacy=0x7f6cb815c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:55.179 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.179+0000 7f6cb5e00640 1 -- 172.21.15.145:0/1066079836 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cb80d0590 con 0x7f6cb815fbc0 2024-05-09T04:34:55.179 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.180+0000 7f6cb5e00640 1 --2- 172.21.15.145:0/1066079836 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6cb815fbc0 0x7f6cb8142200 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8071b90 tx=0x7f6ca802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=837a02ab70a498d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:55.179 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.181+0000 7f6cb4a00640 1 -- 172.21.15.145:0/1066079836 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca8037070 con 0x7f6cb815fbc0 2024-05-09T04:34:55.180 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.181+0000 7f6cb4a00640 1 -- 172.21.15.145:0/1066079836 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ca8032440 con 0x7f6cb815fbc0 2024-05-09T04:34:55.180 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.181+0000 7f6cb4a00640 1 -- 172.21.15.145:0/1066079836 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca802ae60 con 0x7f6cb815fbc0 2024-05-09T04:34:55.181 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.181+0000 7f6cbd000640 1 -- 172.21.15.145:0/1066079836 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6cb815fbc0 msgr2=0x7f6cb8142200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:55.181 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.181+0000 7f6cbd000640 1 --2- 172.21.15.145:0/1066079836 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6cb815fbc0 0x7f6cb8142200 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8071b90 tx=0x7f6ca802a110 comp rx=0 tx=0).stop 2024-05-09T04:34:55.181 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.182+0000 7f6cbd000640 1 -- 172.21.15.145:0/1066079836 shutdown_connections 2024-05-09T04:34:55.181 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.182+0000 7f6cbd000640 1 --2- 172.21.15.145:0/1066079836 >> v2:172.21.15.192:3300/0 conn(0x7f6cb815ffa0 0x7f6cb80cf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:55.181 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.182+0000 7f6cbd000640 1 --2- 172.21.15.145:0/1066079836 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6cb815fbc0 0x7f6cb8142200 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:55.181 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.182+0000 7f6cbd000640 1 -- 172.21.15.145:0/1066079836 >> 172.21.15.145:0/1066079836 conn(0x7f6cb806c870 msgr2=0x7f6cb806cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:55.181 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.182+0000 7f6cbd000640 1 -- 172.21.15.145:0/1066079836 shutdown_connections 2024-05-09T04:34:55.181 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.182+0000 7f6cbd000640 1 -- 172.21.15.145:0/1066079836 wait complete. 2024-05-09T04:34:55.182 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.184+0000 7f6cbd000640 1 Processor -- start 2024-05-09T04:34:55.182 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.184+0000 7f6cbd000640 1 -- start start 2024-05-09T04:34:55.184 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.185+0000 7f6cbd000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6cb80cfe10 0x7f6cb81407e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:55.184 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.185+0000 7f6cbd000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6cb815fbc0 0x7f6cb81f4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:55.184 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.185+0000 7f6cbd000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cb815ffa0 0x7f6cb81f53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:55.184 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.185+0000 7f6cbd000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb8160430 con 0x7f6cb80cfe10 2024-05-09T04:34:55.184 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.185+0000 7f6cbd000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb80d0b50 con 0x7f6cb815ffa0 2024-05-09T04:34:55.184 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.185+0000 7f6cbd000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f6cb80736c0 con 0x7f6cb815fbc0 2024-05-09T04:34:55.184 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.185+0000 7f6cb5e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6cb80cfe10 0x7f6cb81407e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:55.185 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.185+0000 7f6cb5e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6cb80cfe10 0x7f6cb81407e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:41236/0 (socket says 172.21.15.145:41236) 2024-05-09T04:34:55.185 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.185+0000 7f6cb5e00640 1 -- 172.21.15.145:0/3601697146 learned_addr learned my addr 172.21.15.145:0/3601697146 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:55.186 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.186+0000 7f6cb5e00640 1 -- 172.21.15.145:0/3601697146 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6cb815fbc0 msgr2=0x7f6cb81f4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:55.186 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.186+0000 7f6cb5e00640 1 --2- 172.21.15.145:0/3601697146 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6cb815fbc0 0x7f6cb81f4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:55.186 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.186+0000 7f6cb5e00640 1 -- 172.21.15.145:0/3601697146 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cb815ffa0 msgr2=0x7f6cb81f53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:55.186 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.186+0000 7f6cb5e00640 1 --2- 172.21.15.145:0/3601697146 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cb815ffa0 0x7f6cb81f53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:55.186 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.186+0000 7f6cb5e00640 1 -- 172.21.15.145:0/3601697146 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cb80af790 con 0x7f6cb80cfe10 2024-05-09T04:34:55.186 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.186+0000 7f6cb5e00640 1 --2- 172.21.15.145:0/3601697146 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6cb80cfe10 0x7f6cb81407e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8000c00 tx=0x7f6ca80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:55.186 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.186+0000 7f6cae000640 1 -- 172.21.15.145:0/3601697146 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca8037040 con 0x7f6cb80cfe10 2024-05-09T04:34:55.186 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.186+0000 7f6cae000640 1 -- 172.21.15.145:0/3601697146 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ca8042070 con 0x7f6cb80cfe10 2024-05-09T04:34:55.186 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.186+0000 7f6cbd000640 1 -- 172.21.15.145:0/3601697146 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cb81f5b70 con 0x7f6cb80cfe10 2024-05-09T04:34:55.186 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.186+0000 7f6cae000640 1 -- 172.21.15.145:0/3601697146 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca8030bd0 con 0x7f6cb80cfe10 2024-05-09T04:34:55.187 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.187+0000 7f6cbd000640 1 -- 172.21.15.145:0/3601697146 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cb81f6130 con 0x7f6cb80cfe10 2024-05-09T04:34:55.187 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.188+0000 7f6cbd000640 1 -- 172.21.15.145:0/3601697146 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cb806a310 con 0x7f6cb80cfe10 2024-05-09T04:34:55.187 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.189+0000 7f6cae000640 1 -- 172.21.15.145:0/3601697146 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6ca8047460 con 0x7f6cb80cfe10 2024-05-09T04:34:55.189 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.190+0000 7f6cae000640 1 --2- 172.21.15.145:0/3601697146 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f6c980762d0 0x7f6c98078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:55.189 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.191+0000 7f6cae000640 1 -- 172.21.15.145:0/3601697146 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f6ca80b8290 con 0x7f6cb80cfe10 2024-05-09T04:34:55.189 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.191+0000 7f6cb5400640 1 --2- 172.21.15.145:0/3601697146 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f6c980762d0 0x7f6c98078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:55.190 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.191+0000 7f6cb5400640 1 --2- 172.21.15.145:0/3601697146 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f6c980762d0 0x7f6c98078790 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8141180 tx=0x7f6ca4006bf0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:55.193 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.195+0000 7f6cae000640 1 -- 172.21.15.145:0/3601697146 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6ca8032e20 con 0x7f6cb80cfe10 2024-05-09T04:34:55.438 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.440+0000 7f6cbd000640 1 -- 172.21.15.145:0/3601697146 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6cb80b4070 con 0x7f6cb80cfe10 2024-05-09T04:34:55.440 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.441+0000 7f6cae000640 1 -- 172.21.15.145:0/3601697146 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5187 (secure 0 0 0) 0x7f6ca8081be0 con 0x7f6cb80cfe10 2024-05-09T04:34:55.440 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:34:55.440 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4499,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6834/3227728059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3227728059},{"type":"v1","addr":"172.21.15.192:6834","nonce":3227728059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4502,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1603013787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6833","nonce":1603013787},{"type":"v1","addr":"172.21.15.192:6835","nonce":1603013787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4672,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/68778418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":68778418},{"type":"v1","addr":"172.21.15.145:6837","nonce":68778418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T04:34:43.133970+0000","modified":"2024-05-09T04:34:46.181657+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4761},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4761":{"gid":4761,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.145:6835/4260798843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4260798843},{"type":"v1","addr":"172.21.15.145:6835","nonce":4260798843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4761,"qdb_cluster":[4761]},"id":2}]} 2024-05-09T04:34:55.440 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 17 2024-05-09T04:34:55.443 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.443+0000 7f6cbd000640 1 -- 172.21.15.145:0/3601697146 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f6c980762d0 msgr2=0x7f6c98078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:55.443 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.443+0000 7f6cbd000640 1 --2- 172.21.15.145:0/3601697146 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f6c980762d0 0x7f6c98078790 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8141180 tx=0x7f6ca4006bf0 comp rx=0 tx=0).stop 2024-05-09T04:34:55.443 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.443+0000 7f6cbd000640 1 -- 172.21.15.145:0/3601697146 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6cb80cfe10 msgr2=0x7f6cb81407e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:55.443 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.443+0000 7f6cbd000640 1 --2- 172.21.15.145:0/3601697146 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6cb80cfe10 0x7f6cb81407e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8000c00 tx=0x7f6ca80046c0 comp rx=0 tx=0).stop 2024-05-09T04:34:55.443 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.443+0000 7f6cbd000640 1 -- 172.21.15.145:0/3601697146 shutdown_connections 2024-05-09T04:34:55.443 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.443+0000 7f6cbd000640 1 --2- 172.21.15.145:0/3601697146 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f6c980762d0 0x7f6c98078790 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:55.443 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.443+0000 7f6cbd000640 1 --2- 172.21.15.145:0/3601697146 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cb815ffa0 0x7f6cb81f53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:55.443 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.443+0000 7f6cbd000640 1 --2- 172.21.15.145:0/3601697146 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6cb815fbc0 0x7f6cb81f4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:55.443 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.443+0000 7f6cbd000640 1 --2- 172.21.15.145:0/3601697146 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6cb80cfe10 0x7f6cb81407e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:55.443 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.443+0000 7f6cbd000640 1 -- 172.21.15.145:0/3601697146 >> 172.21.15.145:0/3601697146 conn(0x7f6cb806c870 msgr2=0x7f6cb815b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:55.443 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.444+0000 7f6cbd000640 1 -- 172.21.15.145:0/3601697146 shutdown_connections 2024-05-09T04:34:55.443 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.444+0000 7f6cbd000640 1 -- 172.21.15.145:0/3601697146 wait complete. 2024-05-09T04:34:55.462 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-05-09T04:34:55.702 INFO:teuthology.orchestra.run.smithi145.stdout:{"message":"","return_code":0} 2024-05-09T04:34:55.702 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-09T04:34:55.702 DEBUG:teuthology.orchestra.run.smithi145:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f9.00000000 symlink 2024-05-09T04:34:55.797 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.798+0000 7f3186b23f40 1 Processor -- start 2024-05-09T04:34:55.798 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.799+0000 7f3186b23f40 1 -- start start 2024-05-09T04:34:55.800 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.799+0000 7f3186b23f40 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5591fee3ab90 0x5591feebc190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:55.800 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.799+0000 7f3186b23f40 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5591fee3b4b0 0x5591feebe8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:55.800 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.799+0000 7f3186b23f40 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5591feec00d0 0x5591feebee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:55.800 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.799+0000 7f3186b23f40 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5591feec70d0 con 0x5591fee3ab90 2024-05-09T04:34:55.800 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.799+0000 7f3186b23f40 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5591fee39f10 con 0x5591feec00d0 2024-05-09T04:34:55.800 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.799+0000 7f3186b23f40 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x5591fee3ba30 con 0x5591fee3b4b0 2024-05-09T04:34:55.800 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.800+0000 7f3183200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5591fee3ab90 0x5591feebc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:55.801 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.800+0000 7f3183200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5591fee3ab90 0x5591feebc190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:41258/0 (socket says 172.21.15.145:41258) 2024-05-09T04:34:55.801 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.800+0000 7f3183200640 1 -- 172.21.15.145:0/2059190220 learned_addr learned my addr 172.21.15.145:0/2059190220 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:55.801 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.800+0000 7f317a800640 1 --2- 172.21.15.145:0/2059190220 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5591fee3b4b0 0x5591feebe8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:55.801 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.800+0000 7f3183200640 1 -- 172.21.15.145:0/2059190220 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5591fee3b4b0 msgr2=0x5591feebe8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:55.801 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.800+0000 7f3183200640 1 --2- 172.21.15.145:0/2059190220 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5591fee3b4b0 0x5591feebe8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:55.801 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.800+0000 7f3183c00640 1 --2- 172.21.15.145:0/2059190220 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5591feec00d0 0x5591feebee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:55.801 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.800+0000 7f3183200640 1 -- 172.21.15.145:0/2059190220 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5591feec00d0 msgr2=0x5591feebee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:55.801 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.800+0000 7f3183200640 1 --2- 172.21.15.145:0/2059190220 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5591feec00d0 0x5591feebee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:55.801 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.800+0000 7f3183200640 1 -- 172.21.15.145:0/2059190220 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591fed3e300 con 0x5591fee3ab90 2024-05-09T04:34:55.801 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.800+0000 7f3183200640 1 --2- 172.21.15.145:0/2059190220 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5591fee3ab90 0x5591feebc190 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f317400c4b0 tx=0x7f317400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=756a83bdf2360f9b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:55.801 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.801+0000 7f3181e00640 1 -- 172.21.15.145:0/2059190220 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31740116b0 con 0x5591fee3ab90 2024-05-09T04:34:55.801 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.801+0000 7f3181e00640 1 -- 172.21.15.145:0/2059190220 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3174017070 con 0x5591fee3ab90 2024-05-09T04:34:55.801 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.801+0000 7f3181e00640 1 -- 172.21.15.145:0/2059190220 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3174011a90 con 0x5591fee3ab90 2024-05-09T04:34:55.801 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.801+0000 7f3186b23f40 1 -- 172.21.15.145:0/2059190220 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5591fee3ab90 msgr2=0x5591feebc190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:55.802 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.801+0000 7f3186b23f40 1 --2- 172.21.15.145:0/2059190220 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5591fee3ab90 0x5591feebc190 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f317400c4b0 tx=0x7f317400c970 comp rx=0 tx=0).stop 2024-05-09T04:34:55.802 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.801+0000 7f3186b23f40 1 -- 172.21.15.145:0/2059190220 shutdown_connections 2024-05-09T04:34:55.802 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.801+0000 7f3186b23f40 1 --2- 172.21.15.145:0/2059190220 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5591feec00d0 0x5591feebee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:55.802 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.801+0000 7f3186b23f40 1 --2- 172.21.15.145:0/2059190220 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5591fee3b4b0 0x5591feebe8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:55.802 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.801+0000 7f3186b23f40 1 --2- 172.21.15.145:0/2059190220 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5591fee3ab90 0x5591feebc190 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:55.802 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.801+0000 7f3186b23f40 1 -- 172.21.15.145:0/2059190220 >> 172.21.15.145:0/2059190220 conn(0x5591fee1dc80 msgr2=0x5591fee1f570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:55.802 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.801+0000 7f3186b23f40 1 -- 172.21.15.145:0/2059190220 shutdown_connections 2024-05-09T04:34:55.802 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.801+0000 7f3186b23f40 1 -- 172.21.15.145:0/2059190220 wait complete. 2024-05-09T04:34:55.804 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.802+0000 7f3186b23f40 1 Processor -- start 2024-05-09T04:34:55.804 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.802+0000 7f3186b23f40 1 -- start start 2024-05-09T04:34:55.804 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.803+0000 7f3186b23f40 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5591fee3ab90 0x5591feeb5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:55.804 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.803+0000 7f3186b23f40 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5591fee3b4b0 0x5591feeb4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:55.804 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.803+0000 7f3186b23f40 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5591feec00d0 0x5591feebd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:55.804 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.803+0000 7f3186b23f40 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5591feebf370 con 0x5591feec00d0 2024-05-09T04:34:55.804 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.803+0000 7f3186b23f40 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5591feeb5370 con 0x5591fee3ab90 2024-05-09T04:34:55.804 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.803+0000 7f3186b23f40 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x5591fee1e2a0 con 0x5591fee3b4b0 2024-05-09T04:34:55.805 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.803+0000 7f3183c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5591feec00d0 0x5591feebd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:55.805 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.803+0000 7f3183c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5591feec00d0 0x5591feebd4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:41274/0 (socket says 172.21.15.145:41274) 2024-05-09T04:34:55.805 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.803+0000 7f3183c00640 1 -- 172.21.15.145:0/713614865 learned_addr learned my addr 172.21.15.145:0/713614865 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:55.805 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.804+0000 7f3183200640 1 --2- 172.21.15.145:0/713614865 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5591fee3ab90 0x5591feeb5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:55.805 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.804+0000 7f317a800640 1 --2- 172.21.15.145:0/713614865 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5591fee3b4b0 0x5591feeb4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:55.805 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.804+0000 7f3183c00640 1 -- 172.21.15.145:0/713614865 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5591fee3b4b0 msgr2=0x5591feeb4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:55.805 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.804+0000 7f3183c00640 1 --2- 172.21.15.145:0/713614865 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5591fee3b4b0 0x5591feeb4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:55.805 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.804+0000 7f3183c00640 1 -- 172.21.15.145:0/713614865 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5591fee3ab90 msgr2=0x5591feeb5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:55.805 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.804+0000 7f3183c00640 1 --2- 172.21.15.145:0/713614865 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5591fee3ab90 0x5591feeb5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:55.805 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.804+0000 7f3183c00640 1 -- 172.21.15.145:0/713614865 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591fed18b30 con 0x5591feec00d0 2024-05-09T04:34:55.805 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.804+0000 7f317a800640 1 --2- 172.21.15.145:0/713614865 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5591fee3b4b0 0x5591feeb4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:34:55.805 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.804+0000 7f3183c00640 1 --2- 172.21.15.145:0/713614865 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5591feec00d0 0x5591feebd4a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f317c025bf0 tx=0x7f317c002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:55.805 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.804+0000 7f3180a00640 1 -- 172.21.15.145:0/713614865 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f317c064020 con 0x5591feec00d0 2024-05-09T04:34:55.805 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.805+0000 7f3180a00640 1 -- 172.21.15.145:0/713614865 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f317c054930 con 0x5591feec00d0 2024-05-09T04:34:55.805 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.805+0000 7f3186b23f40 1 -- 172.21.15.145:0/713614865 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5591feec2fa0 con 0x5591feec00d0 2024-05-09T04:34:55.805 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.805+0000 7f3180a00640 1 -- 172.21.15.145:0/713614865 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f317c059030 con 0x5591feec00d0 2024-05-09T04:34:55.805 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.805+0000 7f3186b23f40 1 -- 172.21.15.145:0/713614865 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591feec11e0 con 0x5591feec00d0 2024-05-09T04:34:55.806 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.807+0000 7f3180a00640 1 -- 172.21.15.145:0/713614865 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f317c067020 con 0x5591feec00d0 2024-05-09T04:34:55.807 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.808+0000 7f3180a00640 1 --2- 172.21.15.145:0/713614865 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f3160074cb0 0x7f3160077170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:55.807 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.808+0000 7f3180a00640 1 -- 172.21.15.145:0/713614865 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f317c0d9c50 con 0x5591feec00d0 2024-05-09T04:34:55.807 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.808+0000 7f3183200640 1 --2- 172.21.15.145:0/713614865 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f3160074cb0 0x7f3160077170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:55.807 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.808+0000 7f3186b23f40 1 --2- 172.21.15.145:0/713614865 >> [v2:172.21.15.145:6824/1845225555,v1:172.21.15.145:6825/1845225555] conn(0x5591feec29f0 0x5591fef67aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:55.807 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.808+0000 7f3186b23f40 1 -- 172.21.15.145:0/713614865 --> [v2:172.21.15.145:6824/1845225555,v1:172.21.15.145:6825/1845225555] -- osd_op(unknown.0.0:1 5.36 5:6ee5e2c4:::100000001f9.00000000:head [getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5591fef67fe0 con 0x5591feec29f0 2024-05-09T04:34:55.808 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.808+0000 7f317a800640 1 --2- 172.21.15.145:0/713614865 >> [v2:172.21.15.145:6824/1845225555,v1:172.21.15.145:6825/1845225555] conn(0x5591feec29f0 0x5591fef67aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:55.808 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.809+0000 7f3183200640 1 --2- 172.21.15.145:0/713614865 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f3160074cb0 0x7f3160077170 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f317400c4b0 tx=0x7f3174011000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:55.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.809+0000 7f317a800640 1 --2- 172.21.15.145:0/713614865 >> [v2:172.21.15.145:6824/1845225555,v1:172.21.15.145:6825/1845225555] conn(0x5591feec29f0 0x5591fef67aa0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:55.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.809+0000 7f317a800640 1 -- 172.21.15.145:0/713614865 <== osd.1 v2:172.21.15.145:6824/1845225555 1 ==== osd_op_reply(1 100000001f9.00000000 [getxattr (11) out=11b] v0'0 uv1 ondisk = 0) ==== 164+0+11 (crc 0 0 0) 0x7f316c007670 con 0x5591feec29f0 2024-05-09T04:34:55.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.809+0000 7f3186b23f40 1 -- 172.21.15.145:0/713614865 >> [v2:172.21.15.145:6824/1845225555,v1:172.21.15.145:6825/1845225555] conn(0x5591feec29f0 msgr2=0x5591fef67aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:55.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.809+0000 7f3186b23f40 1 --2- 172.21.15.145:0/713614865 >> [v2:172.21.15.145:6824/1845225555,v1:172.21.15.145:6825/1845225555] conn(0x5591feec29f0 0x5591fef67aa0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:55.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.810+0000 7f3186b23f40 1 -- 172.21.15.145:0/713614865 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f3160074cb0 msgr2=0x7f3160077170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:55.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.810+0000 7f3186b23f40 1 --2- 172.21.15.145:0/713614865 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f3160074cb0 0x7f3160077170 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f317400c4b0 tx=0x7f3174011000 comp rx=0 tx=0).stop 2024-05-09T04:34:55.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.810+0000 7f3186b23f40 1 -- 172.21.15.145:0/713614865 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5591feec00d0 msgr2=0x5591feebd4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:55.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.810+0000 7f3186b23f40 1 --2- 172.21.15.145:0/713614865 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5591feec00d0 0x5591feebd4a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f317c025bf0 tx=0x7f317c002fb0 comp rx=0 tx=0).stop 2024-05-09T04:34:55.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.810+0000 7f3183c00640 1 -- 172.21.15.145:0/713614865 reap_dead start 2024-05-09T04:34:55.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.810+0000 7f3186b23f40 1 -- 172.21.15.145:0/713614865 shutdown_connections 2024-05-09T04:34:55.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.810+0000 7f3186b23f40 1 -- 172.21.15.145:0/713614865 >> 172.21.15.145:0/713614865 conn(0x5591fee1dc80 msgr2=0x5591feec5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:55.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.810+0000 7f3186b23f40 1 -- 172.21.15.145:0/713614865 shutdown_connections 2024-05-09T04:34:55.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:55.810+0000 7f3186b23f40 1 -- 172.21.15.145:0/713614865 wait complete. 2024-05-09T04:34:55.813 INFO:teuthology.orchestra.run.smithi145.stdout:./file1 2024-05-09T04:34:55.813 DEBUG:teuthology.orchestra.run.smithi145:> ceph-dencoder type string_wrapper import - decode dump_json 2024-05-09T04:34:55.956 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-05-09T04:34:55.956 INFO:teuthology.orchestra.run.smithi145.stdout: "s": "./file1" 2024-05-09T04:34:55.956 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-05-09T04:34:55.956 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:34:55.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:55.966 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent_a) 2024-05-09T04:34:56.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:56.023 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parent_a/alpha) 2024-05-09T04:34:56.083 DEBUG:teuthology.orchestra.run.smithi145:> stdin-killer -- python3 -c ' 2024-05-09T04:34:56.083 DEBUG:teuthology.orchestra.run.smithi145:> import os 2024-05-09T04:34:56.084 DEBUG:teuthology.orchestra.run.smithi145:> import stat 2024-05-09T04:34:56.084 DEBUG:teuthology.orchestra.run.smithi145:> 2024-05-09T04:34:56.084 DEBUG:teuthology.orchestra.run.smithi145:> print(os.stat("/home/ubuntu/cephtest/mnt.0/parent_a/alpha").st_ino) 2024-05-09T04:34:56.084 DEBUG:teuthology.orchestra.run.smithi145:> ' 2024-05-09T04:34:56.212 INFO:teuthology.orchestra.run.smithi145.stdout:1099511628283 2024-05-09T04:34:56.216 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T04:34:56.225 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T04:34:56.462 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.463+0000 7f8056200640 1 Processor -- start 2024-05-09T04:34:56.462 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.463+0000 7f8056200640 1 -- start start 2024-05-09T04:34:56.463 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.464+0000 7f8056200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f805006e860 0x7f805006ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:56.463 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.464+0000 7f8056200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f805015d780 0x7f8050160c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:56.463 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.464+0000 7f8056200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f805006f180 con 0x7f80500ce620 2024-05-09T04:34:56.463 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.464+0000 7f8056200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f80501615c0 con 0x7f805015d780 2024-05-09T04:34:56.463 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.464+0000 7f8056200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f80501618a0 con 0x7f805006e860 2024-05-09T04:34:56.465 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.465+0000 7f804fe00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f80500ce620 0x7f80500ceab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:58354/0 (socket says 172.21.15.145:58354) 2024-05-09T04:34:56.465 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.465+0000 7f804fe00640 1 -- 172.21.15.145:0/1646076643 learned_addr learned my addr 172.21.15.145:0/1646076643 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:34:56.466 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.465+0000 7f804f400640 1 --2- 172.21.15.145:0/1646076643 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f805006e860 0x7f805006ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:56.466 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.465+0000 7f804ea00640 1 --2- 172.21.15.145:0/1646076643 >> v2:172.21.15.192:3300/0 conn(0x7f805015d780 0x7f8050160c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:56.466 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.465+0000 7f804e000640 1 -- 172.21.15.145:0/1646076643 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1258454957 0 0) 0x7f805006f180 con 0x7f80500ce620 2024-05-09T04:34:56.466 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.465+0000 7f804e000640 1 -- 172.21.15.145:0/1646076643 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8034003520 con 0x7f80500ce620 2024-05-09T04:34:56.466 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.466+0000 7f804e000640 1 -- 172.21.15.145:0/1646076643 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 411305128 0 0) 0x7f8034003520 con 0x7f80500ce620 2024-05-09T04:34:56.466 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.466+0000 7f804e000640 1 -- 172.21.15.145:0/1646076643 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f805006e860 msgr2=0x7f805006ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:56.466 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.466+0000 7f804e000640 1 --2- 172.21.15.145:0/1646076643 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f805006e860 0x7f805006ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:56.466 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.466+0000 7f804e000640 1 -- 172.21.15.145:0/1646076643 >> v2:172.21.15.192:3300/0 conn(0x7f805015d780 msgr2=0x7f8050160c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:56.466 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.466+0000 7f804e000640 1 --2- 172.21.15.145:0/1646076643 >> v2:172.21.15.192:3300/0 conn(0x7f805015d780 0x7f8050160c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:56.466 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.466+0000 7f804e000640 1 -- 172.21.15.145:0/1646076643 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80500b4110 con 0x7f80500ce620 2024-05-09T04:34:56.466 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.466+0000 7f804e000640 1 -- 172.21.15.145:0/1646076643 <== mon.0 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3474413261 0 0) 0x7f8048001a70 con 0x7f80500ce620 2024-05-09T04:34:56.466 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.466+0000 7f804e000640 1 -- 172.21.15.145:0/1646076643 >> v1:172.21.15.145:6789/0 conn(0x7f80500ce620 legacy=0x7f80500ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:56.466 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.466+0000 7f804e000640 1 --2- 172.21.15.145:0/1646076643 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8034003a00 0x7f8034003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:56.466 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.466+0000 7f804e000640 1 --2- 172.21.15.145:0/1646076643 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8034004800 0x7f8034004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:56.467 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.466+0000 7f804e000640 1 --2- 172.21.15.145:0/1646076643 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f805006e860 0x7f80340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:56.467 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.466+0000 7f804e000640 1 -- 172.21.15.145:0/1646076643 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f80501615c0 con 0x7f8034003a00 2024-05-09T04:34:56.467 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.466+0000 7f804e000640 1 -- 172.21.15.145:0/1646076643 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f80501618a0 con 0x7f8034004800 2024-05-09T04:34:56.467 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.466+0000 7f804e000640 1 -- 172.21.15.145:0/1646076643 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f803400ff30 con 0x7f805006e860 2024-05-09T04:34:56.467 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.466+0000 7f804f400640 1 --2- 172.21.15.145:0/1646076643 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8034003a00 0x7f8034003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:56.467 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.466+0000 7f804fe00640 1 --2- 172.21.15.145:0/1646076643 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f805006e860 0x7f80340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:56.467 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.466+0000 7f804ea00640 1 --2- 172.21.15.145:0/1646076643 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8034004800 0x7f8034004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:56.467 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.466+0000 7f804f400640 1 -- 172.21.15.145:0/1646076643 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f805006e860 msgr2=0x7f80340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:56.467 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.466+0000 7f804f400640 1 --2- 172.21.15.145:0/1646076643 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f805006e860 0x7f80340051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:56.467 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.466+0000 7f804f400640 1 -- 172.21.15.145:0/1646076643 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8034004800 msgr2=0x7f8034004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:56.467 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.467+0000 7f804f400640 1 --2- 172.21.15.145:0/1646076643 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8034004800 0x7f8034004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:56.467 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.467+0000 7f804f400640 1 -- 172.21.15.145:0/1646076643 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8034003520 con 0x7f8034003a00 2024-05-09T04:34:56.468 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.468+0000 7f804f400640 1 --2- 172.21.15.145:0/1646076643 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8034003a00 0x7f8034003e20 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f804000ba00 tx=0x7f8034005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c677f340c03b600 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:56.468 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.468+0000 7f804e000640 1 -- 172.21.15.145:0/1646076643 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8040030070 con 0x7f8034003a00 2024-05-09T04:34:56.469 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.468+0000 7f804e000640 1 -- 172.21.15.145:0/1646076643 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f804002ad80 con 0x7f8034003a00 2024-05-09T04:34:56.469 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.468+0000 7f804e000640 1 -- 172.21.15.145:0/1646076643 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f804000b600 con 0x7f8034003a00 2024-05-09T04:34:56.469 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.468+0000 7f8056200640 1 -- 172.21.15.145:0/1646076643 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8034003a00 msgr2=0x7f8034003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:56.469 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.468+0000 7f8056200640 1 --2- 172.21.15.145:0/1646076643 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8034003a00 0x7f8034003e20 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f804000ba00 tx=0x7f8034005830 comp rx=0 tx=0).stop 2024-05-09T04:34:56.469 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.468+0000 7f804fe00640 1 -- 172.21.15.145:0/1646076643 reap_dead start 2024-05-09T04:34:56.469 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.469+0000 7f8056200640 1 -- 172.21.15.145:0/1646076643 shutdown_connections 2024-05-09T04:34:56.469 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.469+0000 7f8056200640 1 -- 172.21.15.145:0/1646076643 >> 172.21.15.145:0/1646076643 conn(0x7f805006c870 msgr2=0x7f805006cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:56.469 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.469+0000 7f8056200640 1 -- 172.21.15.145:0/1646076643 shutdown_connections 2024-05-09T04:34:56.469 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.469+0000 7f8056200640 1 -- 172.21.15.145:0/1646076643 wait complete. 2024-05-09T04:34:56.469 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.470+0000 7f8056200640 1 Processor -- start 2024-05-09T04:34:56.470 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.471+0000 7f8056200640 1 -- start start 2024-05-09T04:34:56.471 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.471+0000 7f8056200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8034003a00 0x7f80501446d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:56.471 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.472+0000 7f8056200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f80500736b0 0x7f805015b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:56.472 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.472+0000 7f8056200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f805015c470 0x7f805015be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:56.472 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.472+0000 7f8056200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8050146d80 con 0x7f80500736b0 2024-05-09T04:34:56.472 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.472+0000 7f8056200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8050146c00 con 0x7f805015c470 2024-05-09T04:34:56.472 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.472+0000 7f8056200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8050161190 con 0x7f8034003a00 2024-05-09T04:34:56.472 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.472+0000 7f804ea00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f80500736b0 0x7f805015b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:56.472 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.472+0000 7f804ea00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f80500736b0 0x7f805015b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:41300/0 (socket says 172.21.15.145:41300) 2024-05-09T04:34:56.472 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.472+0000 7f804ea00640 1 -- 172.21.15.145:0/713254791 learned_addr learned my addr 172.21.15.145:0/713254791 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:56.472 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.472+0000 7f804ea00640 1 -- 172.21.15.145:0/713254791 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8034003a00 msgr2=0x7f80501446d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:56.472 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.472+0000 7f804ea00640 1 --2- 172.21.15.145:0/713254791 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8034003a00 0x7f80501446d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:56.472 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.472+0000 7f804ea00640 1 -- 172.21.15.145:0/713254791 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f805015c470 msgr2=0x7f805015be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:56.472 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.472+0000 7f804ea00640 1 --2- 172.21.15.145:0/713254791 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f805015c470 0x7f805015be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:56.472 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.472+0000 7f804ea00640 1 -- 172.21.15.145:0/713254791 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8050074450 con 0x7f80500736b0 2024-05-09T04:34:56.472 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.473+0000 7f804ea00640 1 --2- 172.21.15.145:0/713254791 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f80500736b0 0x7f805015b8c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f803c00e870 tx=0x7f803c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:56.472 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.473+0000 7f8047400640 1 -- 172.21.15.145:0/713254791 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803c017070 con 0x7f80500736b0 2024-05-09T04:34:56.472 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.473+0000 7f8047400640 1 -- 172.21.15.145:0/713254791 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f803c00c8d0 con 0x7f80500736b0 2024-05-09T04:34:56.473 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.473+0000 7f8047400640 1 -- 172.21.15.145:0/713254791 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803c00f040 con 0x7f80500736b0 2024-05-09T04:34:56.473 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.473+0000 7f8056200640 1 -- 172.21.15.145:0/713254791 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8050073bb0 con 0x7f80500736b0 2024-05-09T04:34:56.473 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.473+0000 7f8056200640 1 -- 172.21.15.145:0/713254791 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8050074170 con 0x7f80500736b0 2024-05-09T04:34:56.473 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.474+0000 7f8056200640 1 -- 172.21.15.145:0/713254791 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f805006a310 con 0x7f80500736b0 2024-05-09T04:34:56.474 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.475+0000 7f8047400640 1 -- 172.21.15.145:0/713254791 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f803c01a070 con 0x7f80500736b0 2024-05-09T04:34:56.475 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.476+0000 7f8047400640 1 --2- 172.21.15.145:0/713254791 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f80300762d0 0x7f8030078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:56.475 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.476+0000 7f8047400640 1 -- 172.21.15.145:0/713254791 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f803c096ff0 con 0x7f80500736b0 2024-05-09T04:34:56.475 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.477+0000 7f804f400640 1 --2- 172.21.15.145:0/713254791 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f80300762d0 0x7f8030078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:56.476 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.477+0000 7f804f400640 1 --2- 172.21.15.145:0/713254791 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f80300762d0 0x7f8030078790 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f80340057e0 tx=0x7f804002c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:56.479 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.481+0000 7f8047400640 1 -- 172.21.15.145:0/713254791 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f803c0609e0 con 0x7f80500736b0 2024-05-09T04:34:56.728 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.730+0000 7f8056200640 1 -- 172.21.15.145:0/713254791 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8050142c30 con 0x7f80500736b0 2024-05-09T04:34:56.729 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.731+0000 7f8047400640 1 -- 172.21.15.145:0/713254791 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5187 (secure 0 0 0) 0x7f803c0602e0 con 0x7f80500736b0 2024-05-09T04:34:56.730 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:34:56.730 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4499,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6834/3227728059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3227728059},{"type":"v1","addr":"172.21.15.192:6834","nonce":3227728059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4502,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1603013787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6833","nonce":1603013787},{"type":"v1","addr":"172.21.15.192:6835","nonce":1603013787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4672,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/68778418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":68778418},{"type":"v1","addr":"172.21.15.145:6837","nonce":68778418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T04:34:43.133970+0000","modified":"2024-05-09T04:34:46.181657+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4761},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4761":{"gid":4761,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.145:6835/4260798843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4260798843},{"type":"v1","addr":"172.21.15.145:6835","nonce":4260798843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4761,"qdb_cluster":[4761]},"id":2}]} 2024-05-09T04:34:56.730 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 17 2024-05-09T04:34:56.733 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.734+0000 7f8056200640 1 -- 172.21.15.145:0/713254791 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f80300762d0 msgr2=0x7f8030078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:56.733 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.734+0000 7f8056200640 1 --2- 172.21.15.145:0/713254791 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f80300762d0 0x7f8030078790 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f80340057e0 tx=0x7f804002c000 comp rx=0 tx=0).stop 2024-05-09T04:34:56.733 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.734+0000 7f8056200640 1 -- 172.21.15.145:0/713254791 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f80500736b0 msgr2=0x7f805015b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:56.733 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.734+0000 7f8056200640 1 --2- 172.21.15.145:0/713254791 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f80500736b0 0x7f805015b8c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f803c00e870 tx=0x7f803c00ed30 comp rx=0 tx=0).stop 2024-05-09T04:34:56.733 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.734+0000 7f8056200640 1 -- 172.21.15.145:0/713254791 shutdown_connections 2024-05-09T04:34:56.733 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.734+0000 7f8056200640 1 --2- 172.21.15.145:0/713254791 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f80300762d0 0x7f8030078790 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:56.733 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.734+0000 7f8056200640 1 --2- 172.21.15.145:0/713254791 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f805015c470 0x7f805015be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:56.733 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.734+0000 7f8056200640 1 --2- 172.21.15.145:0/713254791 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f80500736b0 0x7f805015b8c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:56.734 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.734+0000 7f8056200640 1 --2- 172.21.15.145:0/713254791 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8034003a00 0x7f80501446d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:56.734 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.734+0000 7f8056200640 1 -- 172.21.15.145:0/713254791 >> 172.21.15.145:0/713254791 conn(0x7f805006c870 msgr2=0x7f80500b58a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:56.734 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.734+0000 7f8056200640 1 -- 172.21.15.145:0/713254791 shutdown_connections 2024-05-09T04:34:56.734 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:56.734+0000 7f8056200640 1 -- 172.21.15.145:0/713254791 wait complete. 2024-05-09T04:34:56.752 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-05-09T04:34:56.996 INFO:teuthology.orchestra.run.smithi145.stdout:{"message":"","return_code":0} 2024-05-09T04:34:56.996 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-09T04:34:56.997 DEBUG:teuthology.orchestra.run.smithi145:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001fb.00000000 parent 2024-05-09T04:34:57.102 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.103+0000 7fa31af21f40 1 Processor -- start 2024-05-09T04:34:57.103 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.104+0000 7fa31af21f40 1 -- start start 2024-05-09T04:34:57.104 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.104+0000 7fa31af21f40 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa2fc003630 0x56012ea9a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:57.105 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.104+0000 7fa31af21f40 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56012eaa3ca0 0x56012ea047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:57.105 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.104+0000 7fa31af21f40 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56012eaa2590 0x56012ea04d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:57.105 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.104+0000 7fa31af21f40 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56012e97af50 con 0x56012eaa3ca0 2024-05-09T04:34:57.105 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.104+0000 7fa31af21f40 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x56012e97ef90 con 0x56012eaa2590 2024-05-09T04:34:57.105 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.104+0000 7fa31af21f40 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x56012ea1ef10 con 0x7fa2fc003630 2024-05-09T04:34:57.105 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.105+0000 7fa30ec00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56012eaa3ca0 0x56012ea047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:57.105 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.105+0000 7fa30ec00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56012eaa3ca0 0x56012ea047d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:41318/0 (socket says 172.21.15.145:41318) 2024-05-09T04:34:57.105 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.105+0000 7fa30ec00640 1 -- 172.21.15.145:0/1843246622 learned_addr learned my addr 172.21.15.145:0/1843246622 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:57.106 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.105+0000 7fa30ec00640 1 -- 172.21.15.145:0/1843246622 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa2fc003630 msgr2=0x56012ea9a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:57.106 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.105+0000 7fa30ec00640 1 --2- 172.21.15.145:0/1843246622 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa2fc003630 0x56012ea9a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:57.106 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.105+0000 7fa30ec00640 1 -- 172.21.15.145:0/1843246622 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56012eaa2590 msgr2=0x56012ea04d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:57.106 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.105+0000 7fa318000640 1 --2- 172.21.15.145:0/1843246622 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56012eaa2590 0x56012ea04d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:57.106 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.105+0000 7fa30ec00640 1 --2- 172.21.15.145:0/1843246622 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56012eaa2590 0x56012ea04d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:57.106 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.105+0000 7fa30ec00640 1 -- 172.21.15.145:0/1843246622 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56012eaa2190 con 0x56012eaa3ca0 2024-05-09T04:34:57.106 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.106+0000 7fa30ec00640 1 --2- 172.21.15.145:0/1843246622 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56012eaa3ca0 0x56012ea047d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa30000d340 tx=0x7fa30000d800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd9d0fcf2e80902b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:57.106 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.106+0000 7fa316200640 1 -- 172.21.15.145:0/1843246622 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa300003110 con 0x56012eaa3ca0 2024-05-09T04:34:57.106 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.106+0000 7fa316200640 1 -- 172.21.15.145:0/1843246622 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3000032b0 con 0x56012eaa3ca0 2024-05-09T04:34:57.106 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.106+0000 7fa316200640 1 -- 172.21.15.145:0/1843246622 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa300003480 con 0x56012eaa3ca0 2024-05-09T04:34:57.109 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.106+0000 7fa31af21f40 1 -- 172.21.15.145:0/1843246622 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56012eaa3ca0 msgr2=0x56012ea047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:57.109 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.106+0000 7fa31af21f40 1 --2- 172.21.15.145:0/1843246622 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56012eaa3ca0 0x56012ea047d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa30000d340 tx=0x7fa30000d800 comp rx=0 tx=0).stop 2024-05-09T04:34:57.109 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.107+0000 7fa31af21f40 1 -- 172.21.15.145:0/1843246622 shutdown_connections 2024-05-09T04:34:57.109 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.107+0000 7fa31af21f40 1 --2- 172.21.15.145:0/1843246622 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56012eaa2590 0x56012ea04d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:57.109 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.107+0000 7fa31af21f40 1 --2- 172.21.15.145:0/1843246622 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56012eaa3ca0 0x56012ea047d0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:57.109 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.107+0000 7fa31af21f40 1 --2- 172.21.15.145:0/1843246622 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa2fc003630 0x56012ea9a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:57.109 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.107+0000 7fa31af21f40 1 -- 172.21.15.145:0/1843246622 >> 172.21.15.145:0/1843246622 conn(0x56012ea02c80 msgr2=0x56012eaad170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:57.109 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.107+0000 7fa31af21f40 1 -- 172.21.15.145:0/1843246622 shutdown_connections 2024-05-09T04:34:57.109 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.107+0000 7fa31af21f40 1 -- 172.21.15.145:0/1843246622 wait complete. 2024-05-09T04:34:57.109 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.107+0000 7fa31af21f40 1 Processor -- start 2024-05-09T04:34:57.109 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.108+0000 7fa31af21f40 1 -- start start 2024-05-09T04:34:57.109 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.108+0000 7fa31af21f40 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56012eaa2590 0x56012eaad9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:57.109 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.108+0000 7fa31af21f40 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56012eaa3ca0 0x56012eaa6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:57.110 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.108+0000 7fa31af21f40 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa2fc003630 0x56012ea96870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:57.110 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.108+0000 7fa31af21f40 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56012ea02970 con 0x56012eaa2590 2024-05-09T04:34:57.110 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.108+0000 7fa31af21f40 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x56012ea04370 con 0x56012eaa3ca0 2024-05-09T04:34:57.110 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.108+0000 7fa31af21f40 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x56012eaa6670 con 0x7fa2fc003630 2024-05-09T04:34:57.110 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.109+0000 7fa317600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56012eaa2590 0x56012eaad9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:57.110 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.109+0000 7fa317600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56012eaa2590 0x56012eaad9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:41328/0 (socket says 172.21.15.145:41328) 2024-05-09T04:34:57.110 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.109+0000 7fa317600640 1 -- 172.21.15.145:0/2459219688 learned_addr learned my addr 172.21.15.145:0/2459219688 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:57.110 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.109+0000 7fa317600640 1 -- 172.21.15.145:0/2459219688 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa2fc003630 msgr2=0x56012ea96870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:57.110 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.109+0000 7fa317600640 1 --2- 172.21.15.145:0/2459219688 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa2fc003630 0x56012ea96870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:57.110 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.109+0000 7fa317600640 1 -- 172.21.15.145:0/2459219688 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56012eaa3ca0 msgr2=0x56012eaa6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:57.110 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.109+0000 7fa30ec00640 1 --2- 172.21.15.145:0/2459219688 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56012eaa3ca0 0x56012eaa6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:57.110 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.109+0000 7fa317600640 1 --2- 172.21.15.145:0/2459219688 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56012eaa3ca0 0x56012eaa6130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:57.110 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.109+0000 7fa317600640 1 -- 172.21.15.145:0/2459219688 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56012eaa68b0 con 0x56012eaa2590 2024-05-09T04:34:57.110 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.109+0000 7fa317600640 1 --2- 172.21.15.145:0/2459219688 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56012eaa2590 0x56012eaad9d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa308010560 tx=0x7fa308010a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:57.111 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.110+0000 7fa314e00640 1 -- 172.21.15.145:0/2459219688 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa308017030 con 0x56012eaa2590 2024-05-09T04:34:57.111 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.110+0000 7fa314e00640 1 -- 172.21.15.145:0/2459219688 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3080027d0 con 0x56012eaa2590 2024-05-09T04:34:57.111 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.110+0000 7fa31af21f40 1 -- 172.21.15.145:0/2459219688 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56012eaa70d0 con 0x56012eaa2590 2024-05-09T04:34:57.111 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.110+0000 7fa314e00640 1 -- 172.21.15.145:0/2459219688 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3080029c0 con 0x56012eaa2590 2024-05-09T04:34:57.111 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.110+0000 7fa31af21f40 1 -- 172.21.15.145:0/2459219688 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56012eaa7690 con 0x56012eaa2590 2024-05-09T04:34:57.111 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.112+0000 7fa314e00640 1 -- 172.21.15.145:0/2459219688 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa30801a030 con 0x56012eaa2590 2024-05-09T04:34:57.112 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.113+0000 7fa314e00640 1 --2- 172.21.15.145:0/2459219688 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fa2f4074ce0 0x7fa2f40771a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:57.112 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.113+0000 7fa314e00640 1 -- 172.21.15.145:0/2459219688 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fa308096630 con 0x56012eaa2590 2024-05-09T04:34:57.113 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.113+0000 7fa30ec00640 1 --2- 172.21.15.145:0/2459219688 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fa2f4074ce0 0x7fa2f40771a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:57.113 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.114+0000 7fa31af21f40 1 --2- 172.21.15.145:0/2459219688 >> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] conn(0x56012eaa7bb0 0x56012eaa7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:57.113 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.114+0000 7fa31af21f40 1 -- 172.21.15.145:0/2459219688 --> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] -- osd_op(unknown.0.0:1 5.13 5:ca8fdbba:::100000001fb.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x56012eaa2f80 con 0x56012eaa7bb0 2024-05-09T04:34:57.113 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.114+0000 7fa318000640 1 --2- 172.21.15.145:0/2459219688 >> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] conn(0x56012eaa7bb0 0x56012eaa7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:57.113 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.114+0000 7fa30ec00640 1 --2- 172.21.15.145:0/2459219688 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fa2f4074ce0 0x7fa2f40771a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa30000d2e0 tx=0x7fa300002000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:57.114 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.114+0000 7fa318000640 1 --2- 172.21.15.145:0/2459219688 >> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] conn(0x56012eaa7bb0 0x56012eaa7fc0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:57.114 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.115+0000 7fa318000640 1 -- 172.21.15.145:0/2459219688 <== osd.3 v2:172.21.15.145:6808/337009826 1 ==== osd_op_reply(1 100000001fb.00000000 [getxattr (95) out=95b] v0'0 uv1 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x7fa310001430 con 0x56012eaa7bb0 2024-05-09T04:34:57.115 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.115+0000 7fa31af21f40 1 -- 172.21.15.145:0/2459219688 >> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] conn(0x56012eaa7bb0 msgr2=0x56012eaa7fc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:57.115 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.115+0000 7fa31af21f40 1 --2- 172.21.15.145:0/2459219688 >> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] conn(0x56012eaa7bb0 0x56012eaa7fc0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:57.115 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.115+0000 7fa31af21f40 1 -- 172.21.15.145:0/2459219688 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fa2f4074ce0 msgr2=0x7fa2f40771a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:57.115 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.115+0000 7fa31af21f40 1 --2- 172.21.15.145:0/2459219688 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fa2f4074ce0 0x7fa2f40771a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa30000d2e0 tx=0x7fa300002000 comp rx=0 tx=0).stop 2024-05-09T04:34:57.115 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.115+0000 7fa31af21f40 1 -- 172.21.15.145:0/2459219688 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56012eaa2590 msgr2=0x56012eaad9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:57.115 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.115+0000 7fa31af21f40 1 --2- 172.21.15.145:0/2459219688 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56012eaa2590 0x56012eaad9d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa308010560 tx=0x7fa308010a20 comp rx=0 tx=0).stop 2024-05-09T04:34:57.115 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.115+0000 7fa318000640 1 -- 172.21.15.145:0/2459219688 reap_dead start 2024-05-09T04:34:57.115 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.115+0000 7fa31af21f40 1 -- 172.21.15.145:0/2459219688 shutdown_connections 2024-05-09T04:34:57.115 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.115+0000 7fa31af21f40 1 -- 172.21.15.145:0/2459219688 >> 172.21.15.145:0/2459219688 conn(0x56012ea02c80 msgr2=0x56012ea052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:57.115 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.115+0000 7fa31af21f40 1 -- 172.21.15.145:0/2459219688 shutdown_connections 2024-05-09T04:34:57.115 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.115+0000 7fa31af21f40 1 -- 172.21.15.145:0/2459219688 wait complete. 2024-05-09T04:34:57.119 INFO:teuthology.orchestra.run.smithi145.stdout:Y��alphaparent_a 2024-05-09T04:34:57.119 DEBUG:teuthology.orchestra.run.smithi145:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-05-09T04:34:57.207 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-05-09T04:34:57.208 INFO:teuthology.orchestra.run.smithi145.stdout: "ino": 1099511628283, 2024-05-09T04:34:57.208 INFO:teuthology.orchestra.run.smithi145.stdout: "ancestors": [ 2024-05-09T04:34:57.208 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-05-09T04:34:57.208 INFO:teuthology.orchestra.run.smithi145.stdout: "dirino": 1099511628282, 2024-05-09T04:34:57.208 INFO:teuthology.orchestra.run.smithi145.stdout: "dname": "alpha", 2024-05-09T04:34:57.208 INFO:teuthology.orchestra.run.smithi145.stdout: "version": 2 2024-05-09T04:34:57.208 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:34:57.208 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-05-09T04:34:57.208 INFO:teuthology.orchestra.run.smithi145.stdout: "dirino": 1, 2024-05-09T04:34:57.208 INFO:teuthology.orchestra.run.smithi145.stdout: "dname": "parent_a", 2024-05-09T04:34:57.208 INFO:teuthology.orchestra.run.smithi145.stdout: "version": 18 2024-05-09T04:34:57.208 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-05-09T04:34:57.208 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2024-05-09T04:34:57.208 INFO:teuthology.orchestra.run.smithi145.stdout: "pool": 5, 2024-05-09T04:34:57.209 INFO:teuthology.orchestra.run.smithi145.stdout: "old_pools": [] 2024-05-09T04:34:57.209 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-05-09T04:34:57.209 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:34:57.215 DEBUG:teuthology.orchestra.run.smithi145:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001fb.00000000 layout 2024-05-09T04:34:57.322 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.323+0000 7ff452c0ff40 1 Processor -- start 2024-05-09T04:34:57.322 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.323+0000 7ff452c0ff40 1 -- start start 2024-05-09T04:34:57.323 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.324+0000 7ff452c0ff40 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x565110f34b90 0x565110faf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:57.323 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.324+0000 7ff452c0ff40 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565110f354b0 0x565110f19570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:57.323 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.324+0000 7ff452c0ff40 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x565110fba0d0 0x565110f19ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:57.323 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.324+0000 7ff452c0ff40 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x565110f33f10 con 0x565110fba0d0 2024-05-09T04:34:57.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.324+0000 7ff452c0ff40 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x565110e8ff50 con 0x565110f354b0 2024-05-09T04:34:57.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.324+0000 7ff452c0ff40 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x565110f35a30 con 0x565110f34b90 2024-05-09T04:34:57.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.325+0000 7ff44fc00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x565110fba0d0 0x565110f19ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:57.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.325+0000 7ff44fc00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x565110fba0d0 0x565110f19ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:41354/0 (socket says 172.21.15.145:41354) 2024-05-09T04:34:57.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.325+0000 7ff44fc00640 1 -- 172.21.15.145:0/3709533040 learned_addr learned my addr 172.21.15.145:0/3709533040 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:57.325 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.325+0000 7ff44e800640 1 --2- 172.21.15.145:0/3709533040 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565110f354b0 0x565110f19570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:57.325 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.325+0000 7ff44fc00640 1 -- 172.21.15.145:0/3709533040 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x565110f34b90 msgr2=0x565110faf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:57.326 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.325+0000 7ff44fc00640 1 --2- 172.21.15.145:0/3709533040 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x565110f34b90 0x565110faf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:57.326 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.325+0000 7ff44fc00640 1 -- 172.21.15.145:0/3709533040 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565110f354b0 msgr2=0x565110f19570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:57.326 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.325+0000 7ff44fc00640 1 --2- 172.21.15.145:0/3709533040 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565110f354b0 0x565110f19570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:57.326 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.325+0000 7ff44fc00640 1 -- 172.21.15.145:0/3709533040 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565110e12b30 con 0x565110fba0d0 2024-05-09T04:34:57.326 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.325+0000 7ff44fc00640 1 --2- 172.21.15.145:0/3709533040 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x565110fba0d0 0x565110f19ab0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff448045a20 tx=0x7ff448051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc4ec7d73f0cdfbc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:57.326 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.326+0000 7ff44d400640 1 -- 172.21.15.145:0/3709533040 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff448053a50 con 0x565110fba0d0 2024-05-09T04:34:57.326 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.326+0000 7ff44d400640 1 -- 172.21.15.145:0/3709533040 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff448062070 con 0x565110fba0d0 2024-05-09T04:34:57.326 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.326+0000 7ff44d400640 1 -- 172.21.15.145:0/3709533040 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff448053e30 con 0x565110fba0d0 2024-05-09T04:34:57.326 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.326+0000 7ff452c0ff40 1 -- 172.21.15.145:0/3709533040 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x565110fba0d0 msgr2=0x565110f19ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:57.326 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.326+0000 7ff452c0ff40 1 --2- 172.21.15.145:0/3709533040 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x565110fba0d0 0x565110f19ab0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff448045a20 tx=0x7ff448051000 comp rx=0 tx=0).stop 2024-05-09T04:34:57.326 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.326+0000 7ff452c0ff40 1 -- 172.21.15.145:0/3709533040 shutdown_connections 2024-05-09T04:34:57.326 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.326+0000 7ff452c0ff40 1 --2- 172.21.15.145:0/3709533040 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x565110fba0d0 0x565110f19ab0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:57.326 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.326+0000 7ff452c0ff40 1 --2- 172.21.15.145:0/3709533040 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565110f354b0 0x565110f19570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:57.327 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.326+0000 7ff452c0ff40 1 --2- 172.21.15.145:0/3709533040 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x565110f34b90 0x565110faf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:57.327 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.326+0000 7ff452c0ff40 1 -- 172.21.15.145:0/3709533040 >> 172.21.15.145:0/3709533040 conn(0x565110f17c80 msgr2=0x565110fbdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:57.327 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.326+0000 7ff452c0ff40 1 -- 172.21.15.145:0/3709533040 shutdown_connections 2024-05-09T04:34:57.327 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.326+0000 7ff452c0ff40 1 -- 172.21.15.145:0/3709533040 wait complete. 2024-05-09T04:34:57.327 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.327+0000 7ff452c0ff40 1 Processor -- start 2024-05-09T04:34:57.330 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.327+0000 7ff452c0ff40 1 -- start start 2024-05-09T04:34:57.330 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.328+0000 7ff452c0ff40 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565110f34b90 0x565110fa3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:57.330 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.328+0000 7ff452c0ff40 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x565110f354b0 0x565110fbd640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:57.330 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.328+0000 7ff452c0ff40 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x565110fba0d0 0x565110fc2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:57.330 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.328+0000 7ff452c0ff40 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x565110fc1490 con 0x565110f354b0 2024-05-09T04:34:57.330 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.328+0000 7ff452c0ff40 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x565110fbf540 con 0x565110f34b90 2024-05-09T04:34:57.331 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.328+0000 7ff452c0ff40 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x565110f33150 con 0x565110fba0d0 2024-05-09T04:34:57.331 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.328+0000 7ff44e800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x565110f354b0 0x565110fbd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:57.331 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.328+0000 7ff44e800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x565110f354b0 0x565110fbd640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:41368/0 (socket says 172.21.15.145:41368) 2024-05-09T04:34:57.331 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.328+0000 7ff44e800640 1 -- 172.21.15.145:0/3849656337 learned_addr learned my addr 172.21.15.145:0/3849656337 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:57.331 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.329+0000 7ff44f200640 1 --2- 172.21.15.145:0/3849656337 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565110f34b90 0x565110fa3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:57.331 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.329+0000 7ff44e800640 1 -- 172.21.15.145:0/3849656337 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x565110fba0d0 msgr2=0x565110fc2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:57.331 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.329+0000 7ff44e800640 1 --2- 172.21.15.145:0/3849656337 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x565110fba0d0 0x565110fc2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:57.331 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.329+0000 7ff44e800640 1 -- 172.21.15.145:0/3849656337 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565110f34b90 msgr2=0x565110fa3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:57.331 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.329+0000 7ff44e800640 1 --2- 172.21.15.145:0/3849656337 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565110f34b90 0x565110fa3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:57.331 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.329+0000 7ff44e800640 1 -- 172.21.15.145:0/3849656337 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565110fc1200 con 0x565110f354b0 2024-05-09T04:34:57.331 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.329+0000 7ff44f200640 1 --2- 172.21.15.145:0/3849656337 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x565110f34b90 0x565110fa3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:34:57.331 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.329+0000 7ff44e800640 1 --2- 172.21.15.145:0/3849656337 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x565110f354b0 0x565110fbd640 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff438007e00 tx=0x7ff438006750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:57.331 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.329+0000 7ff447e00640 1 -- 172.21.15.145:0/3849656337 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff438017070 con 0x565110f354b0 2024-05-09T04:34:57.331 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.330+0000 7ff447e00640 1 -- 172.21.15.145:0/3849656337 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff438010730 con 0x565110f354b0 2024-05-09T04:34:57.331 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.330+0000 7ff452c0ff40 1 -- 172.21.15.145:0/3849656337 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x565110fc0f20 con 0x565110f354b0 2024-05-09T04:34:57.332 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.330+0000 7ff447e00640 1 -- 172.21.15.145:0/3849656337 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff438010b20 con 0x565110f354b0 2024-05-09T04:34:57.332 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.330+0000 7ff452c0ff40 1 -- 172.21.15.145:0/3849656337 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565110fabce0 con 0x565110f354b0 2024-05-09T04:34:57.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.332+0000 7ff447e00640 1 -- 172.21.15.145:0/3849656337 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff43801a070 con 0x565110f354b0 2024-05-09T04:34:57.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.333+0000 7ff447e00640 1 --2- 172.21.15.145:0/3849656337 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff42c074d00 0x7ff42c0771c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:57.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.333+0000 7ff447e00640 1 -- 172.21.15.145:0/3849656337 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7ff4380957e0 con 0x565110f354b0 2024-05-09T04:34:57.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.333+0000 7ff44f200640 1 --2- 172.21.15.145:0/3849656337 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff42c074d00 0x7ff42c0771c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:57.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.333+0000 7ff452c0ff40 1 --2- 172.21.15.145:0/3849656337 >> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] conn(0x565110fbaf50 0x565110fbb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:57.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.333+0000 7ff452c0ff40 1 -- 172.21.15.145:0/3849656337 --> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] -- osd_op(unknown.0.0:1 5.13 5:ca8fdbba:::100000001fb.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x565110fbb8e0 con 0x565110fbaf50 2024-05-09T04:34:57.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.334+0000 7ff44fc00640 1 --2- 172.21.15.145:0/3849656337 >> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] conn(0x565110fbaf50 0x565110fbb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:57.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.334+0000 7ff44f200640 1 --2- 172.21.15.145:0/3849656337 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff42c074d00 0x7ff42c0771c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff44000b4a0 tx=0x7ff440007040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:57.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.334+0000 7ff44fc00640 1 --2- 172.21.15.145:0/3849656337 >> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] conn(0x565110fbaf50 0x565110fbb3a0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:57.334 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.335+0000 7ff44fc00640 1 -- 172.21.15.145:0/3849656337 <== osd.3 v2:172.21.15.145:6808/337009826 1 ==== osd_op_reply(1 100000001fb.00000000 [getxattr (30) out=30b] v0'0 uv1 ondisk = 0) ==== 164+0+30 (crc 0 0 0) 0x7ff448053970 con 0x565110fbaf50 2024-05-09T04:34:57.335 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.335+0000 7ff452c0ff40 1 -- 172.21.15.145:0/3849656337 >> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] conn(0x565110fbaf50 msgr2=0x565110fbb3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:57.335 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.335+0000 7ff452c0ff40 1 --2- 172.21.15.145:0/3849656337 >> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] conn(0x565110fbaf50 0x565110fbb3a0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:57.335 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.335+0000 7ff452c0ff40 1 -- 172.21.15.145:0/3849656337 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff42c074d00 msgr2=0x7ff42c0771c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:57.335 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.335+0000 7ff452c0ff40 1 --2- 172.21.15.145:0/3849656337 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff42c074d00 0x7ff42c0771c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff44000b4a0 tx=0x7ff440007040 comp rx=0 tx=0).stop 2024-05-09T04:34:57.335 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.335+0000 7ff452c0ff40 1 -- 172.21.15.145:0/3849656337 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x565110f354b0 msgr2=0x565110fbd640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:57.335 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.335+0000 7ff452c0ff40 1 --2- 172.21.15.145:0/3849656337 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x565110f354b0 0x565110fbd640 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff438007e00 tx=0x7ff438006750 comp rx=0 tx=0).stop 2024-05-09T04:34:57.335 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.335+0000 7ff44fc00640 1 -- 172.21.15.145:0/3849656337 reap_dead start 2024-05-09T04:34:57.335 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.335+0000 7ff452c0ff40 1 -- 172.21.15.145:0/3849656337 shutdown_connections 2024-05-09T04:34:57.335 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.335+0000 7ff452c0ff40 1 -- 172.21.15.145:0/3849656337 >> 172.21.15.145:0/3849656337 conn(0x565110f17c80 msgr2=0x565110f18ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:57.335 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.335+0000 7ff452c0ff40 1 -- 172.21.15.145:0/3849656337 shutdown_connections 2024-05-09T04:34:57.335 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.335+0000 7ff452c0ff40 1 -- 172.21.15.145:0/3849656337 wait complete. 2024-05-09T04:34:57.338 INFO:teuthology.orchestra.run.smithi145.stdout:@@ 2024-05-09T04:34:57.339 DEBUG:teuthology.orchestra.run.smithi145:> ceph-dencoder type file_layout_t import - decode dump_json 2024-05-09T04:34:57.442 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-05-09T04:34:57.442 INFO:teuthology.orchestra.run.smithi145.stdout: "stripe_unit": 4194304, 2024-05-09T04:34:57.442 INFO:teuthology.orchestra.run.smithi145.stdout: "stripe_count": 1, 2024-05-09T04:34:57.442 INFO:teuthology.orchestra.run.smithi145.stdout: "object_size": 4194304, 2024-05-09T04:34:57.442 INFO:teuthology.orchestra.run.smithi145.stdout: "pool_id": 5, 2024-05-09T04:34:57.442 INFO:teuthology.orchestra.run.smithi145.stdout: "pool_ns": "" 2024-05-09T04:34:57.442 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-05-09T04:34:57.442 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:34:57.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:57.447 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent_b) 2024-05-09T04:34:57.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:34:57.504 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest/mnt.0 && exec mv parent_a/alpha parent_b/alpha) 2024-05-09T04:34:57.571 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T04:34:57.770 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.771+0000 7fb327200640 1 Processor -- start 2024-05-09T04:34:57.771 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.772+0000 7fb327200640 1 -- start start 2024-05-09T04:34:57.772 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.773+0000 7fb327200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fb3280d3180 0x7fb3280d5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:57.772 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.773+0000 7fb327200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb3280b4760 0x7fb3280b4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:57.772 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.773+0000 7fb327200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3280d3e70 con 0x7fb3280d22f0 2024-05-09T04:34:57.772 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.773+0000 7fb327200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fb3280d4150 con 0x7fb3280d3180 2024-05-09T04:34:57.772 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.773+0000 7fb327200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fb3280b6200 con 0x7fb3280b4760 2024-05-09T04:34:57.772 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.773+0000 7fb325e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fb3280d3180 0x7fb3280d5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:57.772 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.773+0000 7fb325e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fb3280d3180 0x7fb3280d5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:41378/0 (socket says 172.21.15.145:41378) 2024-05-09T04:34:57.772 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.773+0000 7fb325e00640 1 -- 172.21.15.145:0/1218730755 learned_addr learned my addr 172.21.15.145:0/1218730755 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:57.773 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.774+0000 7fb326800640 1 --2- 172.21.15.145:0/1218730755 >> v2:172.21.15.192:3300/0 conn(0x7fb3280b4760 0x7fb3280b4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:57.773 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.774+0000 7fb325e00640 1 -- 172.21.15.145:0/1218730755 >> v2:172.21.15.192:3300/0 conn(0x7fb3280b4760 msgr2=0x7fb3280b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:57.773 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.774+0000 7fb325e00640 1 --2- 172.21.15.145:0/1218730755 >> v2:172.21.15.192:3300/0 conn(0x7fb3280b4760 0x7fb3280b4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:57.773 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.774+0000 7fb325e00640 1 -- 172.21.15.145:0/1218730755 >> v1:172.21.15.192:6789/0 conn(0x7fb3280d22f0 legacy=0x7fb3280d3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:57.773 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.774+0000 7fb325e00640 1 -- 172.21.15.145:0/1218730755 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3280b6450 con 0x7fb3280d3180 2024-05-09T04:34:57.773 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.774+0000 7fb326800640 1 --2- 172.21.15.145:0/1218730755 >> v2:172.21.15.192:3300/0 conn(0x7fb3280b4760 0x7fb3280b4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:34:57.775 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.775+0000 7fb325e00640 1 --2- 172.21.15.145:0/1218730755 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb3280d3180 0x7fb3280d5ad0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb3280d4b10 tx=0x7fb31802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1de1ac9f9e92268e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:57.775 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.775+0000 7fb324a00640 1 -- 172.21.15.145:0/1218730755 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb318036070 con 0x7fb3280d3180 2024-05-09T04:34:57.775 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.775+0000 7fb324a00640 1 -- 172.21.15.145:0/1218730755 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb318031600 con 0x7fb3280d3180 2024-05-09T04:34:57.775 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.775+0000 7fb324a00640 1 -- 172.21.15.145:0/1218730755 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3180319d0 con 0x7fb3280d3180 2024-05-09T04:34:57.775 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.776+0000 7fb327200640 1 -- 172.21.15.145:0/1218730755 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb3280d3180 msgr2=0x7fb3280d5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:57.775 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.776+0000 7fb327200640 1 --2- 172.21.15.145:0/1218730755 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb3280d3180 0x7fb3280d5ad0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb3280d4b10 tx=0x7fb31802a610 comp rx=0 tx=0).stop 2024-05-09T04:34:57.775 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.776+0000 7fb327200640 1 -- 172.21.15.145:0/1218730755 shutdown_connections 2024-05-09T04:34:57.775 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.776+0000 7fb327200640 1 --2- 172.21.15.145:0/1218730755 >> v2:172.21.15.192:3300/0 conn(0x7fb3280b4760 0x7fb3280b4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:57.776 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.776+0000 7fb327200640 1 --2- 172.21.15.145:0/1218730755 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb3280d3180 0x7fb3280d5ad0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:57.776 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.776+0000 7fb327200640 1 -- 172.21.15.145:0/1218730755 >> 172.21.15.145:0/1218730755 conn(0x7fb3280d4fb0 msgr2=0x7fb3280d53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:57.776 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.776+0000 7fb327200640 1 -- 172.21.15.145:0/1218730755 shutdown_connections 2024-05-09T04:34:57.776 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.776+0000 7fb327200640 1 -- 172.21.15.145:0/1218730755 wait complete. 2024-05-09T04:34:57.777 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.778+0000 7fb327200640 1 Processor -- start 2024-05-09T04:34:57.777 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.779+0000 7fb327200640 1 -- start start 2024-05-09T04:34:57.779 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.780+0000 7fb327200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb3280b4760 0x7fb3280cb280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:57.779 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.780+0000 7fb327200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb3280d22f0 0x7fb3281580a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:57.779 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.780+0000 7fb327200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb3280d3180 0x7fb3280c8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:57.779 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.780+0000 7fb327200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb3280d2780 con 0x7fb3280d22f0 2024-05-09T04:34:57.779 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.780+0000 7fb327200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb3280d1e60 con 0x7fb3280b4760 2024-05-09T04:34:57.779 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.780+0000 7fb327200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb3280b3fa0 con 0x7fb3280d3180 2024-05-09T04:34:57.779 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.780+0000 7fb325400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb3280d22f0 0x7fb3281580a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:57.779 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.780+0000 7fb325400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb3280d22f0 0x7fb3281580a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:41380/0 (socket says 172.21.15.145:41380) 2024-05-09T04:34:57.780 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.780+0000 7fb325400640 1 -- 172.21.15.145:0/621826350 learned_addr learned my addr 172.21.15.145:0/621826350 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:57.781 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.781+0000 7fb325400640 1 -- 172.21.15.145:0/621826350 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb3280d3180 msgr2=0x7fb3280c8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:57.781 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.781+0000 7fb326800640 1 --2- 172.21.15.145:0/621826350 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb3280d3180 0x7fb3280c8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:57.781 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.781+0000 7fb325e00640 1 --2- 172.21.15.145:0/621826350 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb3280b4760 0x7fb3280cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:57.781 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.781+0000 7fb325400640 1 --2- 172.21.15.145:0/621826350 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb3280d3180 0x7fb3280c8ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:57.781 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.781+0000 7fb325400640 1 -- 172.21.15.145:0/621826350 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb3280b4760 msgr2=0x7fb3280cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:57.781 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.781+0000 7fb325400640 1 --2- 172.21.15.145:0/621826350 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb3280b4760 0x7fb3280cb280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:57.781 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.781+0000 7fb325400640 1 -- 172.21.15.145:0/621826350 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3281585e0 con 0x7fb3280d22f0 2024-05-09T04:34:57.781 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.781+0000 7fb325400640 1 --2- 172.21.15.145:0/621826350 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb3280d22f0 0x7fb3281580a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb31000c410 tx=0x7fb31000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:57.781 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.781+0000 7fb31e000640 1 -- 172.21.15.145:0/621826350 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb310017070 con 0x7fb3280d22f0 2024-05-09T04:34:57.781 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.781+0000 7fb31e000640 1 -- 172.21.15.145:0/621826350 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3100035f0 con 0x7fb3280d22f0 2024-05-09T04:34:57.781 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.781+0000 7fb31e000640 1 -- 172.21.15.145:0/621826350 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb310003a20 con 0x7fb3280d22f0 2024-05-09T04:34:57.781 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.781+0000 7fb327200640 1 -- 172.21.15.145:0/621826350 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3280cb8f0 con 0x7fb3280d22f0 2024-05-09T04:34:57.782 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.781+0000 7fb327200640 1 -- 172.21.15.145:0/621826350 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb32815c910 con 0x7fb3280d22f0 2024-05-09T04:34:57.782 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.783+0000 7fb327200640 1 -- 172.21.15.145:0/621826350 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb32806a310 con 0x7fb3280d22f0 2024-05-09T04:34:57.782 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.784+0000 7fb31e000640 1 -- 172.21.15.145:0/621826350 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb310003bc0 con 0x7fb3280d22f0 2024-05-09T04:34:57.784 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.785+0000 7fb31e000640 1 --2- 172.21.15.145:0/621826350 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fb3040762d0 0x7fb304078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:57.784 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.785+0000 7fb31e000640 1 -- 172.21.15.145:0/621826350 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fb310096930 con 0x7fb3280d22f0 2024-05-09T04:34:57.784 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.785+0000 7fb325e00640 1 --2- 172.21.15.145:0/621826350 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fb3040762d0 0x7fb304078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:57.784 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.786+0000 7fb325e00640 1 --2- 172.21.15.145:0/621826350 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fb3040762d0 0x7fb304078790 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb318033000 tx=0x7fb318009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:57.788 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:57.790+0000 7fb31e000640 1 -- 172.21.15.145:0/621826350 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb3281585e0 con 0x7fb3280d22f0 2024-05-09T04:34:58.041 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.042+0000 7fb327200640 1 -- 172.21.15.145:0/621826350 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb328162b50 con 0x7fb3280d22f0 2024-05-09T04:34:58.042 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.043+0000 7fb31e000640 1 -- 172.21.15.145:0/621826350 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5187 (secure 0 0 0) 0x7fb310060280 con 0x7fb3280d22f0 2024-05-09T04:34:58.042 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:34:58.042 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4499,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6834/3227728059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3227728059},{"type":"v1","addr":"172.21.15.192:6834","nonce":3227728059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4502,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1603013787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6833","nonce":1603013787},{"type":"v1","addr":"172.21.15.192:6835","nonce":1603013787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4672,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/68778418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":68778418},{"type":"v1","addr":"172.21.15.145:6837","nonce":68778418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T04:34:43.133970+0000","modified":"2024-05-09T04:34:46.181657+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4761},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4761":{"gid":4761,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.145:6835/4260798843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4260798843},{"type":"v1","addr":"172.21.15.145:6835","nonce":4260798843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4761,"qdb_cluster":[4761]},"id":2}]} 2024-05-09T04:34:58.043 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 17 2024-05-09T04:34:58.046 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.047+0000 7fb327200640 1 -- 172.21.15.145:0/621826350 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fb3040762d0 msgr2=0x7fb304078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:58.046 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.047+0000 7fb327200640 1 --2- 172.21.15.145:0/621826350 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fb3040762d0 0x7fb304078790 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb318033000 tx=0x7fb318009000 comp rx=0 tx=0).stop 2024-05-09T04:34:58.046 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.047+0000 7fb327200640 1 -- 172.21.15.145:0/621826350 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb3280d22f0 msgr2=0x7fb3281580a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:58.046 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.047+0000 7fb327200640 1 --2- 172.21.15.145:0/621826350 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb3280d22f0 0x7fb3281580a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb31000c410 tx=0x7fb31000c8d0 comp rx=0 tx=0).stop 2024-05-09T04:34:58.046 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.047+0000 7fb327200640 1 -- 172.21.15.145:0/621826350 shutdown_connections 2024-05-09T04:34:58.046 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.047+0000 7fb327200640 1 --2- 172.21.15.145:0/621826350 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fb3040762d0 0x7fb304078790 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:58.046 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.047+0000 7fb327200640 1 --2- 172.21.15.145:0/621826350 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb3280d3180 0x7fb3280c8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:58.047 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.047+0000 7fb327200640 1 --2- 172.21.15.145:0/621826350 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb3280d22f0 0x7fb3281580a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:58.047 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.047+0000 7fb327200640 1 --2- 172.21.15.145:0/621826350 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb3280b4760 0x7fb3280cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:58.047 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.047+0000 7fb327200640 1 -- 172.21.15.145:0/621826350 >> 172.21.15.145:0/621826350 conn(0x7fb3280d4fb0 msgr2=0x7fb3280b2920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:58.047 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.047+0000 7fb327200640 1 -- 172.21.15.145:0/621826350 shutdown_connections 2024-05-09T04:34:58.047 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.048+0000 7fb327200640 1 -- 172.21.15.145:0/621826350 wait complete. 2024-05-09T04:34:58.065 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-05-09T04:34:58.302 INFO:teuthology.orchestra.run.smithi145.stdout:{"message":"","return_code":0} 2024-05-09T04:34:58.303 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-09T04:34:58.303 DEBUG:teuthology.orchestra.run.smithi145:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001fb.00000000 parent 2024-05-09T04:34:58.406 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.407+0000 7f8caec34f40 1 Processor -- start 2024-05-09T04:34:58.406 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.407+0000 7f8caec34f40 1 -- start start 2024-05-09T04:34:58.408 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.408+0000 7f8caec34f40 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562326c76ee0 0x562326d13590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:58.408 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.408+0000 7f8caec34f40 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562326c90b90 0x562326c728b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:58.408 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.408+0000 7f8caec34f40 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x562326d160d0 0x562326c72df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:58.408 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.408+0000 7f8caec34f40 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x562326bebf50 con 0x562326c76ee0 2024-05-09T04:34:58.408 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.408+0000 7f8caec34f40 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x562326beff90 con 0x562326c90b90 2024-05-09T04:34:58.408 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.408+0000 7f8caec34f40 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x562326c8ff10 con 0x562326d160d0 2024-05-09T04:34:58.408 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.408+0000 7f8cabe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562326c76ee0 0x562326d13590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:58.408 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.408+0000 7f8cabe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562326c76ee0 0x562326d13590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:41404/0 (socket says 172.21.15.145:41404) 2024-05-09T04:34:58.408 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.408+0000 7f8cabe00640 1 -- 172.21.15.145:0/743599141 learned_addr learned my addr 172.21.15.145:0/743599141 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:58.409 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.409+0000 7f8cabe00640 1 -- 172.21.15.145:0/743599141 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x562326d160d0 msgr2=0x562326c72df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:58.409 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.409+0000 7f8cabe00640 1 --2- 172.21.15.145:0/743599141 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x562326d160d0 0x562326c72df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:58.409 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.409+0000 7f8cabe00640 1 -- 172.21.15.145:0/743599141 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562326c90b90 msgr2=0x562326c728b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:58.409 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.409+0000 7f8cabe00640 1 --2- 172.21.15.145:0/743599141 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562326c90b90 0x562326c728b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:58.409 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.409+0000 7f8cabe00640 1 -- 172.21.15.145:0/743599141 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562326c92b00 con 0x562326c76ee0 2024-05-09T04:34:58.409 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.409+0000 7f8cabe00640 1 --2- 172.21.15.145:0/743599141 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562326c76ee0 0x562326d13590 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c0460e0 tx=0x7f8c9c051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5f40048f4b13e86 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:58.409 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.409+0000 7f8caa000640 1 -- 172.21.15.145:0/743599141 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c067070 con 0x562326c76ee0 2024-05-09T04:34:58.409 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.409+0000 7f8caa000640 1 -- 172.21.15.145:0/743599141 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c9c058b50 con 0x562326c76ee0 2024-05-09T04:34:58.409 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.409+0000 7f8caa000640 1 -- 172.21.15.145:0/743599141 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c0623f0 con 0x562326c76ee0 2024-05-09T04:34:58.411 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.410+0000 7f8caec34f40 1 -- 172.21.15.145:0/743599141 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562326c76ee0 msgr2=0x562326d13590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:58.411 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.410+0000 7f8caec34f40 1 --2- 172.21.15.145:0/743599141 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562326c76ee0 0x562326d13590 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c0460e0 tx=0x7f8c9c051000 comp rx=0 tx=0).stop 2024-05-09T04:34:58.411 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.410+0000 7f8caec34f40 1 -- 172.21.15.145:0/743599141 shutdown_connections 2024-05-09T04:34:58.411 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.410+0000 7f8caec34f40 1 --2- 172.21.15.145:0/743599141 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x562326d160d0 0x562326c72df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:58.411 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.410+0000 7f8caec34f40 1 --2- 172.21.15.145:0/743599141 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562326c90b90 0x562326c728b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:58.411 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.410+0000 7f8caec34f40 1 --2- 172.21.15.145:0/743599141 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562326c76ee0 0x562326d13590 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:58.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.410+0000 7f8caec34f40 1 -- 172.21.15.145:0/743599141 >> 172.21.15.145:0/743599141 conn(0x562326c73c80 msgr2=0x562326c922c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:58.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.410+0000 7f8caec34f40 1 -- 172.21.15.145:0/743599141 shutdown_connections 2024-05-09T04:34:58.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.410+0000 7f8caec34f40 1 -- 172.21.15.145:0/743599141 wait complete. 2024-05-09T04:34:58.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.411+0000 7f8caec34f40 1 Processor -- start 2024-05-09T04:34:58.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.411+0000 7f8caec34f40 1 -- start start 2024-05-09T04:34:58.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.412+0000 7f8caec34f40 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x562326c76ee0 0x562326da4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:58.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.412+0000 7f8caec34f40 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562326c90b90 0x562326d06170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:58.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.412+0000 7f8caec34f40 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562326d160d0 0x562326d05620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:58.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.412+0000 7f8caec34f40 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x562326c45530 con 0x562326c90b90 2024-05-09T04:34:58.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.412+0000 7f8caec34f40 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x562326c73950 con 0x562326d160d0 2024-05-09T04:34:58.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.412+0000 7f8caec34f40 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x562326c72420 con 0x562326c76ee0 2024-05-09T04:34:58.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.412+0000 7f8cab400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562326c90b90 0x562326d06170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:58.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.412+0000 7f8cab400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562326c90b90 0x562326d06170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:41414/0 (socket says 172.21.15.145:41414) 2024-05-09T04:34:58.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.412+0000 7f8cab400640 1 -- 172.21.15.145:0/2689592225 learned_addr learned my addr 172.21.15.145:0/2689592225 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:58.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.412+0000 7f8cab400640 1 -- 172.21.15.145:0/2689592225 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x562326c76ee0 msgr2=0x562326da4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:58.413 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.412+0000 7f8cab400640 1 --2- 172.21.15.145:0/2689592225 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x562326c76ee0 0x562326da4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:58.413 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.412+0000 7f8cab400640 1 -- 172.21.15.145:0/2689592225 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562326d160d0 msgr2=0x562326d05620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:58.413 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.412+0000 7f8cab400640 1 --2- 172.21.15.145:0/2689592225 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x562326d160d0 0x562326d05620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:58.413 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.412+0000 7f8cab400640 1 -- 172.21.15.145:0/2689592225 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562326da4680 con 0x562326c90b90 2024-05-09T04:34:58.413 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.413+0000 7f8cab400640 1 --2- 172.21.15.145:0/2689592225 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562326c90b90 0x562326d06170 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f8c94001e50 tx=0x7f8c94009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:58.413 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.413+0000 7f8ca8c00640 1 -- 172.21.15.145:0/2689592225 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c94008070 con 0x562326c90b90 2024-05-09T04:34:58.414 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.413+0000 7f8ca8c00640 1 -- 172.21.15.145:0/2689592225 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c9400f630 con 0x562326c90b90 2024-05-09T04:34:58.414 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.413+0000 7f8ca8c00640 1 -- 172.21.15.145:0/2689592225 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9400fa60 con 0x562326c90b90 2024-05-09T04:34:58.414 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.413+0000 7f8caec34f40 1 -- 172.21.15.145:0/2689592225 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562326d11b20 con 0x562326c90b90 2024-05-09T04:34:58.414 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.413+0000 7f8caec34f40 1 -- 172.21.15.145:0/2689592225 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562326d1e560 con 0x562326c90b90 2024-05-09T04:34:58.414 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.415+0000 7f8ca8c00640 1 -- 172.21.15.145:0/2689592225 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8c9400fc00 con 0x562326c90b90 2024-05-09T04:34:58.415 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.416+0000 7f8ca8c00640 1 --2- 172.21.15.145:0/2689592225 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f8c88074cb0 0x7f8c88077170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:58.415 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.417+0000 7f8ca8c00640 1 -- 172.21.15.145:0/2689592225 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f8c94096420 con 0x562326c90b90 2024-05-09T04:34:58.415 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.417+0000 7f8cabe00640 1 --2- 172.21.15.145:0/2689592225 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f8c88074cb0 0x7f8c88077170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:58.416 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.417+0000 7f8caec34f40 1 --2- 172.21.15.145:0/2689592225 >> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] conn(0x562326da5470 0x562326d18200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:58.416 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.417+0000 7f8caec34f40 1 -- 172.21.15.145:0/2689592225 --> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] -- osd_op(unknown.0.0:1 5.13 5:ca8fdbba:::100000001fb.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x562326d18740 con 0x562326da5470 2024-05-09T04:34:58.416 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.417+0000 7f8cac800640 1 --2- 172.21.15.145:0/2689592225 >> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] conn(0x562326da5470 0x562326d18200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:58.416 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.417+0000 7f8cabe00640 1 --2- 172.21.15.145:0/2689592225 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f8c88074cb0 0x7f8c88077170 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c064000 tx=0x7f8c9c005d80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:58.417 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.418+0000 7f8cac800640 1 --2- 172.21.15.145:0/2689592225 >> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] conn(0x562326da5470 0x562326d18200 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:58.417 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.418+0000 7f8cac800640 1 -- 172.21.15.145:0/2689592225 <== osd.3 v2:172.21.15.145:6808/337009826 1 ==== osd_op_reply(1 100000001fb.00000000 [getxattr (95) out=95b] v0'0 uv2 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x7f8ca400d2c0 con 0x562326da5470 2024-05-09T04:34:58.417 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.418+0000 7f8caec34f40 1 -- 172.21.15.145:0/2689592225 >> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] conn(0x562326da5470 msgr2=0x562326d18200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:58.418 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.418+0000 7f8caec34f40 1 --2- 172.21.15.145:0/2689592225 >> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] conn(0x562326da5470 0x562326d18200 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:58.418 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.418+0000 7f8caec34f40 1 -- 172.21.15.145:0/2689592225 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f8c88074cb0 msgr2=0x7f8c88077170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:58.418 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.418+0000 7f8caec34f40 1 --2- 172.21.15.145:0/2689592225 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f8c88074cb0 0x7f8c88077170 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c064000 tx=0x7f8c9c005d80 comp rx=0 tx=0).stop 2024-05-09T04:34:58.418 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.418+0000 7f8caec34f40 1 -- 172.21.15.145:0/2689592225 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562326c90b90 msgr2=0x562326d06170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:58.418 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.418+0000 7f8caec34f40 1 --2- 172.21.15.145:0/2689592225 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x562326c90b90 0x562326d06170 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f8c94001e50 tx=0x7f8c94009560 comp rx=0 tx=0).stop 2024-05-09T04:34:58.418 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.418+0000 7f8cac800640 1 -- 172.21.15.145:0/2689592225 reap_dead start 2024-05-09T04:34:58.418 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.419+0000 7f8caec34f40 1 -- 172.21.15.145:0/2689592225 shutdown_connections 2024-05-09T04:34:58.418 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.419+0000 7f8caec34f40 1 -- 172.21.15.145:0/2689592225 >> 172.21.15.145:0/2689592225 conn(0x562326c73c80 msgr2=0x562326c92200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:58.418 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.419+0000 7f8caec34f40 1 -- 172.21.15.145:0/2689592225 shutdown_connections 2024-05-09T04:34:58.418 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.419+0000 7f8caec34f40 1 -- 172.21.15.145:0/2689592225 wait complete. 2024-05-09T04:34:58.422 INFO:teuthology.orchestra.run.smithi145.stdout:Y��alphaparent_b 2024-05-09T04:34:58.422 DEBUG:teuthology.orchestra.run.smithi145:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-05-09T04:34:58.517 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-05-09T04:34:58.575 INFO:teuthology.orchestra.run.smithi145.stdout: "ino": 1099511628283, 2024-05-09T04:34:58.575 INFO:teuthology.orchestra.run.smithi145.stdout: "ancestors": [ 2024-05-09T04:34:58.575 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-05-09T04:34:58.575 INFO:teuthology.orchestra.run.smithi145.stdout: "dirino": 1099511628284, 2024-05-09T04:34:58.576 INFO:teuthology.orchestra.run.smithi145.stdout: "dname": "alpha", 2024-05-09T04:34:58.576 INFO:teuthology.orchestra.run.smithi145.stdout: "version": 5 2024-05-09T04:34:58.576 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:34:58.576 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-05-09T04:34:58.576 INFO:teuthology.orchestra.run.smithi145.stdout: "dirino": 1, 2024-05-09T04:34:58.576 INFO:teuthology.orchestra.run.smithi145.stdout: "dname": "parent_b", 2024-05-09T04:34:58.576 INFO:teuthology.orchestra.run.smithi145.stdout: "version": 27 2024-05-09T04:34:58.576 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-05-09T04:34:58.576 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2024-05-09T04:34:58.576 INFO:teuthology.orchestra.run.smithi145.stdout: "pool": 5, 2024-05-09T04:34:58.576 INFO:teuthology.orchestra.run.smithi145.stdout: "old_pools": [] 2024-05-09T04:34:58.576 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-05-09T04:34:58.576 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:34:58.578 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create data_new --pg_num_min 64 2024-05-09T04:34:58.784 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.785+0000 7f4aeb200640 1 Processor -- start 2024-05-09T04:34:58.784 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.785+0000 7f4aeb200640 1 -- start start 2024-05-09T04:34:58.786 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.787+0000 7f4aeb200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f4aec143590 0x7f4aec15e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:58.786 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.787+0000 7f4aeb200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4aec0b4af0 0x7f4aec0cf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:58.786 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.787+0000 7f4aeb200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f4aec0cfcc0 0x7f4aec160c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:58.786 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.787+0000 7f4aeb200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f4aec0b5ba0 con 0x7f4aec143590 2024-05-09T04:34:58.786 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.787+0000 7f4aeb200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f4aec15e600 con 0x7f4aec0cfcc0 2024-05-09T04:34:58.786 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.787+0000 7f4aeb200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f4aec0b6030 con 0x7f4aec0b4af0 2024-05-09T04:34:58.786 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.787+0000 7f4ae9e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f4aec143590 0x7f4aec15e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:58.786 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.787+0000 7f4ae9e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f4aec143590 0x7f4aec15e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:41422/0 (socket says 172.21.15.145:41422) 2024-05-09T04:34:58.786 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.787+0000 7f4ae9e00640 1 -- 172.21.15.145:0/4232539658 learned_addr learned my addr 172.21.15.145:0/4232539658 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:58.787 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.788+0000 7f4ae9400640 1 --2- 172.21.15.145:0/4232539658 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4aec0b4af0 0x7f4aec0cf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:58.787 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.788+0000 7f4ae9e00640 1 -- 172.21.15.145:0/4232539658 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4aec0b4af0 msgr2=0x7f4aec0cf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:58.787 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.788+0000 7f4ae9e00640 1 --2- 172.21.15.145:0/4232539658 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4aec0b4af0 0x7f4aec0cf780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:58.787 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.788+0000 7f4ae9e00640 1 -- 172.21.15.145:0/4232539658 >> v2:172.21.15.192:3300/0 conn(0x7f4aec0cfcc0 msgr2=0x7f4aec160c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:58.787 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.788+0000 7f4ae9e00640 1 --2- 172.21.15.145:0/4232539658 >> v2:172.21.15.192:3300/0 conn(0x7f4aec0cfcc0 0x7f4aec160c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:58.787 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.788+0000 7f4ae9e00640 1 -- 172.21.15.145:0/4232539658 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aec0b4f90 con 0x7f4aec143590 2024-05-09T04:34:58.788 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.789+0000 7f4ae9e00640 1 --2- 172.21.15.145:0/4232539658 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4aec143590 0x7f4aec15e0c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4aec0ce220 tx=0x7f4adc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15320a7abbd44ff3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:58.788 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.789+0000 7f4ae8a00640 1 -- 172.21.15.145:0/4232539658 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4adc037070 con 0x7f4aec143590 2024-05-09T04:34:58.788 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.789+0000 7f4ae8a00640 1 -- 172.21.15.145:0/4232539658 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4adc032440 con 0x7f4aec143590 2024-05-09T04:34:58.788 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.789+0000 7f4ae8a00640 1 -- 172.21.15.145:0/4232539658 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4adc032810 con 0x7f4aec143590 2024-05-09T04:34:58.789 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.790+0000 7f4aeb200640 1 -- 172.21.15.145:0/4232539658 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4aec143590 msgr2=0x7f4aec15e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:58.789 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.790+0000 7f4aeb200640 1 --2- 172.21.15.145:0/4232539658 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4aec143590 0x7f4aec15e0c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4aec0ce220 tx=0x7f4adc02a1e0 comp rx=0 tx=0).stop 2024-05-09T04:34:58.789 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.790+0000 7f4aeb200640 1 -- 172.21.15.145:0/4232539658 shutdown_connections 2024-05-09T04:34:58.789 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.790+0000 7f4aeb200640 1 --2- 172.21.15.145:0/4232539658 >> v2:172.21.15.192:3300/0 conn(0x7f4aec0cfcc0 0x7f4aec160c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:58.789 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.790+0000 7f4aeb200640 1 --2- 172.21.15.145:0/4232539658 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4aec0b4af0 0x7f4aec0cf780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:58.789 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.790+0000 7f4aeb200640 1 --2- 172.21.15.145:0/4232539658 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4aec143590 0x7f4aec15e0c0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:58.789 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.790+0000 7f4aeb200640 1 -- 172.21.15.145:0/4232539658 >> 172.21.15.145:0/4232539658 conn(0x7f4aec0b2850 msgr2=0x7f4aec0b2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:58.789 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.790+0000 7f4aeb200640 1 -- 172.21.15.145:0/4232539658 shutdown_connections 2024-05-09T04:34:58.789 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.790+0000 7f4aeb200640 1 -- 172.21.15.145:0/4232539658 wait complete. 2024-05-09T04:34:58.790 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.792+0000 7f4aeb200640 1 Processor -- start 2024-05-09T04:34:58.790 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.792+0000 7f4aeb200640 1 -- start start 2024-05-09T04:34:58.792 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.793+0000 7f4aeb200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4aec0b4af0 0x7f4aec142450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:58.792 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.793+0000 7f4aeb200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4aec0cfcc0 0x7f4aec1533b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:58.792 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.793+0000 7f4aeb200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4aec143590 0x7f4aec1e8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:58.792 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.793+0000 7f4aeb200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4aec0b5830 con 0x7f4aec0cfcc0 2024-05-09T04:34:58.792 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.793+0000 7f4aeb200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4aec0b5550 con 0x7f4aec143590 2024-05-09T04:34:58.792 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.793+0000 7f4aeb200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f4aec0cec40 con 0x7f4aec0b4af0 2024-05-09T04:34:58.792 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.793+0000 7f4ae9400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4aec0cfcc0 0x7f4aec1533b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:58.792 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.793+0000 7f4ae9400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4aec0cfcc0 0x7f4aec1533b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:41432/0 (socket says 172.21.15.145:41432) 2024-05-09T04:34:58.793 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.793+0000 7f4ae9400640 1 -- 172.21.15.145:0/1925563954 learned_addr learned my addr 172.21.15.145:0/1925563954 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:58.793 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.793+0000 7f4ae9400640 1 -- 172.21.15.145:0/1925563954 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4aec0b4af0 msgr2=0x7f4aec142450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:58.793 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.793+0000 7f4ae9400640 1 --2- 172.21.15.145:0/1925563954 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4aec0b4af0 0x7f4aec142450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:58.793 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.793+0000 7f4ae9400640 1 -- 172.21.15.145:0/1925563954 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4aec143590 msgr2=0x7f4aec1e8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:58.793 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.793+0000 7f4ae9400640 1 --2- 172.21.15.145:0/1925563954 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4aec143590 0x7f4aec1e8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:58.793 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.793+0000 7f4ae9400640 1 -- 172.21.15.145:0/1925563954 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aec156600 con 0x7f4aec0cfcc0 2024-05-09T04:34:58.794 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.794+0000 7f4ae9400640 1 --2- 172.21.15.145:0/1925563954 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4aec0cfcc0 0x7f4aec1533b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4ad800c820 tx=0x7f4ad800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:58.794 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.794+0000 7f4ae2000640 1 -- 172.21.15.145:0/1925563954 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad8015070 con 0x7f4aec0cfcc0 2024-05-09T04:34:58.794 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.794+0000 7f4ae2000640 1 -- 172.21.15.145:0/1925563954 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ad8004020 con 0x7f4aec0cfcc0 2024-05-09T04:34:58.794 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.794+0000 7f4aeb200640 1 -- 172.21.15.145:0/1925563954 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4aec1568c0 con 0x7f4aec0cfcc0 2024-05-09T04:34:58.794 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.794+0000 7f4ae2000640 1 -- 172.21.15.145:0/1925563954 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad80043f0 con 0x7f4aec0cfcc0 2024-05-09T04:34:58.794 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.794+0000 7f4aeb200640 1 -- 172.21.15.145:0/1925563954 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4aec1e87a0 con 0x7f4aec0cfcc0 2024-05-09T04:34:58.794 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.796+0000 7f4aeb200640 1 -- 172.21.15.145:0/1925563954 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4aec076930 con 0x7f4aec0cfcc0 2024-05-09T04:34:58.795 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.797+0000 7f4ae2000640 1 -- 172.21.15.145:0/1925563954 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4ad8007830 con 0x7f4aec0cfcc0 2024-05-09T04:34:58.797 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.798+0000 7f4ae2000640 1 --2- 172.21.15.145:0/1925563954 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f4acc0762d0 0x7f4acc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:58.797 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.798+0000 7f4ae2000640 1 -- 172.21.15.145:0/1925563954 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f4ad8097350 con 0x7f4aec0cfcc0 2024-05-09T04:34:58.797 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.798+0000 7f4ae9e00640 1 --2- 172.21.15.145:0/1925563954 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f4acc0762d0 0x7f4acc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:58.798 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.799+0000 7f4ae9e00640 1 --2- 172.21.15.145:0/1925563954 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f4acc0762d0 0x7f4acc078790 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4adc034000 tx=0x7f4adc02c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:58.801 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:58.802+0000 7f4ae2000640 1 -- 172.21.15.145:0/1925563954 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f4ad8060d40 con 0x7f4aec0cfcc0 2024-05-09T04:34:59.007 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.009+0000 7f4aeb200640 1 -- 172.21.15.145:0/1925563954 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "data_new", "pg_num_min": 64} v 0) -- 0x7f4aec075400 con 0x7f4aec0cfcc0 2024-05-09T04:34:59.610 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.611+0000 7f4ae2000640 1 -- 172.21.15.145:0/1925563954 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "data_new", "pg_num_min": 64}]=0 pool 'data_new' created v33) ==== 124+0+0 (secure 0 0 0) 0x7f4ad8060640 con 0x7f4aec0cfcc0 2024-05-09T04:34:59.611 INFO:teuthology.orchestra.run.smithi145.stderr:pool 'data_new' created 2024-05-09T04:34:59.614 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.615+0000 7f4aeb200640 1 -- 172.21.15.145:0/1925563954 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f4acc0762d0 msgr2=0x7f4acc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:59.614 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.615+0000 7f4aeb200640 1 --2- 172.21.15.145:0/1925563954 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f4acc0762d0 0x7f4acc078790 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4adc034000 tx=0x7f4adc02c000 comp rx=0 tx=0).stop 2024-05-09T04:34:59.614 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.615+0000 7f4aeb200640 1 -- 172.21.15.145:0/1925563954 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4aec0cfcc0 msgr2=0x7f4aec1533b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:59.614 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.615+0000 7f4aeb200640 1 --2- 172.21.15.145:0/1925563954 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4aec0cfcc0 0x7f4aec1533b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4ad800c820 tx=0x7f4ad800cce0 comp rx=0 tx=0).stop 2024-05-09T04:34:59.615 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.615+0000 7f4aeb200640 1 -- 172.21.15.145:0/1925563954 shutdown_connections 2024-05-09T04:34:59.615 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.615+0000 7f4aeb200640 1 --2- 172.21.15.145:0/1925563954 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f4acc0762d0 0x7f4acc078790 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:59.615 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.615+0000 7f4aeb200640 1 --2- 172.21.15.145:0/1925563954 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4aec143590 0x7f4aec1e8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:59.615 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.615+0000 7f4aeb200640 1 --2- 172.21.15.145:0/1925563954 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4aec0cfcc0 0x7f4aec1533b0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:59.615 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.615+0000 7f4aeb200640 1 --2- 172.21.15.145:0/1925563954 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4aec0b4af0 0x7f4aec142450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:59.615 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.615+0000 7f4aeb200640 1 -- 172.21.15.145:0/1925563954 >> 172.21.15.145:0/1925563954 conn(0x7f4aec0b2850 msgr2=0x7f4aec0d1390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:59.615 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.616+0000 7f4aeb200640 1 -- 172.21.15.145:0/1925563954 shutdown_connections 2024-05-09T04:34:59.615 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.616+0000 7f4aeb200640 1 -- 172.21.15.145:0/1925563954 wait complete. 2024-05-09T04:34:59.643 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs data_new 2024-05-09T04:34:59.833 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.835+0000 7f4da7000640 1 Processor -- start 2024-05-09T04:34:59.834 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.835+0000 7f4da7000640 1 -- start start 2024-05-09T04:34:59.835 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.836+0000 7f4da7000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f4da00b8960 0x7f4da0071250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:59.835 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.836+0000 7f4da7000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4da00cd7c0 0x7f4da00d23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:59.835 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.836+0000 7f4da7000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f4da00d28f0 0x7f4da0153d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:59.835 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.836+0000 7f4da7000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f4da015e490 con 0x7f4da00d28f0 2024-05-09T04:34:59.835 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.836+0000 7f4da7000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f4da015d900 con 0x7f4da00b8960 2024-05-09T04:34:59.835 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.836+0000 7f4da7000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f4da015e920 con 0x7f4da00cd7c0 2024-05-09T04:34:59.836 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.837+0000 7f4da5000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f4da00d28f0 0x7f4da0153d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:59.836 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.837+0000 7f4da5000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f4da00d28f0 0x7f4da0153d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:41440/0 (socket says 172.21.15.145:41440) 2024-05-09T04:34:59.837 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.837+0000 7f4da5000640 1 -- 172.21.15.145:0/2014274809 learned_addr learned my addr 172.21.15.145:0/2014274809 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:59.837 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.837+0000 7f4d9f400640 1 --2- 172.21.15.145:0/2014274809 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4da00cd7c0 0x7f4da00d23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:59.837 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.837+0000 7f4d9fe00640 1 --2- 172.21.15.145:0/2014274809 >> v2:172.21.15.192:3300/0 conn(0x7f4da00b8960 0x7f4da0071250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:59.837 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.838+0000 7f4da5000640 1 -- 172.21.15.145:0/2014274809 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4da00cd7c0 msgr2=0x7f4da00d23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:59.837 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.838+0000 7f4da5000640 1 --2- 172.21.15.145:0/2014274809 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4da00cd7c0 0x7f4da00d23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:59.837 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.838+0000 7f4da5000640 1 -- 172.21.15.145:0/2014274809 >> v2:172.21.15.192:3300/0 conn(0x7f4da00b8960 msgr2=0x7f4da0071250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:34:59.837 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.838+0000 7f4da5000640 1 --2- 172.21.15.145:0/2014274809 >> v2:172.21.15.192:3300/0 conn(0x7f4da00b8960 0x7f4da0071250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:59.837 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.838+0000 7f4da5000640 1 -- 172.21.15.145:0/2014274809 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4da00b8d40 con 0x7f4da00d28f0 2024-05-09T04:34:59.837 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.838+0000 7f4d9fe00640 1 --2- 172.21.15.145:0/2014274809 >> v2:172.21.15.192:3300/0 conn(0x7f4da00b8960 0x7f4da0071250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:34:59.837 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.839+0000 7f4da5000640 1 --2- 172.21.15.145:0/2014274809 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4da00d28f0 0x7f4da0153d30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f4da00d0c00 tx=0x7f4d9802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34e140b78d3b5515 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:59.838 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.839+0000 7f4d9ea00640 1 -- 172.21.15.145:0/2014274809 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d98037070 con 0x7f4da00d28f0 2024-05-09T04:34:59.838 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.839+0000 7f4d9ea00640 1 -- 172.21.15.145:0/2014274809 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d98032440 con 0x7f4da00d28f0 2024-05-09T04:34:59.838 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.839+0000 7f4d9ea00640 1 -- 172.21.15.145:0/2014274809 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d98032810 con 0x7f4da00d28f0 2024-05-09T04:34:59.839 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.840+0000 7f4da7000640 1 -- 172.21.15.145:0/2014274809 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4da00d28f0 msgr2=0x7f4da0153d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:59.839 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.840+0000 7f4da7000640 1 --2- 172.21.15.145:0/2014274809 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4da00d28f0 0x7f4da0153d30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f4da00d0c00 tx=0x7f4d9802a210 comp rx=0 tx=0).stop 2024-05-09T04:34:59.839 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.840+0000 7f4da7000640 1 -- 172.21.15.145:0/2014274809 shutdown_connections 2024-05-09T04:34:59.839 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.840+0000 7f4da7000640 1 --2- 172.21.15.145:0/2014274809 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4da00d28f0 0x7f4da0153d30 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:59.839 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.840+0000 7f4da7000640 1 --2- 172.21.15.145:0/2014274809 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4da00cd7c0 0x7f4da00d23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:59.839 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.840+0000 7f4da7000640 1 --2- 172.21.15.145:0/2014274809 >> v2:172.21.15.192:3300/0 conn(0x7f4da00b8960 0x7f4da0071250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:59.839 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.840+0000 7f4da7000640 1 -- 172.21.15.145:0/2014274809 >> 172.21.15.145:0/2014274809 conn(0x7f4da00d3170 msgr2=0x7f4da00d3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:34:59.839 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.840+0000 7f4da7000640 1 -- 172.21.15.145:0/2014274809 shutdown_connections 2024-05-09T04:34:59.840 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.840+0000 7f4da7000640 1 -- 172.21.15.145:0/2014274809 wait complete. 2024-05-09T04:34:59.841 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.842+0000 7f4da7000640 1 Processor -- start 2024-05-09T04:34:59.841 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.842+0000 7f4da7000640 1 -- start start 2024-05-09T04:34:59.842 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.843+0000 7f4da7000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4da00b8960 0x7f4da015c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:59.842 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.843+0000 7f4da7000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4da00cd7c0 0x7f4da0154fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:59.842 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.843+0000 7f4da7000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4da00d28f0 0x7f4da00cbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:59.842 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.843+0000 7f4da7000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4da0071d30 con 0x7f4da00cd7c0 2024-05-09T04:34:59.842 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.843+0000 7f4da7000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4da00cdf80 con 0x7f4da00b8960 2024-05-09T04:34:59.842 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.843+0000 7f4da7000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f4da015de40 con 0x7f4da00d28f0 2024-05-09T04:34:59.842 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.843+0000 7f4d9f400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4da00cd7c0 0x7f4da0154fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:59.842 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.843+0000 7f4d9f400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4da00cd7c0 0x7f4da0154fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:41456/0 (socket says 172.21.15.145:41456) 2024-05-09T04:34:59.843 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.843+0000 7f4d9f400640 1 -- 172.21.15.145:0/1143692606 learned_addr learned my addr 172.21.15.145:0/1143692606 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:34:59.844 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.844+0000 7f4da5000640 1 --2- 172.21.15.145:0/1143692606 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4da00d28f0 0x7f4da00cbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:59.844 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.844+0000 7f4d9fe00640 1 --2- 172.21.15.145:0/1143692606 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4da00b8960 0x7f4da015c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:59.844 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.844+0000 7f4d9f400640 1 -- 172.21.15.145:0/1143692606 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4da00d28f0 msgr2=0x7f4da00cbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:59.844 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.844+0000 7f4d9f400640 1 --2- 172.21.15.145:0/1143692606 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4da00d28f0 0x7f4da00cbcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:59.844 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.844+0000 7f4d9f400640 1 -- 172.21.15.145:0/1143692606 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4da00b8960 msgr2=0x7f4da015c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:34:59.844 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.844+0000 7f4d9f400640 1 --2- 172.21.15.145:0/1143692606 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4da00b8960 0x7f4da015c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:34:59.844 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.844+0000 7f4d9f400640 1 -- 172.21.15.145:0/1143692606 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4da00cac50 con 0x7f4da00cd7c0 2024-05-09T04:34:59.844 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.844+0000 7f4d9f400640 1 --2- 172.21.15.145:0/1143692606 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4da00cd7c0 0x7f4da0154fb0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4d90007e70 tx=0x7f4d9000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:59.844 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.845+0000 7f4d97e00640 1 -- 172.21.15.145:0/1143692606 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d90014620 con 0x7f4da00cd7c0 2024-05-09T04:34:59.844 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.845+0000 7f4d97e00640 1 -- 172.21.15.145:0/1143692606 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d90019070 con 0x7f4da00cd7c0 2024-05-09T04:34:59.844 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.845+0000 7f4d97e00640 1 -- 172.21.15.145:0/1143692606 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d90014a00 con 0x7f4da00cd7c0 2024-05-09T04:34:59.845 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.845+0000 7f4da7000640 1 -- 172.21.15.145:0/1143692606 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4da00caf30 con 0x7f4da00cd7c0 2024-05-09T04:34:59.845 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.845+0000 7f4da7000640 1 -- 172.21.15.145:0/1143692606 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4da00cb4f0 con 0x7f4da00cd7c0 2024-05-09T04:34:59.845 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.846+0000 7f4da7000640 1 -- 172.21.15.145:0/1143692606 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4da00694e0 con 0x7f4da00cd7c0 2024-05-09T04:34:59.846 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.847+0000 7f4d97e00640 1 -- 172.21.15.145:0/1143692606 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4d90014ba0 con 0x7f4da00cd7c0 2024-05-09T04:34:59.847 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.849+0000 7f4d97e00640 1 --2- 172.21.15.145:0/1143692606 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f4d7c0762d0 0x7f4d7c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:34:59.847 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.849+0000 7f4d97e00640 1 -- 172.21.15.145:0/1143692606 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 6072+0+0 (secure 0 0 0) 0x7f4d90097e80 con 0x7f4da00cd7c0 2024-05-09T04:34:59.848 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.849+0000 7f4d9fe00640 1 --2- 172.21.15.145:0/1143692606 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f4d7c0762d0 0x7f4d7c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:34:59.848 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.850+0000 7f4d9fe00640 1 --2- 172.21.15.145:0/1143692606 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f4d7c0762d0 0x7f4d7c078790 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4d88006fd0 tx=0x7f4d88008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:34:59.853 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:34:59.854+0000 7f4d97e00640 1 -- 172.21.15.145:0/1143692606 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f4d90023020 con 0x7f4da00cd7c0 2024-05-09T04:35:00.104 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.106+0000 7f4da7000640 1 -- 172.21.15.145:0/1143692606 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "data_new"} v 0) -- 0x7f4da00d4080 con 0x7f4da00cd7c0 2024-05-09T04:35:00.637 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.638+0000 7f4d97e00640 1 -- 172.21.15.145:0/1143692606 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "data_new"}]=0 added data pool 6 to fsmap v18) ==== 131+0+0 (secure 0 0 0) 0x7f4d90061660 con 0x7f4da00cd7c0 2024-05-09T04:35:00.638 INFO:teuthology.orchestra.run.smithi145.stderr:added data pool 6 to fsmap 2024-05-09T04:35:00.643 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.644+0000 7f4da7000640 1 -- 172.21.15.145:0/1143692606 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f4d7c0762d0 msgr2=0x7f4d7c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:00.643 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.644+0000 7f4da7000640 1 --2- 172.21.15.145:0/1143692606 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f4d7c0762d0 0x7f4d7c078790 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4d88006fd0 tx=0x7f4d88008040 comp rx=0 tx=0).stop 2024-05-09T04:35:00.643 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.644+0000 7f4da7000640 1 -- 172.21.15.145:0/1143692606 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4da00cd7c0 msgr2=0x7f4da0154fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:00.643 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.644+0000 7f4da7000640 1 --2- 172.21.15.145:0/1143692606 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4da00cd7c0 0x7f4da0154fb0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4d90007e70 tx=0x7f4d9000e4b0 comp rx=0 tx=0).stop 2024-05-09T04:35:00.646 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.645+0000 7f4da7000640 1 -- 172.21.15.145:0/1143692606 shutdown_connections 2024-05-09T04:35:00.646 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.645+0000 7f4da7000640 1 --2- 172.21.15.145:0/1143692606 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f4d7c0762d0 0x7f4d7c078790 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:00.646 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.645+0000 7f4da7000640 1 --2- 172.21.15.145:0/1143692606 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4da00d28f0 0x7f4da00cbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:00.646 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.645+0000 7f4da7000640 1 --2- 172.21.15.145:0/1143692606 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4da00cd7c0 0x7f4da0154fb0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:00.646 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.645+0000 7f4da7000640 1 --2- 172.21.15.145:0/1143692606 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4da00b8960 0x7f4da015c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:00.646 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.645+0000 7f4da7000640 1 -- 172.21.15.145:0/1143692606 >> 172.21.15.145:0/1143692606 conn(0x7f4da00d3170 msgr2=0x7f4da00b3740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:00.646 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.646+0000 7f4da7000640 1 -- 172.21.15.145:0/1143692606 shutdown_connections 2024-05-09T04:35:00.646 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.646+0000 7f4da7000640 1 -- 172.21.15.145:0/1143692606 wait complete. 2024-05-09T04:35:00.667 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T04:35:00.838 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.839+0000 7f5ac7200640 1 Processor -- start 2024-05-09T04:35:00.838 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.839+0000 7f5ac7200640 1 -- start start 2024-05-09T04:35:00.839 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.840+0000 7f5ac7200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f5ac81572c0 0x7f5ac815deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:00.840 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.840+0000 7f5ac7200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f5ac80ae710 0x7f5ac8161220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:00.840 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.840+0000 7f5ac7200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ac80aecd0 con 0x7f5ac80cff30 2024-05-09T04:35:00.840 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.840+0000 7f5ac7200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f5ac80b6290 con 0x7f5ac80ae710 2024-05-09T04:35:00.840 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.840+0000 7f5ac7200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f5ac80aefb0 con 0x7f5ac81572c0 2024-05-09T04:35:00.840 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.841+0000 7f5ac5400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f5ac80ae710 0x7f5ac8161220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:00.840 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.841+0000 7f5ac5400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f5ac80ae710 0x7f5ac8161220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:52986/0 (socket says 172.21.15.145:52986) 2024-05-09T04:35:00.840 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.841+0000 7f5ac5400640 1 -- 172.21.15.145:0/3404479631 learned_addr learned my addr 172.21.15.145:0/3404479631 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:00.840 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.841+0000 7f5ac6800640 1 --1- 172.21.15.145:0/3404479631 >> v1:172.21.15.145:6789/0 conn(0x7f5ac80cff30 0x7f5ac80d0350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:45180/0 (socket says 172.21.15.145:45180) 2024-05-09T04:35:00.841 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.841+0000 7f5ac5400640 1 -- 172.21.15.145:0/3404479631 >> v2:172.21.15.192:3300/0 conn(0x7f5ac81572c0 msgr2=0x7f5ac815deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:00.841 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.841+0000 7f5ac5e00640 1 --2- 172.21.15.145:0/3404479631 >> v2:172.21.15.192:3300/0 conn(0x7f5ac81572c0 0x7f5ac815deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:00.841 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.841+0000 7f5ac5400640 1 --2- 172.21.15.145:0/3404479631 >> v2:172.21.15.192:3300/0 conn(0x7f5ac81572c0 0x7f5ac815deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:00.841 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.841+0000 7f5ac4a00640 1 -- 172.21.15.145:0/3404479631 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3579975727 0 0) 0x7f5ac80aecd0 con 0x7f5ac80cff30 2024-05-09T04:35:00.841 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.841+0000 7f5ac5400640 1 -- 172.21.15.145:0/3404479631 >> v1:172.21.15.145:6789/0 conn(0x7f5ac80cff30 legacy=0x7f5ac80d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:00.841 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.841+0000 7f5ac5400640 1 -- 172.21.15.145:0/3404479631 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ac80af290 con 0x7f5ac80ae710 2024-05-09T04:35:00.841 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.841+0000 7f5ac5e00640 1 --2- 172.21.15.145:0/3404479631 >> v2:172.21.15.192:3300/0 conn(0x7f5ac81572c0 0x7f5ac815deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:35:00.841 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.842+0000 7f5ac5400640 1 --2- 172.21.15.145:0/3404479631 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5ac80ae710 0x7f5ac8161220 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5ac806ffc0 tx=0x7f5ab402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6a47b7ef6c17ed7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:00.842 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.843+0000 7f5ac4a00640 1 -- 172.21.15.145:0/3404479631 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab4037070 con 0x7f5ac80ae710 2024-05-09T04:35:00.843 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.843+0000 7f5ac4a00640 1 -- 172.21.15.145:0/3404479631 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ab4032440 con 0x7f5ac80ae710 2024-05-09T04:35:00.843 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.843+0000 7f5ac4a00640 1 -- 172.21.15.145:0/3404479631 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab4032810 con 0x7f5ac80ae710 2024-05-09T04:35:00.843 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.843+0000 7f5ac7200640 1 -- 172.21.15.145:0/3404479631 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5ac80ae710 msgr2=0x7f5ac8161220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:00.843 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.843+0000 7f5ac7200640 1 --2- 172.21.15.145:0/3404479631 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5ac80ae710 0x7f5ac8161220 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5ac806ffc0 tx=0x7f5ab402a270 comp rx=0 tx=0).stop 2024-05-09T04:35:00.843 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.843+0000 7f5ac7200640 1 -- 172.21.15.145:0/3404479631 shutdown_connections 2024-05-09T04:35:00.843 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.843+0000 7f5ac7200640 1 --2- 172.21.15.145:0/3404479631 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5ac80ae710 0x7f5ac8161220 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:00.843 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.843+0000 7f5ac7200640 1 --2- 172.21.15.145:0/3404479631 >> v2:172.21.15.192:3300/0 conn(0x7f5ac81572c0 0x7f5ac815deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:00.843 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.843+0000 7f5ac7200640 1 -- 172.21.15.145:0/3404479631 >> 172.21.15.145:0/3404479631 conn(0x7f5ac806c8a0 msgr2=0x7f5ac806ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:00.843 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.843+0000 7f5ac7200640 1 -- 172.21.15.145:0/3404479631 shutdown_connections 2024-05-09T04:35:00.843 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.844+0000 7f5ac7200640 1 -- 172.21.15.145:0/3404479631 wait complete. 2024-05-09T04:35:00.844 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.845+0000 7f5ac7200640 1 Processor -- start 2024-05-09T04:35:00.844 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.845+0000 7f5ac7200640 1 -- start start 2024-05-09T04:35:00.846 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.846+0000 7f5ac7200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5ac80ae710 0x7f5ac809c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:00.846 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.846+0000 7f5ac7200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ac80cff30 0x7f5ac809baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:00.846 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.846+0000 7f5ac7200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ac81572c0 0x7f5ac8154590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:00.846 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.846+0000 7f5ac7200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac8161760 con 0x7f5ac80ae710 2024-05-09T04:35:00.846 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.846+0000 7f5ac7200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac8160d00 con 0x7f5ac80cff30 2024-05-09T04:35:00.846 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.846+0000 7f5ac7200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f5ac81618e0 con 0x7f5ac81572c0 2024-05-09T04:35:00.846 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.847+0000 7f5ac5e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5ac80ae710 0x7f5ac809c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:00.846 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.847+0000 7f5ac5e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5ac80ae710 0x7f5ac809c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:52992/0 (socket says 172.21.15.145:52992) 2024-05-09T04:35:00.846 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.847+0000 7f5ac5e00640 1 -- 172.21.15.145:0/2090428567 learned_addr learned my addr 172.21.15.145:0/2090428567 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:00.847 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.847+0000 7f5ac5400640 1 --2- 172.21.15.145:0/2090428567 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ac80cff30 0x7f5ac809baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:00.847 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.847+0000 7f5ac5e00640 1 -- 172.21.15.145:0/2090428567 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ac81572c0 msgr2=0x7f5ac8154590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:00.847 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.847+0000 7f5ac5e00640 1 --2- 172.21.15.145:0/2090428567 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ac81572c0 0x7f5ac8154590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:00.847 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.847+0000 7f5ac5e00640 1 -- 172.21.15.145:0/2090428567 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ac80cff30 msgr2=0x7f5ac809baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:00.847 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.847+0000 7f5ac5e00640 1 --2- 172.21.15.145:0/2090428567 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ac80cff30 0x7f5ac809baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:00.847 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.847+0000 7f5ac5e00640 1 -- 172.21.15.145:0/2090428567 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ac815c880 con 0x7f5ac80ae710 2024-05-09T04:35:00.847 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.847+0000 7f5ac5e00640 1 --2- 172.21.15.145:0/2090428567 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5ac80ae710 0x7f5ac809c390 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5ab800c7f0 tx=0x7f5ab800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:00.848 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.848+0000 7f5abe000640 1 -- 172.21.15.145:0/2090428567 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab8015070 con 0x7f5ac80ae710 2024-05-09T04:35:00.848 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.848+0000 7f5abe000640 1 -- 172.21.15.145:0/2090428567 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ab8007a30 con 0x7f5ac80ae710 2024-05-09T04:35:00.848 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.848+0000 7f5abe000640 1 -- 172.21.15.145:0/2090428567 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab8007e00 con 0x7f5ac80ae710 2024-05-09T04:35:00.848 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.848+0000 7f5ac7200640 1 -- 172.21.15.145:0/2090428567 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ac815cb60 con 0x7f5ac80ae710 2024-05-09T04:35:00.848 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.848+0000 7f5ac7200640 1 -- 172.21.15.145:0/2090428567 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ac809ca20 con 0x7f5ac80ae710 2024-05-09T04:35:00.848 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.849+0000 7f5ac7200640 1 -- 172.21.15.145:0/2090428567 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ac806a310 con 0x7f5ac80ae710 2024-05-09T04:35:00.849 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.850+0000 7f5abe000640 1 -- 172.21.15.145:0/2090428567 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5ab8018070 con 0x7f5ac80ae710 2024-05-09T04:35:00.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.851+0000 7f5abe000640 1 --2- 172.21.15.145:0/2090428567 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f5aa8076300 0x7f5aa80787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:00.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.851+0000 7f5abe000640 1 -- 172.21.15.145:0/2090428567 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 6104+0+0 (secure 0 0 0) 0x7f5ab80976e0 con 0x7f5ac80ae710 2024-05-09T04:35:00.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.852+0000 7f5ac5400640 1 --2- 172.21.15.145:0/2090428567 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f5aa8076300 0x7f5aa80787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:00.851 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.852+0000 7f5ac5400640 1 --2- 172.21.15.145:0/2090428567 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f5aa8076300 0x7f5aa80787c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4034000 tx=0x7f5ab402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:00.855 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:00.856+0000 7f5abe000640 1 -- 172.21.15.145:0/2090428567 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5ab8060eb0 con 0x7f5ac80ae710 2024-05-09T04:35:01.098 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.098+0000 7f5ac7200640 1 -- 172.21.15.145:0/2090428567 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ac80b45d0 con 0x7f5ac80ae710 2024-05-09T04:35:01.099 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.099+0000 7f5abe000640 1 -- 172.21.15.145:0/2090428567 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5189 (secure 0 0 0) 0x7f5ab80607b0 con 0x7f5ac80ae710 2024-05-09T04:35:01.100 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:35:01.100 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4499,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6834/3227728059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3227728059},{"type":"v1","addr":"172.21.15.192:6834","nonce":3227728059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4502,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1603013787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6833","nonce":1603013787},{"type":"v1","addr":"172.21.15.192:6835","nonce":1603013787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4672,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/68778418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":68778418},{"type":"v1","addr":"172.21.15.145:6837","nonce":68778418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T04:34:43.133970+0000","modified":"2024-05-09T04:35:00.107166+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4761},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4761":{"gid":4761,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.145:6835/4260798843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4260798843},{"type":"v1","addr":"172.21.15.145:6835","nonce":4260798843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5,6],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4761,"qdb_cluster":[4761]},"id":2}]} 2024-05-09T04:35:01.100 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 18 2024-05-09T04:35:01.102 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.102+0000 7f5ac7200640 1 -- 172.21.15.145:0/2090428567 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f5aa8076300 msgr2=0x7f5aa80787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:01.102 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.102+0000 7f5ac7200640 1 --2- 172.21.15.145:0/2090428567 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f5aa8076300 0x7f5aa80787c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4034000 tx=0x7f5ab402e000 comp rx=0 tx=0).stop 2024-05-09T04:35:01.102 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.102+0000 7f5ac7200640 1 -- 172.21.15.145:0/2090428567 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5ac80ae710 msgr2=0x7f5ac809c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:01.102 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.102+0000 7f5ac7200640 1 --2- 172.21.15.145:0/2090428567 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5ac80ae710 0x7f5ac809c390 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5ab800c7f0 tx=0x7f5ab800cbf0 comp rx=0 tx=0).stop 2024-05-09T04:35:01.102 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.102+0000 7f5ac7200640 1 -- 172.21.15.145:0/2090428567 shutdown_connections 2024-05-09T04:35:01.103 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.102+0000 7f5ac7200640 1 --2- 172.21.15.145:0/2090428567 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f5aa8076300 0x7f5aa80787c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:01.103 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.102+0000 7f5ac7200640 1 --2- 172.21.15.145:0/2090428567 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ac81572c0 0x7f5ac8154590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:01.103 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.102+0000 7f5ac7200640 1 --2- 172.21.15.145:0/2090428567 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ac80cff30 0x7f5ac809baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:01.103 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.102+0000 7f5ac7200640 1 --2- 172.21.15.145:0/2090428567 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5ac80ae710 0x7f5ac809c390 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:01.103 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.102+0000 7f5ac7200640 1 -- 172.21.15.145:0/2090428567 >> 172.21.15.145:0/2090428567 conn(0x7f5ac806c8a0 msgr2=0x7f5ac81551a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:01.103 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.103+0000 7f5ac7200640 1 -- 172.21.15.145:0/2090428567 shutdown_connections 2024-05-09T04:35:01.103 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.103+0000 7f5ac7200640 1 -- 172.21.15.145:0/2090428567 wait complete. 2024-05-09T04:35:01.120 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T04:35:01.316 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.316+0000 7f225fe00640 1 Processor -- start 2024-05-09T04:35:01.317 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.317+0000 7f225fe00640 1 -- start start 2024-05-09T04:35:01.318 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.318+0000 7f225fe00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f22600d3180 0x7f22600d5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:01.318 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.318+0000 7f225fe00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f22600b4760 0x7f22600b4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:01.318 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.318+0000 7f225fe00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22600d3e70 con 0x7f22600d22f0 2024-05-09T04:35:01.318 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.318+0000 7f225fe00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f22600d4150 con 0x7f22600b4760 2024-05-09T04:35:01.318 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.318+0000 7f225fe00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f22600b6200 con 0x7f22600d3180 2024-05-09T04:35:01.318 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.318+0000 7f225f400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f22600b4760 0x7f22600b4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:01.318 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.318+0000 7f225f400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f22600b4760 0x7f22600b4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:52994/0 (socket says 172.21.15.145:52994) 2024-05-09T04:35:01.318 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.318+0000 7f225f400640 1 -- 172.21.15.145:0/4030057922 learned_addr learned my addr 172.21.15.145:0/4030057922 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:01.318 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.318+0000 7f225ea00640 1 --2- 172.21.15.145:0/4030057922 >> v2:172.21.15.192:3300/0 conn(0x7f22600d3180 0x7f22600d5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:01.319 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.319+0000 7f225f400640 1 -- 172.21.15.145:0/4030057922 >> v2:172.21.15.192:3300/0 conn(0x7f22600d3180 msgr2=0x7f22600d5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:01.320 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.319+0000 7f225f400640 1 --2- 172.21.15.145:0/4030057922 >> v2:172.21.15.192:3300/0 conn(0x7f22600d3180 0x7f22600d5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:01.320 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.319+0000 7f225f400640 1 -- 172.21.15.145:0/4030057922 >> v1:172.21.15.192:6789/0 conn(0x7f22600d22f0 legacy=0x7f22600d3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:01.320 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.319+0000 7f225f400640 1 -- 172.21.15.145:0/4030057922 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22600b6450 con 0x7f22600b4760 2024-05-09T04:35:01.320 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.320+0000 7f225f400640 1 --2- 172.21.15.145:0/4030057922 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f22600b4760 0x7f22600b4bd0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f226008dad0 tx=0x7f225802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17dd41c61e9daf5d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:01.320 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.320+0000 7f225d600640 1 -- 172.21.15.145:0/4030057922 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2258037070 con 0x7f22600b4760 2024-05-09T04:35:01.320 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.320+0000 7f225d600640 1 -- 172.21.15.145:0/4030057922 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2258032440 con 0x7f22600b4760 2024-05-09T04:35:01.320 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.320+0000 7f225d600640 1 -- 172.21.15.145:0/4030057922 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2258032810 con 0x7f22600b4760 2024-05-09T04:35:01.321 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.321+0000 7f225fe00640 1 -- 172.21.15.145:0/4030057922 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f22600b4760 msgr2=0x7f22600b4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:01.321 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.321+0000 7f225fe00640 1 --2- 172.21.15.145:0/4030057922 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f22600b4760 0x7f22600b4bd0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f226008dad0 tx=0x7f225802a140 comp rx=0 tx=0).stop 2024-05-09T04:35:01.321 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.321+0000 7f225fe00640 1 -- 172.21.15.145:0/4030057922 shutdown_connections 2024-05-09T04:35:01.321 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.321+0000 7f225fe00640 1 --2- 172.21.15.145:0/4030057922 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f22600b4760 0x7f22600b4bd0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:01.321 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.321+0000 7f225fe00640 1 --2- 172.21.15.145:0/4030057922 >> v2:172.21.15.192:3300/0 conn(0x7f22600d3180 0x7f22600d5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:01.321 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.321+0000 7f225fe00640 1 -- 172.21.15.145:0/4030057922 >> 172.21.15.145:0/4030057922 conn(0x7f22600d4fb0 msgr2=0x7f22600d53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:01.321 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.321+0000 7f225fe00640 1 -- 172.21.15.145:0/4030057922 shutdown_connections 2024-05-09T04:35:01.321 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.321+0000 7f225fe00640 1 -- 172.21.15.145:0/4030057922 wait complete. 2024-05-09T04:35:01.322 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.322+0000 7f225fe00640 1 Processor -- start 2024-05-09T04:35:01.322 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.323+0000 7f225fe00640 1 -- start start 2024-05-09T04:35:01.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.323+0000 7f225fe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f22600b4760 0x7f2260153270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:01.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.324+0000 7f225fe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22600d22f0 0x7f22600ca0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:01.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.324+0000 7f225fe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22600d3180 0x7f22600ca600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:01.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.324+0000 7f225fe00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f22600d2780 con 0x7f22600b4760 2024-05-09T04:35:01.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.324+0000 7f225fe00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f22600d1e60 con 0x7f22600d3180 2024-05-09T04:35:01.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.324+0000 7f225fe00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f22600b3fa0 con 0x7f22600d22f0 2024-05-09T04:35:01.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.324+0000 7f225ea00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f22600b4760 0x7f2260153270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:01.325 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.324+0000 7f225ea00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f22600b4760 0x7f2260153270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:53002/0 (socket says 172.21.15.145:53002) 2024-05-09T04:35:01.325 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.324+0000 7f225ea00640 1 -- 172.21.15.145:0/656695226 learned_addr learned my addr 172.21.15.145:0/656695226 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:01.325 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.324+0000 7f225ea00640 1 -- 172.21.15.145:0/656695226 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22600d22f0 msgr2=0x7f22600ca0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:01.325 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.324+0000 7f225ea00640 1 --2- 172.21.15.145:0/656695226 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22600d22f0 0x7f22600ca0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:01.325 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.324+0000 7f225ea00640 1 -- 172.21.15.145:0/656695226 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22600d3180 msgr2=0x7f22600ca600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:01.325 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.324+0000 7f225ea00640 1 --2- 172.21.15.145:0/656695226 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22600d3180 0x7f22600ca600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:01.325 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.324+0000 7f225ea00640 1 -- 172.21.15.145:0/656695226 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22600ceea0 con 0x7f22600b4760 2024-05-09T04:35:01.325 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.325+0000 7f225ea00640 1 --2- 172.21.15.145:0/656695226 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f22600b4760 0x7f2260153270 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f225000d5f0 tx=0x7f225000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:01.326 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.325+0000 7f2256a00640 1 -- 172.21.15.145:0/656695226 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22500125f0 con 0x7f22600b4760 2024-05-09T04:35:01.326 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.325+0000 7f2256a00640 1 -- 172.21.15.145:0/656695226 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2250019070 con 0x7f22600b4760 2024-05-09T04:35:01.326 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.325+0000 7f2256a00640 1 -- 172.21.15.145:0/656695226 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22500129d0 con 0x7f22600b4760 2024-05-09T04:35:01.326 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.325+0000 7f225fe00640 1 -- 172.21.15.145:0/656695226 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22600cb380 con 0x7f22600b4760 2024-05-09T04:35:01.326 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.325+0000 7f225fe00640 1 -- 172.21.15.145:0/656695226 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f226015bba0 con 0x7f22600b4760 2024-05-09T04:35:01.326 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.326+0000 7f225fe00640 1 -- 172.21.15.145:0/656695226 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f226015b6f0 con 0x7f22600b4760 2024-05-09T04:35:01.327 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.327+0000 7f2256a00640 1 -- 172.21.15.145:0/656695226 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2250012b70 con 0x7f22600b4760 2024-05-09T04:35:01.328 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.328+0000 7f2256a00640 1 --2- 172.21.15.145:0/656695226 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f223c076300 0x7f223c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:01.328 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.328+0000 7f2256a00640 1 -- 172.21.15.145:0/656695226 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 6104+0+0 (secure 0 0 0) 0x7f2250097c30 con 0x7f22600b4760 2024-05-09T04:35:01.328 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.328+0000 7f225e000640 1 --2- 172.21.15.145:0/656695226 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f223c076300 0x7f223c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:01.328 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.329+0000 7f225e000640 1 --2- 172.21.15.145:0/656695226 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f223c076300 0x7f223c0787c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f22600cbd00 tx=0x7f2248007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:01.332 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.333+0000 7f2256a00640 1 -- 172.21.15.145:0/656695226 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2250061360 con 0x7f22600b4760 2024-05-09T04:35:01.524 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.524+0000 7f225fe00640 1 -- 172.21.15.145:0/656695226 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f226006a310 con 0x7f22600b4760 2024-05-09T04:35:01.526 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.527+0000 7f2256a00640 1 -- 172.21.15.145:0/656695226 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+18542 (secure 0 0 0) 0x7f2250060c60 con 0x7f22600b4760 2024-05-09T04:35:01.527 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:35:01.527 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":34,"fsid":"957ee6c1-073b-44cf-9b93-9491b3fdfc1d","created":"2024-05-09T04:33:26.544367+0000","modified":"2024-05-09T04:35:00.106877+0000","last_up_change":"2024-05-09T04:33:39.039318+0000","last_in_change":"2024-05-09T04:33:29.902361+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T04:33:39.795344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-09T04:34:40.995268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-09T04:34:42.107773+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"data_new","create_time":"2024-05-09T04:34:59.021572+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22e550ad-847b-4952-80e6-63d859ed1a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6801","nonce":3182572912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6803","nonce":3182572912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6807","nonce":3182572912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6805","nonce":3182572912}]},"public_addr":"172.21.15.145:6801/3182572912","cluster_addr":"172.21.15.145:6803/3182572912","heartbeat_back_addr":"172.21.15.145:6807/3182572912","heartbeat_front_addr":"172.21.15.145:6805/3182572912","state":["exists","up"]},{"osd":1,"uuid":"0f6b9fca-0c41-4145-8d74-598ddb42f2a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6825","nonce":1845225555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6827","nonce":1845225555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6831","nonce":1845225555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6829","nonce":1845225555}]},"public_addr":"172.21.15.145:6825/1845225555","cluster_addr":"172.21.15.145:6827/1845225555","heartbeat_back_addr":"172.21.15.145:6831/1845225555","heartbeat_front_addr":"172.21.15.145:6829/1845225555","state":["exists","up"]},{"osd":2,"uuid":"b3b1f67e-5790-483f-8610-e05a5bce4c30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6817","nonce":2168958993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6819","nonce":2168958993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6823","nonce":2168958993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6821","nonce":2168958993}]},"public_addr":"172.21.15.145:6817/2168958993","cluster_addr":"172.21.15.145:6819/2168958993","heartbeat_back_addr":"172.21.15.145:6823/2168958993","heartbeat_front_addr":"172.21.15.145:6821/2168958993","state":["exists","up"]},{"osd":3,"uuid":"884787a5-e037-4d31-9aae-743030eba4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6809","nonce":337009826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6811","nonce":337009826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6815","nonce":337009826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6813","nonce":337009826}]},"public_addr":"172.21.15.145:6809/337009826","cluster_addr":"172.21.15.145:6811/337009826","heartbeat_back_addr":"172.21.15.145:6815/337009826","heartbeat_front_addr":"172.21.15.145:6813/337009826","state":["exists","up"]},{"osd":4,"uuid":"0759fc1e-c5fc-4a2f-86bb-e8eccd83f833","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6818","nonce":3117700995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6821","nonce":3117700995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6827","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6829","nonce":3117700995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6823","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6825","nonce":3117700995}]},"public_addr":"172.21.15.192:6818/3117700995","cluster_addr":"172.21.15.192:6821/3117700995","heartbeat_back_addr":"172.21.15.192:6829/3117700995","heartbeat_front_addr":"172.21.15.192:6825/3117700995","state":["exists","up"]},{"osd":5,"uuid":"9443cc22-9379-46e3-9173-98b592bf2c16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6801","nonce":1328373268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6803","nonce":1328373268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6807","nonce":1328373268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6805","nonce":1328373268}]},"public_addr":"172.21.15.192:6801/1328373268","cluster_addr":"172.21.15.192:6803/1328373268","heartbeat_back_addr":"172.21.15.192:6807/1328373268","heartbeat_front_addr":"172.21.15.192:6805/1328373268","state":["exists","up"]},{"osd":6,"uuid":"d05a3311-505a-45b5-85b9-aa3113e96987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6817","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6820","nonce":1868011057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6824","nonce":1868011057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6831","nonce":1868011057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6828","nonce":1868011057}]},"public_addr":"172.21.15.192:6820/1868011057","cluster_addr":"172.21.15.192:6824/1868011057","heartbeat_back_addr":"172.21.15.192:6831/1868011057","heartbeat_front_addr":"172.21.15.192:6828/1868011057","state":["exists","up"]},{"osd":7,"uuid":"c7c23dbc-2029-43ec-9f53-25f3d58019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6809","nonce":3368512157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6811","nonce":3368512157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6815","nonce":3368512157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6813","nonce":3368512157}]},"public_addr":"172.21.15.192:6809/3368512157","cluster_addr":"172.21.15.192:6811/3368512157","heartbeat_back_addr":"172.21.15.192:6815/3368512157","heartbeat_front_addr":"172.21.15.192:6813/3368512157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.091188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.032567+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:34.797468+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.228839+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.622400+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.473103+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.566092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.349395+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T04:35:01.529 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.529+0000 7f225fe00640 1 -- 172.21.15.145:0/656695226 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f223c076300 msgr2=0x7f223c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:01.529 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.529+0000 7f225fe00640 1 --2- 172.21.15.145:0/656695226 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f223c076300 0x7f223c0787c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f22600cbd00 tx=0x7f2248007d30 comp rx=0 tx=0).stop 2024-05-09T04:35:01.529 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.529+0000 7f225fe00640 1 -- 172.21.15.145:0/656695226 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f22600b4760 msgr2=0x7f2260153270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:01.529 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.529+0000 7f225fe00640 1 --2- 172.21.15.145:0/656695226 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f22600b4760 0x7f2260153270 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f225000d5f0 tx=0x7f225000dab0 comp rx=0 tx=0).stop 2024-05-09T04:35:01.530 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.530+0000 7f225fe00640 1 -- 172.21.15.145:0/656695226 shutdown_connections 2024-05-09T04:35:01.530 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.530+0000 7f225fe00640 1 --2- 172.21.15.145:0/656695226 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f223c076300 0x7f223c0787c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:01.530 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.530+0000 7f225fe00640 1 --2- 172.21.15.145:0/656695226 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22600d3180 0x7f22600ca600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:01.530 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.530+0000 7f225fe00640 1 --2- 172.21.15.145:0/656695226 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22600d22f0 0x7f22600ca0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:01.530 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.530+0000 7f225fe00640 1 --2- 172.21.15.145:0/656695226 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f22600b4760 0x7f2260153270 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:01.531 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.530+0000 7f225fe00640 1 -- 172.21.15.145:0/656695226 >> 172.21.15.145:0/656695226 conn(0x7f22600d4fb0 msgr2=0x7f22600b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:01.531 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.530+0000 7f225fe00640 1 -- 172.21.15.145:0/656695226 shutdown_connections 2024-05-09T04:35:01.531 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.530+0000 7f225fe00640 1 -- 172.21.15.145:0/656695226 wait complete. 2024-05-09T04:35:01.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:35:01.548 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.file.layout.pool -v data_new ./parent_b/alpha) 2024-05-09T04:35:01.615 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T04:35:01.803 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.803+0000 7f7593200640 1 Processor -- start 2024-05-09T04:35:01.803 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.804+0000 7f7593200640 1 -- start start 2024-05-09T04:35:01.807 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.805+0000 7f7593200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f75940d3180 0x7f75940d5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:01.807 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.805+0000 7f7593200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75940d3e70 con 0x7f75940d22f0 2024-05-09T04:35:01.807 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.805+0000 7f7593200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75940b6450 con 0x7f75940b4760 2024-05-09T04:35:01.807 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.805+0000 7f7593200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f75940d4150 con 0x7f75940d3180 2024-05-09T04:35:01.807 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.805+0000 7f7591400640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f75940d22f0 0x7f75940d3560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:45182/0 (socket says 172.21.15.145:45182) 2024-05-09T04:35:01.807 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.805+0000 7f7591400640 1 -- 172.21.15.145:0/246592295 learned_addr learned my addr 172.21.15.145:0/246592295 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:35:01.808 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.806+0000 7f7591e00640 1 --2- 172.21.15.145:0/246592295 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f75940d3180 0x7f75940d5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:01.808 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.806+0000 7f7590a00640 1 -- 172.21.15.145:0/246592295 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 95384079 0 0) 0x7f75940d3e70 con 0x7f75940d22f0 2024-05-09T04:35:01.808 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.806+0000 7f7590a00640 1 -- 172.21.15.145:0/246592295 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7578003520 con 0x7f75940d22f0 2024-05-09T04:35:01.808 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.806+0000 7f7590a00640 1 -- 172.21.15.145:0/246592295 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3395434250 0 0) 0x7f7578003520 con 0x7f75940d22f0 2024-05-09T04:35:01.808 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.806+0000 7f7590a00640 1 -- 172.21.15.145:0/246592295 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f75940d3180 msgr2=0x7f75940d5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:01.808 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.806+0000 7f7590a00640 1 --2- 172.21.15.145:0/246592295 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f75940d3180 0x7f75940d5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:01.808 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.806+0000 7f7590a00640 1 -- 172.21.15.145:0/246592295 >> v1:172.21.15.192:6789/0 conn(0x7f75940b4760 legacy=0x7f75940b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:01.808 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.806+0000 7f7590a00640 1 -- 172.21.15.145:0/246592295 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75940b6730 con 0x7f75940d22f0 2024-05-09T04:35:01.808 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.806+0000 7f7590a00640 1 -- 172.21.15.145:0/246592295 <== mon.0 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3474413261 0 0) 0x7f75800016f0 con 0x7f75940d22f0 2024-05-09T04:35:01.808 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.806+0000 7f7590a00640 1 -- 172.21.15.145:0/246592295 >> v1:172.21.15.145:6789/0 conn(0x7f75940d22f0 legacy=0x7f75940d3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:01.808 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.806+0000 7f7590a00640 1 --2- 172.21.15.145:0/246592295 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7578003a80 0x7f7578003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:01.808 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.806+0000 7f7590a00640 1 --2- 172.21.15.145:0/246592295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7578004800 0x7f7578004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:01.808 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.806+0000 7f7590a00640 1 --2- 172.21.15.145:0/246592295 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75780052e0 0x7f757800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:01.808 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.806+0000 7f7590a00640 1 -- 172.21.15.145:0/246592295 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f757800c1d0 con 0x7f75780052e0 2024-05-09T04:35:01.808 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.806+0000 7f7590a00640 1 -- 172.21.15.145:0/246592295 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f757800c380 con 0x7f7578004800 2024-05-09T04:35:01.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.806+0000 7f7590a00640 1 -- 172.21.15.145:0/246592295 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f757800c580 con 0x7f7578003a80 2024-05-09T04:35:01.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.806+0000 7f7591e00640 1 --2- 172.21.15.145:0/246592295 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f75940d3180 0x7f75940d5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:35:01.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.807+0000 7f7592800640 1 --2- 172.21.15.145:0/246592295 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75780052e0 0x7f757800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:01.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.807+0000 7f7591e00640 1 --2- 172.21.15.145:0/246592295 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7578003a80 0x7f7578003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:01.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.807+0000 7f7591400640 1 --2- 172.21.15.145:0/246592295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7578004800 0x7f7578004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:01.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.807+0000 7f7592800640 1 -- 172.21.15.145:0/246592295 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7578003a80 msgr2=0x7f7578003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:01.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.807+0000 7f7592800640 1 --2- 172.21.15.145:0/246592295 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7578003a80 0x7f7578003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:01.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.807+0000 7f7592800640 1 -- 172.21.15.145:0/246592295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7578004800 msgr2=0x7f7578004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:01.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.807+0000 7f7592800640 1 --2- 172.21.15.145:0/246592295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7578004800 0x7f7578004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:01.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.807+0000 7f7592800640 1 -- 172.21.15.145:0/246592295 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7578003520 con 0x7f75780052e0 2024-05-09T04:35:01.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.807+0000 7f7591e00640 1 --2- 172.21.15.145:0/246592295 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7578003a80 0x7f7578003ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:35:01.811 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.808+0000 7f7592800640 1 --2- 172.21.15.145:0/246592295 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75780052e0 0x7f757800b9c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f758c001e90 tx=0x7f7578005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f039ddc28d386a45 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:01.811 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.808+0000 7f7590a00640 1 -- 172.21.15.145:0/246592295 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758c001ab0 con 0x7f75780052e0 2024-05-09T04:35:01.811 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.808+0000 7f7590a00640 1 -- 172.21.15.145:0/246592295 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f758c02ae10 con 0x7f75780052e0 2024-05-09T04:35:01.811 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.808+0000 7f7590a00640 1 -- 172.21.15.145:0/246592295 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758c003b70 con 0x7f75780052e0 2024-05-09T04:35:01.811 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.809+0000 7f7593200640 1 -- 172.21.15.145:0/246592295 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75780052e0 msgr2=0x7f757800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:01.811 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.809+0000 7f7593200640 1 --2- 172.21.15.145:0/246592295 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f75780052e0 0x7f757800b9c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f758c001e90 tx=0x7f7578005830 comp rx=0 tx=0).stop 2024-05-09T04:35:01.811 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.809+0000 7f7592800640 1 -- 172.21.15.145:0/246592295 reap_dead start 2024-05-09T04:35:01.811 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.809+0000 7f7593200640 1 -- 172.21.15.145:0/246592295 shutdown_connections 2024-05-09T04:35:01.811 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.809+0000 7f7593200640 1 -- 172.21.15.145:0/246592295 >> 172.21.15.145:0/246592295 conn(0x7f75940d4fb0 msgr2=0x7f75940d53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:01.811 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.809+0000 7f7593200640 1 -- 172.21.15.145:0/246592295 shutdown_connections 2024-05-09T04:35:01.811 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.809+0000 7f7593200640 1 -- 172.21.15.145:0/246592295 wait complete. 2024-05-09T04:35:01.812 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.811+0000 7f7593200640 1 Processor -- start 2024-05-09T04:35:01.812 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.811+0000 7f7593200640 1 -- start start 2024-05-09T04:35:01.813 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.812+0000 7f7593200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f759414c4c0 0x7f759415dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:01.813 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.812+0000 7f7593200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f75940ce900 0x7f759415c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:01.813 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.812+0000 7f7593200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f759415adb0 0x7f759415b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:01.814 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.812+0000 7f7593200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f75940d1e60 con 0x7f759414c4c0 2024-05-09T04:35:01.814 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.812+0000 7f7593200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f75940b6200 con 0x7f75940ce900 2024-05-09T04:35:01.814 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.812+0000 7f7593200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f75940b52e0 con 0x7f759415adb0 2024-05-09T04:35:01.814 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.812+0000 7f7591e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f759414c4c0 0x7f759415dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:01.814 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.812+0000 7f7591e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f759414c4c0 0x7f759415dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:53020/0 (socket says 172.21.15.145:53020) 2024-05-09T04:35:01.814 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.812+0000 7f7591e00640 1 -- 172.21.15.145:0/2243918569 learned_addr learned my addr 172.21.15.145:0/2243918569 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:01.814 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.812+0000 7f7591e00640 1 -- 172.21.15.145:0/2243918569 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f759415adb0 msgr2=0x7f759415b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:01.814 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.812+0000 7f7591e00640 1 --2- 172.21.15.145:0/2243918569 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f759415adb0 0x7f759415b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:01.814 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.812+0000 7f7591e00640 1 -- 172.21.15.145:0/2243918569 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f75940ce900 msgr2=0x7f759415c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:01.814 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.812+0000 7f7591e00640 1 --2- 172.21.15.145:0/2243918569 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f75940ce900 0x7f759415c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:01.814 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.812+0000 7f7591e00640 1 -- 172.21.15.145:0/2243918569 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f759414c8a0 con 0x7f759414c4c0 2024-05-09T04:35:01.814 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.812+0000 7f7591e00640 1 --2- 172.21.15.145:0/2243918569 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f759414c4c0 0x7f759415dfd0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f758400f4f0 tx=0x7f758400f9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:01.815 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.813+0000 7f758a000640 1 -- 172.21.15.145:0/2243918569 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7584018020 con 0x7f759414c4c0 2024-05-09T04:35:01.815 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.813+0000 7f758a000640 1 -- 172.21.15.145:0/2243918569 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7584014890 con 0x7f759414c4c0 2024-05-09T04:35:01.815 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.813+0000 7f758a000640 1 -- 172.21.15.145:0/2243918569 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7584014c60 con 0x7f759414c4c0 2024-05-09T04:35:01.815 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.813+0000 7f7593200640 1 -- 172.21.15.145:0/2243918569 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f759415cf30 con 0x7f759414c4c0 2024-05-09T04:35:01.815 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.813+0000 7f7593200640 1 -- 172.21.15.145:0/2243918569 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f759415d4f0 con 0x7f759414c4c0 2024-05-09T04:35:01.815 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.814+0000 7f7593200640 1 -- 172.21.15.145:0/2243918569 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f759406a310 con 0x7f759414c4c0 2024-05-09T04:35:01.815 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.815+0000 7f758a000640 1 -- 172.21.15.145:0/2243918569 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f758401b020 con 0x7f759414c4c0 2024-05-09T04:35:01.815 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.816+0000 7f758a000640 1 --2- 172.21.15.145:0/2243918569 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f7574076300 0x7f75740787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:01.816 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.816+0000 7f758a000640 1 -- 172.21.15.145:0/2243918569 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 6104+0+0 (secure 0 0 0) 0x7f7584097d00 con 0x7f759414c4c0 2024-05-09T04:35:01.816 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.816+0000 7f7591400640 1 --2- 172.21.15.145:0/2243918569 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f7574076300 0x7f75740787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:01.816 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.817+0000 7f7591400640 1 --2- 172.21.15.145:0/2243918569 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f7574076300 0x7f75740787c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f75941f89e0 tx=0x7f7580009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:01.820 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:01.821+0000 7f758a000640 1 -- 172.21.15.145:0/2243918569 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f7584022020 con 0x7f759414c4c0 2024-05-09T04:35:02.047 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.047+0000 7f7593200640 1 -- 172.21.15.145:0/2243918569 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f75940bae10 con 0x7f759414c4c0 2024-05-09T04:35:02.049 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.049+0000 7f758a000640 1 -- 172.21.15.145:0/2243918569 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5189 (secure 0 0 0) 0x7f75840613a0 con 0x7f759414c4c0 2024-05-09T04:35:02.049 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:35:02.049 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4499,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6834/3227728059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3227728059},{"type":"v1","addr":"172.21.15.192:6834","nonce":3227728059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4502,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1603013787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6833","nonce":1603013787},{"type":"v1","addr":"172.21.15.192:6835","nonce":1603013787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4672,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/68778418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":68778418},{"type":"v1","addr":"172.21.15.145:6837","nonce":68778418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T04:34:43.133970+0000","modified":"2024-05-09T04:35:00.107166+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4761},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4761":{"gid":4761,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.145:6835/4260798843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4260798843},{"type":"v1","addr":"172.21.15.145:6835","nonce":4260798843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5,6],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4761,"qdb_cluster":[4761]},"id":2}]} 2024-05-09T04:35:02.049 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 18 2024-05-09T04:35:02.051 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.051+0000 7f7593200640 1 -- 172.21.15.145:0/2243918569 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f7574076300 msgr2=0x7f75740787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:02.051 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.051+0000 7f7593200640 1 --2- 172.21.15.145:0/2243918569 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f7574076300 0x7f75740787c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f75941f89e0 tx=0x7f7580009040 comp rx=0 tx=0).stop 2024-05-09T04:35:02.051 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.051+0000 7f7593200640 1 -- 172.21.15.145:0/2243918569 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f759414c4c0 msgr2=0x7f759415dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:02.052 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.051+0000 7f7593200640 1 --2- 172.21.15.145:0/2243918569 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f759414c4c0 0x7f759415dfd0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f758400f4f0 tx=0x7f758400f9b0 comp rx=0 tx=0).stop 2024-05-09T04:35:02.052 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.051+0000 7f7593200640 1 -- 172.21.15.145:0/2243918569 shutdown_connections 2024-05-09T04:35:02.052 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.051+0000 7f7593200640 1 --2- 172.21.15.145:0/2243918569 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f7574076300 0x7f75740787c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:02.052 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.051+0000 7f7593200640 1 --2- 172.21.15.145:0/2243918569 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f759415adb0 0x7f759415b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:02.052 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.051+0000 7f7593200640 1 --2- 172.21.15.145:0/2243918569 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f75940ce900 0x7f759415c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:02.052 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.051+0000 7f7593200640 1 --2- 172.21.15.145:0/2243918569 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f759414c4c0 0x7f759415dfd0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:02.052 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.051+0000 7f7593200640 1 -- 172.21.15.145:0/2243918569 >> 172.21.15.145:0/2243918569 conn(0x7f75940d4fb0 msgr2=0x7f75940b55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:02.052 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.052+0000 7f7593200640 1 -- 172.21.15.145:0/2243918569 shutdown_connections 2024-05-09T04:35:02.052 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.052+0000 7f7593200640 1 -- 172.21.15.145:0/2243918569 wait complete. 2024-05-09T04:35:02.070 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-05-09T04:35:02.286 INFO:teuthology.orchestra.run.smithi145.stdout:{"message":"","return_code":0} 2024-05-09T04:35:02.286 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-09T04:35:02.287 DEBUG:teuthology.orchestra.run.smithi145:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001fb.00000000 parent 2024-05-09T04:35:02.390 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.390+0000 7fe71f48cf40 1 Processor -- start 2024-05-09T04:35:02.390 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.390+0000 7fe71f48cf40 1 -- start start 2024-05-09T04:35:02.392 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.391+0000 7fe71f48cf40 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55b602a72b90 0x55b602b8c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:02.392 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.391+0000 7fe71f48cf40 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b602a734b0 0x55b602aed460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:02.392 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.391+0000 7fe71f48cf40 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b602af80d0 0x55b602a575d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:02.392 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.391+0000 7fe71f48cf40 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55b602a71f10 con 0x55b602a734b0 2024-05-09T04:35:02.392 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.391+0000 7fe71f48cf40 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55b6029cdf50 con 0x55b602af80d0 2024-05-09T04:35:02.392 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.391+0000 7fe71f48cf40 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x55b602a73a30 con 0x55b602a72b90 2024-05-09T04:35:02.392 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.391+0000 7fe713c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b602a734b0 0x55b602aed460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:02.392 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.391+0000 7fe713c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b602a734b0 0x55b602aed460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:53026/0 (socket says 172.21.15.145:53026) 2024-05-09T04:35:02.392 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.391+0000 7fe713c00640 1 -- 172.21.15.145:0/349206573 learned_addr learned my addr 172.21.15.145:0/349206573 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:02.392 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.392+0000 7fe71d000640 1 --2- 172.21.15.145:0/349206573 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b602af80d0 0x55b602a575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:02.392 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.392+0000 7fe713c00640 1 -- 172.21.15.145:0/349206573 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55b602a72b90 msgr2=0x55b602b8c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:02.392 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.392+0000 7fe71c600640 1 --2- 172.21.15.145:0/349206573 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55b602a72b90 0x55b602b8c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:02.392 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.392+0000 7fe713c00640 1 --2- 172.21.15.145:0/349206573 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55b602a72b90 0x55b602b8c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:02.392 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.392+0000 7fe713c00640 1 -- 172.21.15.145:0/349206573 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b602af80d0 msgr2=0x55b602a575d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:02.393 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.392+0000 7fe713c00640 1 --2- 172.21.15.145:0/349206573 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b602af80d0 0x55b602a575d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:02.393 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.392+0000 7fe713c00640 1 -- 172.21.15.145:0/349206573 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b602950b30 con 0x55b602a734b0 2024-05-09T04:35:02.393 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.392+0000 7fe713c00640 1 --2- 172.21.15.145:0/349206573 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b602a734b0 0x55b602aed460 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe704000c00 tx=0x7fe704051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b32b530af64577 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:02.394 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.392+0000 7fe71b200640 1 -- 172.21.15.145:0/349206573 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe704069070 con 0x55b602a734b0 2024-05-09T04:35:02.394 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.392+0000 7fe71b200640 1 -- 172.21.15.145:0/349206573 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe704051890 con 0x55b602a734b0 2024-05-09T04:35:02.394 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.392+0000 7fe71b200640 1 -- 172.21.15.145:0/349206573 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe704055070 con 0x55b602a734b0 2024-05-09T04:35:02.394 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.393+0000 7fe71f48cf40 1 -- 172.21.15.145:0/349206573 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b602a734b0 msgr2=0x55b602aed460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:02.394 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.393+0000 7fe71f48cf40 1 --2- 172.21.15.145:0/349206573 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b602a734b0 0x55b602aed460 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe704000c00 tx=0x7fe704051000 comp rx=0 tx=0).stop 2024-05-09T04:35:02.394 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.393+0000 7fe71f48cf40 1 -- 172.21.15.145:0/349206573 shutdown_connections 2024-05-09T04:35:02.395 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.393+0000 7fe71f48cf40 1 --2- 172.21.15.145:0/349206573 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b602af80d0 0x55b602a575d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:02.395 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.393+0000 7fe71f48cf40 1 --2- 172.21.15.145:0/349206573 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b602a734b0 0x55b602aed460 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:02.395 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.393+0000 7fe71f48cf40 1 --2- 172.21.15.145:0/349206573 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55b602a72b90 0x55b602b8c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:02.395 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.393+0000 7fe71f48cf40 1 -- 172.21.15.145:0/349206573 >> 172.21.15.145:0/349206573 conn(0x55b602a55c80 msgr2=0x55b602a53bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:02.395 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.393+0000 7fe71f48cf40 1 -- 172.21.15.145:0/349206573 shutdown_connections 2024-05-09T04:35:02.395 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.393+0000 7fe71f48cf40 1 -- 172.21.15.145:0/349206573 wait complete. 2024-05-09T04:35:02.395 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.394+0000 7fe71f48cf40 1 Processor -- start 2024-05-09T04:35:02.395 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.394+0000 7fe71f48cf40 1 -- start start 2024-05-09T04:35:02.397 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.395+0000 7fe71f48cf40 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b602a72b90 0x55b602aec820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:02.397 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.395+0000 7fe71f48cf40 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55b602a734b0 0x55b602afb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:02.397 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.395+0000 7fe71f48cf40 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b602af80d0 0x55b602b97940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:02.397 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.395+0000 7fe71f48cf40 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55b602b8c980 con 0x55b602af80d0 2024-05-09T04:35:02.397 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.395+0000 7fe71f48cf40 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55b602af6340 con 0x55b602a72b90 2024-05-09T04:35:02.397 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.395+0000 7fe71f48cf40 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x55b602aed9a0 con 0x55b602a734b0 2024-05-09T04:35:02.397 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.395+0000 7fe71d000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b602af80d0 0x55b602b97940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:02.397 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.395+0000 7fe71d000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b602af80d0 0x55b602b97940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:53038/0 (socket says 172.21.15.145:53038) 2024-05-09T04:35:02.398 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.395+0000 7fe71d000640 1 -- 172.21.15.145:0/3421108416 learned_addr learned my addr 172.21.15.145:0/3421108416 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:02.398 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.396+0000 7fe713c00640 1 --2- 172.21.15.145:0/3421108416 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55b602a734b0 0x55b602afb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:02.398 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.396+0000 7fe71c600640 1 --2- 172.21.15.145:0/3421108416 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b602a72b90 0x55b602aec820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:02.398 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.396+0000 7fe71d000640 1 -- 172.21.15.145:0/3421108416 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55b602a734b0 msgr2=0x55b602afb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:02.398 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.396+0000 7fe71d000640 1 --2- 172.21.15.145:0/3421108416 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55b602a734b0 0x55b602afb2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:02.398 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.396+0000 7fe71d000640 1 -- 172.21.15.145:0/3421108416 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b602a72b90 msgr2=0x55b602aec820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:02.398 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.396+0000 7fe71d000640 1 --2- 172.21.15.145:0/3421108416 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55b602a72b90 0x55b602aec820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:02.398 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.396+0000 7fe71d000640 1 -- 172.21.15.145:0/3421108416 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b602af9920 con 0x55b602af80d0 2024-05-09T04:35:02.398 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.396+0000 7fe71d000640 1 --2- 172.21.15.145:0/3421108416 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b602af80d0 0x55b602b97940 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fe714007e30 tx=0x7fe714006750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:02.398 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.396+0000 7fe719e00640 1 -- 172.21.15.145:0/3421108416 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe714017070 con 0x55b602af80d0 2024-05-09T04:35:02.398 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.396+0000 7fe719e00640 1 -- 172.21.15.145:0/3421108416 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe714010730 con 0x55b602af80d0 2024-05-09T04:35:02.398 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.396+0000 7fe719e00640 1 -- 172.21.15.145:0/3421108416 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe714006e60 con 0x55b602af80d0 2024-05-09T04:35:02.398 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.396+0000 7fe71f48cf40 1 -- 172.21.15.145:0/3421108416 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b602b8eaa0 con 0x55b602af80d0 2024-05-09T04:35:02.399 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.397+0000 7fe71f48cf40 1 -- 172.21.15.145:0/3421108416 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b602af6d80 con 0x55b602af80d0 2024-05-09T04:35:02.400 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.398+0000 7fe719e00640 1 -- 172.21.15.145:0/3421108416 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe71401a070 con 0x55b602af80d0 2024-05-09T04:35:02.400 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.399+0000 7fe719e00640 1 --2- 172.21.15.145:0/3421108416 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fe6f8074ce0 0x7fe6f80771a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:02.400 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.400+0000 7fe719e00640 1 -- 172.21.15.145:0/3421108416 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 6104+0+0 (secure 0 0 0) 0x7fe714095860 con 0x55b602af80d0 2024-05-09T04:35:02.400 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.400+0000 7fe71c600640 1 --2- 172.21.15.145:0/3421108416 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fe6f8074ce0 0x7fe6f80771a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:02.400 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.400+0000 7fe71f48cf40 1 --2- 172.21.15.145:0/3421108416 >> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] conn(0x55b602afb810 0x55b602af86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:02.400 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.400+0000 7fe71f48cf40 1 -- 172.21.15.145:0/3421108416 --> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] -- osd_op(unknown.0.0:1 5.13 5:ca8fdbba:::100000001fb.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) -- 0x55b602b00d00 con 0x55b602afb810 2024-05-09T04:35:02.400 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.400+0000 7fe713c00640 1 --2- 172.21.15.145:0/3421108416 >> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] conn(0x55b602afb810 0x55b602af86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:02.401 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.400+0000 7fe71c600640 1 --2- 172.21.15.145:0/3421108416 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fe6f8074ce0 0x7fe6f80771a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe70c00e330 tx=0x7fe70c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:02.402 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.401+0000 7fe713c00640 1 --2- 172.21.15.145:0/3421108416 >> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] conn(0x55b602afb810 0x55b602af86c0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:02.402 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.401+0000 7fe713c00640 1 -- 172.21.15.145:0/3421108416 <== osd.3 v2:172.21.15.145:6808/337009826 1 ==== osd_op_reply(1 100000001fb.00000000 [getxattr (103) out=103b] v0'0 uv3 ondisk = 0) ==== 164+0+103 (crc 0 0 0) 0x7fe704057030 con 0x55b602afb810 2024-05-09T04:35:02.402 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.401+0000 7fe71f48cf40 1 -- 172.21.15.145:0/3421108416 >> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] conn(0x55b602afb810 msgr2=0x55b602af86c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:02.402 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.401+0000 7fe71f48cf40 1 --2- 172.21.15.145:0/3421108416 >> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] conn(0x55b602afb810 0x55b602af86c0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:02.402 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.401+0000 7fe71f48cf40 1 -- 172.21.15.145:0/3421108416 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fe6f8074ce0 msgr2=0x7fe6f80771a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:02.402 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.401+0000 7fe71f48cf40 1 --2- 172.21.15.145:0/3421108416 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fe6f8074ce0 0x7fe6f80771a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe70c00e330 tx=0x7fe70c008040 comp rx=0 tx=0).stop 2024-05-09T04:35:02.402 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.401+0000 7fe71f48cf40 1 -- 172.21.15.145:0/3421108416 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b602af80d0 msgr2=0x55b602b97940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:02.402 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.401+0000 7fe71f48cf40 1 --2- 172.21.15.145:0/3421108416 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55b602af80d0 0x55b602b97940 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fe714007e30 tx=0x7fe714006750 comp rx=0 tx=0).stop 2024-05-09T04:35:02.402 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.401+0000 7fe71d000640 1 -- 172.21.15.145:0/3421108416 reap_dead start 2024-05-09T04:35:02.402 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.401+0000 7fe71f48cf40 1 -- 172.21.15.145:0/3421108416 shutdown_connections 2024-05-09T04:35:02.402 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.401+0000 7fe71f48cf40 1 -- 172.21.15.145:0/3421108416 >> 172.21.15.145:0/3421108416 conn(0x55b602a55c80 msgr2=0x55b602b8ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:02.403 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.402+0000 7fe71f48cf40 1 -- 172.21.15.145:0/3421108416 shutdown_connections 2024-05-09T04:35:02.403 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.402+0000 7fe71f48cf40 1 -- 172.21.15.145:0/3421108416 wait complete. 2024-05-09T04:35:02.406 INFO:teuthology.orchestra.run.smithi145.stdout:a��alpha parent_b$ 2024-05-09T04:35:02.406 DEBUG:teuthology.orchestra.run.smithi145:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-05-09T04:35:02.502 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-05-09T04:35:02.502 INFO:teuthology.orchestra.run.smithi145.stdout: "ino": 1099511628283, 2024-05-09T04:35:02.502 INFO:teuthology.orchestra.run.smithi145.stdout: "ancestors": [ 2024-05-09T04:35:02.502 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-05-09T04:35:02.502 INFO:teuthology.orchestra.run.smithi145.stdout: "dirino": 1099511628284, 2024-05-09T04:35:02.502 INFO:teuthology.orchestra.run.smithi145.stdout: "dname": "alpha", 2024-05-09T04:35:02.502 INFO:teuthology.orchestra.run.smithi145.stdout: "version": 9 2024-05-09T04:35:02.502 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:35:02.502 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-05-09T04:35:02.502 INFO:teuthology.orchestra.run.smithi145.stdout: "dirino": 1, 2024-05-09T04:35:02.503 INFO:teuthology.orchestra.run.smithi145.stdout: "dname": "parent_b", 2024-05-09T04:35:02.503 INFO:teuthology.orchestra.run.smithi145.stdout: "version": 36 2024-05-09T04:35:02.503 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-05-09T04:35:02.503 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2024-05-09T04:35:02.503 INFO:teuthology.orchestra.run.smithi145.stdout: "pool": 6, 2024-05-09T04:35:02.503 INFO:teuthology.orchestra.run.smithi145.stdout: "old_pools": [ 2024-05-09T04:35:02.503 INFO:teuthology.orchestra.run.smithi145.stdout: 5 2024-05-09T04:35:02.503 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-05-09T04:35:02.503 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-05-09T04:35:02.503 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:35:02.507 DEBUG:teuthology.orchestra.run.smithi145:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool data_new getxattr 100000001fb.00000000 parent 2024-05-09T04:35:02.614 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.614+0000 7f4d0c01ef40 1 Processor -- start 2024-05-09T04:35:02.614 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.614+0000 7f4d0c01ef40 1 -- start start 2024-05-09T04:35:02.615 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.615+0000 7f4d0c01ef40 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c4c503f50 0x555c4c5bb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:02.615 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.615+0000 7f4d0c01ef40 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x555c4c504330 0x555c4c5da150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:02.615 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.615+0000 7f4d0c01ef40 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555c4c54d6a0 0x555c4c5da690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:02.616 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.615+0000 7f4d0c01ef40 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x555c4c506d70 con 0x555c4c54d6a0 2024-05-09T04:35:02.616 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.615+0000 7f4d0c01ef40 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x555c4c4abf50 con 0x555c4c503f50 2024-05-09T04:35:02.616 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.615+0000 7f4d0c01ef40 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x555c4c507000 con 0x555c4c504330 2024-05-09T04:35:02.616 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.615+0000 7f4d09c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555c4c54d6a0 0x555c4c5da690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:02.616 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.615+0000 7f4d09c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555c4c54d6a0 0x555c4c5da690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:53054/0 (socket says 172.21.15.145:53054) 2024-05-09T04:35:02.616 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.615+0000 7f4d09c00640 1 -- 172.21.15.145:0/1447235594 learned_addr learned my addr 172.21.15.145:0/1447235594 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:02.616 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.615+0000 7f4d09200640 1 --2- 172.21.15.145:0/1447235594 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c4c503f50 0x555c4c5bb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:02.616 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.616+0000 7f4d08800640 1 --2- 172.21.15.145:0/1447235594 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x555c4c504330 0x555c4c5da150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:02.616 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.616+0000 7f4d09c00640 1 -- 172.21.15.145:0/1447235594 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x555c4c504330 msgr2=0x555c4c5da150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:02.616 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.616+0000 7f4d09c00640 1 --2- 172.21.15.145:0/1447235594 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x555c4c504330 0x555c4c5da150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:02.616 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.616+0000 7f4d09c00640 1 -- 172.21.15.145:0/1447235594 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c4c503f50 msgr2=0x555c4c5bb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:02.616 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.616+0000 7f4d09c00640 1 --2- 172.21.15.145:0/1447235594 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c4c503f50 0x555c4c5bb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:02.616 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.616+0000 7f4d09c00640 1 -- 172.21.15.145:0/1447235594 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c4c42eb30 con 0x555c4c54d6a0 2024-05-09T04:35:02.618 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.616+0000 7f4d09c00640 1 --2- 172.21.15.145:0/1447235594 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555c4c54d6a0 0x555c4c5da690 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4d0000a3e0 tx=0x7f4d0000a8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3c4e539783a18d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:02.618 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.616+0000 7f4d07400640 1 -- 172.21.15.145:0/1447235594 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0000b920 con 0x555c4c54d6a0 2024-05-09T04:35:02.618 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.616+0000 7f4d07400640 1 -- 172.21.15.145:0/1447235594 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d0000f030 con 0x555c4c54d6a0 2024-05-09T04:35:02.618 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.616+0000 7f4d07400640 1 -- 172.21.15.145:0/1447235594 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0000bd20 con 0x555c4c54d6a0 2024-05-09T04:35:02.618 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.617+0000 7f4d0c01ef40 1 -- 172.21.15.145:0/1447235594 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555c4c54d6a0 msgr2=0x555c4c5da690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:02.618 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.617+0000 7f4d0c01ef40 1 --2- 172.21.15.145:0/1447235594 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555c4c54d6a0 0x555c4c5da690 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4d0000a3e0 tx=0x7f4d0000a8a0 comp rx=0 tx=0).stop 2024-05-09T04:35:02.618 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.617+0000 7f4d0c01ef40 1 -- 172.21.15.145:0/1447235594 shutdown_connections 2024-05-09T04:35:02.618 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.617+0000 7f4d0c01ef40 1 --2- 172.21.15.145:0/1447235594 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555c4c54d6a0 0x555c4c5da690 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:02.618 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.617+0000 7f4d0c01ef40 1 --2- 172.21.15.145:0/1447235594 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x555c4c504330 0x555c4c5da150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:02.618 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.617+0000 7f4d0c01ef40 1 --2- 172.21.15.145:0/1447235594 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c4c503f50 0x555c4c5bb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:02.618 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.617+0000 7f4d0c01ef40 1 -- 172.21.15.145:0/1447235594 >> 172.21.15.145:0/1447235594 conn(0x555c4c531240 msgr2=0x555c4c502ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:02.618 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.617+0000 7f4d0c01ef40 1 -- 172.21.15.145:0/1447235594 shutdown_connections 2024-05-09T04:35:02.619 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.617+0000 7f4d0c01ef40 1 -- 172.21.15.145:0/1447235594 wait complete. 2024-05-09T04:35:02.619 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.618+0000 7f4d0c01ef40 1 Processor -- start 2024-05-09T04:35:02.619 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.618+0000 7f4d0c01ef40 1 -- start start 2024-05-09T04:35:02.621 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.619+0000 7f4d0c01ef40 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x555c4c503f50 0x555c4c5d1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:02.621 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.619+0000 7f4d0c01ef40 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555c4c504330 0x555c4c531b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:02.621 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.619+0000 7f4d0c01ef40 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c4c54d6a0 0x555c4c532040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:02.621 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.619+0000 7f4d0c01ef40 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x555c4c5db270 con 0x555c4c504330 2024-05-09T04:35:02.621 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.619+0000 7f4d0c01ef40 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x555c4c5bbd40 con 0x555c4c54d6a0 2024-05-09T04:35:02.621 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.619+0000 7f4d0c01ef40 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x555c4c5db550 con 0x555c4c503f50 2024-05-09T04:35:02.621 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.619+0000 7f4d08800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555c4c504330 0x555c4c531b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:02.621 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.619+0000 7f4d08800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555c4c504330 0x555c4c531b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:53058/0 (socket says 172.21.15.145:53058) 2024-05-09T04:35:02.621 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.619+0000 7f4d08800640 1 -- 172.21.15.145:0/4262393737 learned_addr learned my addr 172.21.15.145:0/4262393737 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:02.622 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.619+0000 7f4d09200640 1 --2- 172.21.15.145:0/4262393737 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x555c4c503f50 0x555c4c5d1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:02.622 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.620+0000 7f4d09c00640 1 --2- 172.21.15.145:0/4262393737 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c4c54d6a0 0x555c4c532040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:02.622 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.620+0000 7f4d08800640 1 -- 172.21.15.145:0/4262393737 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x555c4c503f50 msgr2=0x555c4c5d1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:02.622 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.620+0000 7f4d08800640 1 --2- 172.21.15.145:0/4262393737 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x555c4c503f50 0x555c4c5d1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:02.622 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.620+0000 7f4d08800640 1 -- 172.21.15.145:0/4262393737 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c4c54d6a0 msgr2=0x555c4c532040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:02.622 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.620+0000 7f4d08800640 1 --2- 172.21.15.145:0/4262393737 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555c4c54d6a0 0x555c4c532040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:02.622 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.620+0000 7f4d08800640 1 -- 172.21.15.145:0/4262393737 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c4c52e5e0 con 0x555c4c504330 2024-05-09T04:35:02.622 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.620+0000 7f4d08800640 1 --2- 172.21.15.145:0/4262393737 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555c4c504330 0x555c4c531b00 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4cf400be20 tx=0x7f4cf400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:02.622 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.620+0000 7f4d06000640 1 -- 172.21.15.145:0/4262393737 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf4019070 con 0x555c4c504330 2024-05-09T04:35:02.622 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.620+0000 7f4d06000640 1 -- 172.21.15.145:0/4262393737 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cf4007b30 con 0x555c4c504330 2024-05-09T04:35:02.622 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.620+0000 7f4d06000640 1 -- 172.21.15.145:0/4262393737 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf4014410 con 0x555c4c504330 2024-05-09T04:35:02.622 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.620+0000 7f4d0c01ef40 1 -- 172.21.15.145:0/4262393737 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555c4c5338f0 con 0x555c4c504330 2024-05-09T04:35:02.622 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.621+0000 7f4d0c01ef40 1 -- 172.21.15.145:0/4262393737 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c4c5d1b40 con 0x555c4c504330 2024-05-09T04:35:02.623 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.623+0000 7f4d06000640 1 -- 172.21.15.145:0/4262393737 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4cf401c070 con 0x555c4c504330 2024-05-09T04:35:02.623 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.624+0000 7f4d06000640 1 --2- 172.21.15.145:0/4262393737 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f4ce8074cb0 0x7f4ce8077170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:02.623 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.624+0000 7f4d06000640 1 -- 172.21.15.145:0/4262393737 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 6104+0+0 (secure 0 0 0) 0x7f4cf40954f0 con 0x555c4c504330 2024-05-09T04:35:02.624 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.624+0000 7f4d09200640 1 --2- 172.21.15.145:0/4262393737 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f4ce8074cb0 0x7f4ce8077170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:02.624 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.624+0000 7f4d0c01ef40 1 --2- 172.21.15.145:0/4262393737 >> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] conn(0x555c4c5d9af0 0x555c4c5b36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:02.624 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.624+0000 7f4d0c01ef40 1 -- 172.21.15.145:0/4262393737 --> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] -- osd_op(unknown.0.0:1 6.13 6:ca8fdbba:::100000001fb.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) -- 0x555c4c5cbc80 con 0x555c4c5d9af0 2024-05-09T04:35:02.624 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.625+0000 7f4d09c00640 1 --2- 172.21.15.145:0/4262393737 >> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] conn(0x555c4c5d9af0 0x555c4c5b36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:02.624 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.625+0000 7f4d09200640 1 --2- 172.21.15.145:0/4262393737 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f4ce8074cb0 0x7f4ce8077170 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc056000 tx=0x7f4cfc007030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:02.625 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.625+0000 7f4d09c00640 1 --2- 172.21.15.145:0/4262393737 >> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] conn(0x555c4c5d9af0 0x555c4c5b36f0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:02.626 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.626+0000 7f4d09c00640 1 -- 172.21.15.145:0/4262393737 <== osd.3 v2:172.21.15.145:6808/337009826 1 ==== osd_op_reply(1 100000001fb.00000000 [getxattr (103) out=103b] v0'0 uv1 ondisk = 0) ==== 164+0+103 (crc 0 0 0) 0x7f4d0000c8b0 con 0x555c4c5d9af0 2024-05-09T04:35:02.626 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.626+0000 7f4d0c01ef40 1 -- 172.21.15.145:0/4262393737 >> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] conn(0x555c4c5d9af0 msgr2=0x555c4c5b36f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:02.626 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.626+0000 7f4d0c01ef40 1 --2- 172.21.15.145:0/4262393737 >> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] conn(0x555c4c5d9af0 0x555c4c5b36f0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:02.626 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.626+0000 7f4d0c01ef40 1 -- 172.21.15.145:0/4262393737 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f4ce8074cb0 msgr2=0x7f4ce8077170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:02.626 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.626+0000 7f4d0c01ef40 1 --2- 172.21.15.145:0/4262393737 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f4ce8074cb0 0x7f4ce8077170 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc056000 tx=0x7f4cfc007030 comp rx=0 tx=0).stop 2024-05-09T04:35:02.626 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.626+0000 7f4d0c01ef40 1 -- 172.21.15.145:0/4262393737 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555c4c504330 msgr2=0x555c4c531b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:02.626 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.626+0000 7f4d0c01ef40 1 --2- 172.21.15.145:0/4262393737 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x555c4c504330 0x555c4c531b00 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4cf400be20 tx=0x7f4cf400bf10 comp rx=0 tx=0).stop 2024-05-09T04:35:02.626 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.626+0000 7f4d09c00640 1 -- 172.21.15.145:0/4262393737 reap_dead start 2024-05-09T04:35:02.626 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.626+0000 7f4d0c01ef40 1 -- 172.21.15.145:0/4262393737 shutdown_connections 2024-05-09T04:35:02.627 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.626+0000 7f4d0c01ef40 1 -- 172.21.15.145:0/4262393737 >> 172.21.15.145:0/4262393737 conn(0x555c4c531240 msgr2=0x555c4c5bbec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:02.627 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.626+0000 7f4d0c01ef40 1 -- 172.21.15.145:0/4262393737 shutdown_connections 2024-05-09T04:35:02.627 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.626+0000 7f4d0c01ef40 1 -- 172.21.15.145:0/4262393737 wait complete. 2024-05-09T04:35:02.630 INFO:teuthology.orchestra.run.smithi145.stdout:a��alpha parent_b$ 2024-05-09T04:35:02.631 DEBUG:teuthology.orchestra.run.smithi145:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-05-09T04:35:02.732 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-05-09T04:35:02.732 INFO:teuthology.orchestra.run.smithi145.stdout: "ino": 1099511628283, 2024-05-09T04:35:02.732 INFO:teuthology.orchestra.run.smithi145.stdout: "ancestors": [ 2024-05-09T04:35:02.732 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-05-09T04:35:02.732 INFO:teuthology.orchestra.run.smithi145.stdout: "dirino": 1099511628284, 2024-05-09T04:35:02.732 INFO:teuthology.orchestra.run.smithi145.stdout: "dname": "alpha", 2024-05-09T04:35:02.732 INFO:teuthology.orchestra.run.smithi145.stdout: "version": 9 2024-05-09T04:35:02.732 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:35:02.732 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-05-09T04:35:02.732 INFO:teuthology.orchestra.run.smithi145.stdout: "dirino": 1, 2024-05-09T04:35:02.732 INFO:teuthology.orchestra.run.smithi145.stdout: "dname": "parent_b", 2024-05-09T04:35:02.732 INFO:teuthology.orchestra.run.smithi145.stdout: "version": 36 2024-05-09T04:35:02.733 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-05-09T04:35:02.733 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2024-05-09T04:35:02.733 INFO:teuthology.orchestra.run.smithi145.stdout: "pool": 6, 2024-05-09T04:35:02.733 INFO:teuthology.orchestra.run.smithi145.stdout: "old_pools": [ 2024-05-09T04:35:02.733 INFO:teuthology.orchestra.run.smithi145.stdout: 5 2024-05-09T04:35:02.733 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-05-09T04:35:02.733 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-05-09T04:35:02.733 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:35:02.737 DEBUG:teuthology.orchestra.run.smithi145:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool data_new getxattr 100000001fb.00000000 layout 2024-05-09T04:35:02.843 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.843+0000 7f91779dbf40 1 Processor -- start 2024-05-09T04:35:02.843 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.843+0000 7f91779dbf40 1 -- start start 2024-05-09T04:35:02.844 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.844+0000 7f91779dbf40 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c91d218b90 0x55c91d29b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:02.844 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.844+0000 7f91779dbf40 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55c91d2194b0 0x55c91d1fd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:02.844 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.844+0000 7f91779dbf40 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c91d29e0d0 0x55c91d1fdab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:02.845 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.844+0000 7f91779dbf40 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55c91d2a50d0 con 0x55c91d29e0d0 2024-05-09T04:35:02.845 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.844+0000 7f91779dbf40 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55c91d217f10 con 0x55c91d218b90 2024-05-09T04:35:02.845 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.844+0000 7f91779dbf40 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x55c91d219a30 con 0x55c91d2194b0 2024-05-09T04:35:02.845 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.844+0000 7f9175600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c91d29e0d0 0x55c91d1fdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:02.845 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.844+0000 7f9175600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c91d29e0d0 0x55c91d1fdab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:53070/0 (socket says 172.21.15.145:53070) 2024-05-09T04:35:02.845 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.844+0000 7f9175600640 1 -- 172.21.15.145:0/1712058374 learned_addr learned my addr 172.21.15.145:0/1712058374 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:02.845 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.845+0000 7f9174c00640 1 --2- 172.21.15.145:0/1712058374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c91d218b90 0x55c91d29b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:02.845 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.845+0000 7f9174200640 1 --2- 172.21.15.145:0/1712058374 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55c91d2194b0 0x55c91d1fd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:02.845 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.845+0000 7f9175600640 1 -- 172.21.15.145:0/1712058374 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55c91d2194b0 msgr2=0x55c91d1fd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:02.845 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.845+0000 7f9175600640 1 --2- 172.21.15.145:0/1712058374 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55c91d2194b0 0x55c91d1fd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:02.845 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.845+0000 7f9175600640 1 -- 172.21.15.145:0/1712058374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c91d218b90 msgr2=0x55c91d29b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:02.845 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.845+0000 7f9175600640 1 --2- 172.21.15.145:0/1712058374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c91d218b90 0x55c91d29b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:02.845 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.845+0000 7f9175600640 1 -- 172.21.15.145:0/1712058374 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c91d11c300 con 0x55c91d29e0d0 2024-05-09T04:35:02.847 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.845+0000 7f9175600640 1 --2- 172.21.15.145:0/1712058374 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c91d29e0d0 0x55c91d1fdab0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f916c00a770 tx=0x7f916c00a860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c22de2429c0ec664 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:02.847 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.845+0000 7f9172e00640 1 -- 172.21.15.145:0/1712058374 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916c018070 con 0x55c91d29e0d0 2024-05-09T04:35:02.847 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.845+0000 7f9172e00640 1 -- 172.21.15.145:0/1712058374 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f916c007600 con 0x55c91d29e0d0 2024-05-09T04:35:02.847 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.845+0000 7f9172e00640 1 -- 172.21.15.145:0/1712058374 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916c0079d0 con 0x55c91d29e0d0 2024-05-09T04:35:02.847 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.846+0000 7f91779dbf40 1 -- 172.21.15.145:0/1712058374 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c91d29e0d0 msgr2=0x55c91d1fdab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:02.847 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.846+0000 7f91779dbf40 1 --2- 172.21.15.145:0/1712058374 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c91d29e0d0 0x55c91d1fdab0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f916c00a770 tx=0x7f916c00a860 comp rx=0 tx=0).stop 2024-05-09T04:35:02.847 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.846+0000 7f91779dbf40 1 -- 172.21.15.145:0/1712058374 shutdown_connections 2024-05-09T04:35:02.847 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.846+0000 7f91779dbf40 1 --2- 172.21.15.145:0/1712058374 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c91d29e0d0 0x55c91d1fdab0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:02.847 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.846+0000 7f91779dbf40 1 --2- 172.21.15.145:0/1712058374 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55c91d2194b0 0x55c91d1fd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:02.847 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.846+0000 7f91779dbf40 1 --2- 172.21.15.145:0/1712058374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c91d218b90 0x55c91d29b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:02.847 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.846+0000 7f91779dbf40 1 -- 172.21.15.145:0/1712058374 >> 172.21.15.145:0/1712058374 conn(0x55c91d1fbc80 msgr2=0x55c91d2a1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:02.847 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.846+0000 7f91779dbf40 1 -- 172.21.15.145:0/1712058374 shutdown_connections 2024-05-09T04:35:02.848 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.846+0000 7f91779dbf40 1 -- 172.21.15.145:0/1712058374 wait complete. 2024-05-09T04:35:02.848 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.847+0000 7f91779dbf40 1 Processor -- start 2024-05-09T04:35:02.848 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.847+0000 7f91779dbf40 1 -- start start 2024-05-09T04:35:02.849 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.848+0000 7f91779dbf40 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55c91d218b90 0x55c91d292c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:02.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.848+0000 7f91779dbf40 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c91d2194b0 0x55c91d33a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:02.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.848+0000 7f91779dbf40 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c91d29e0d0 0x55c91d294040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:02.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.848+0000 7f91779dbf40 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55c91d29ab20 con 0x55c91d29e0d0 2024-05-09T04:35:02.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.848+0000 7f91779dbf40 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55c91d1cd6d0 con 0x55c91d2194b0 2024-05-09T04:35:02.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.848+0000 7f91779dbf40 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x55c91d293530 con 0x55c91d218b90 2024-05-09T04:35:02.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.848+0000 7f9175600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c91d29e0d0 0x55c91d294040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:02.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.848+0000 7f9175600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c91d29e0d0 0x55c91d294040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:53084/0 (socket says 172.21.15.145:53084) 2024-05-09T04:35:02.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.848+0000 7f9175600640 1 -- 172.21.15.145:0/2225133946 learned_addr learned my addr 172.21.15.145:0/2225133946 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:02.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.849+0000 7f9174200640 1 --2- 172.21.15.145:0/2225133946 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c91d2194b0 0x55c91d33a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:02.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.849+0000 7f9175600640 1 -- 172.21.15.145:0/2225133946 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55c91d218b90 msgr2=0x55c91d292c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:02.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.849+0000 7f9175600640 1 --2- 172.21.15.145:0/2225133946 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55c91d218b90 0x55c91d292c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:02.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.849+0000 7f9175600640 1 -- 172.21.15.145:0/2225133946 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c91d2194b0 msgr2=0x55c91d33a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:02.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.849+0000 7f9175600640 1 --2- 172.21.15.145:0/2225133946 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55c91d2194b0 0x55c91d33a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:02.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.849+0000 7f9175600640 1 -- 172.21.15.145:0/2225133946 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c91d0f6b30 con 0x55c91d29e0d0 2024-05-09T04:35:02.851 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.849+0000 7f9175600640 1 --2- 172.21.15.145:0/2225133946 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c91d29e0d0 0x55c91d294040 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f916c002000 tx=0x7f916c003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:02.851 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.849+0000 7f9171a00640 1 -- 172.21.15.145:0/2225133946 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916c023020 con 0x55c91d29e0d0 2024-05-09T04:35:02.851 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.849+0000 7f9171a00640 1 -- 172.21.15.145:0/2225133946 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f916c019040 con 0x55c91d29e0d0 2024-05-09T04:35:02.851 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.849+0000 7f9171a00640 1 -- 172.21.15.145:0/2225133946 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916c018070 con 0x55c91d29e0d0 2024-05-09T04:35:02.851 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.849+0000 7f91779dbf40 1 -- 172.21.15.145:0/2225133946 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c91d294b00 con 0x55c91d29e0d0 2024-05-09T04:35:02.851 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.849+0000 7f91779dbf40 1 -- 172.21.15.145:0/2225133946 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c91d2a04a0 con 0x55c91d29e0d0 2024-05-09T04:35:02.851 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.852+0000 7f9171a00640 1 -- 172.21.15.145:0/2225133946 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f916c026020 con 0x55c91d29e0d0 2024-05-09T04:35:02.852 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.852+0000 7f9171a00640 1 --2- 172.21.15.145:0/2225133946 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f9150074cb0 0x7f9150077170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:02.852 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.853+0000 7f9171a00640 1 -- 172.21.15.145:0/2225133946 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 6104+0+0 (secure 0 0 0) 0x7f916c097890 con 0x55c91d29e0d0 2024-05-09T04:35:02.853 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.853+0000 7f9174c00640 1 --2- 172.21.15.145:0/2225133946 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f9150074cb0 0x7f9150077170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:02.853 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.853+0000 7f91779dbf40 1 --2- 172.21.15.145:0/2225133946 >> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] conn(0x55c91d33aa70 0x55c91d29a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:02.853 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.853+0000 7f91779dbf40 1 -- 172.21.15.145:0/2225133946 --> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] -- osd_op(unknown.0.0:1 6.13 6:ca8fdbba:::100000001fb.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) -- 0x55c91d2a0f80 con 0x55c91d33aa70 2024-05-09T04:35:02.853 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.853+0000 7f9174200640 1 --2- 172.21.15.145:0/2225133946 >> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] conn(0x55c91d33aa70 0x55c91d29a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:02.853 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.854+0000 7f9174c00640 1 --2- 172.21.15.145:0/2225133946 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f9150074cb0 0x7f9150077170 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9164056000 tx=0x7f9164050000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:02.854 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.854+0000 7f9174200640 1 --2- 172.21.15.145:0/2225133946 >> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] conn(0x55c91d33aa70 0x55c91d29a340 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:02.855 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.854+0000 7f9174200640 1 -- 172.21.15.145:0/2225133946 <== osd.3 v2:172.21.15.145:6808/337009826 1 ==== osd_op_reply(1 100000001fb.00000000 [getxattr (30) out=30b] v0'0 uv1 ondisk = 0) ==== 164+0+30 (crc 0 0 0) 0x7f915c009730 con 0x55c91d33aa70 2024-05-09T04:35:02.855 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.854+0000 7f91779dbf40 1 -- 172.21.15.145:0/2225133946 >> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] conn(0x55c91d33aa70 msgr2=0x55c91d29a340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:02.855 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.854+0000 7f91779dbf40 1 --2- 172.21.15.145:0/2225133946 >> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] conn(0x55c91d33aa70 0x55c91d29a340 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:02.855 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.855+0000 7f91779dbf40 1 -- 172.21.15.145:0/2225133946 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f9150074cb0 msgr2=0x7f9150077170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:02.855 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.855+0000 7f91779dbf40 1 --2- 172.21.15.145:0/2225133946 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f9150074cb0 0x7f9150077170 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9164056000 tx=0x7f9164050000 comp rx=0 tx=0).stop 2024-05-09T04:35:02.855 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.855+0000 7f91779dbf40 1 -- 172.21.15.145:0/2225133946 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c91d29e0d0 msgr2=0x55c91d294040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:02.855 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.855+0000 7f91779dbf40 1 --2- 172.21.15.145:0/2225133946 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55c91d29e0d0 0x55c91d294040 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f916c002000 tx=0x7f916c003020 comp rx=0 tx=0).stop 2024-05-09T04:35:02.855 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.855+0000 7f9175600640 1 -- 172.21.15.145:0/2225133946 reap_dead start 2024-05-09T04:35:02.855 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.855+0000 7f91779dbf40 1 -- 172.21.15.145:0/2225133946 shutdown_connections 2024-05-09T04:35:02.855 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.855+0000 7f91779dbf40 1 -- 172.21.15.145:0/2225133946 >> 172.21.15.145:0/2225133946 conn(0x55c91d1fbc80 msgr2=0x55c91d1f98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:02.855 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.855+0000 7f91779dbf40 1 -- 172.21.15.145:0/2225133946 shutdown_connections 2024-05-09T04:35:02.856 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:02.855+0000 7f91779dbf40 1 -- 172.21.15.145:0/2225133946 wait complete. 2024-05-09T04:35:02.862 INFO:teuthology.orchestra.run.smithi145.stdout:@@ 2024-05-09T04:35:02.862 DEBUG:teuthology.orchestra.run.smithi145:> ceph-dencoder type file_layout_t import - decode dump_json 2024-05-09T04:35:02.954 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-05-09T04:35:02.955 INFO:teuthology.orchestra.run.smithi145.stdout: "stripe_unit": 4194304, 2024-05-09T04:35:02.955 INFO:teuthology.orchestra.run.smithi145.stdout: "stripe_count": 1, 2024-05-09T04:35:02.955 INFO:teuthology.orchestra.run.smithi145.stdout: "object_size": 4194304, 2024-05-09T04:35:02.955 INFO:teuthology.orchestra.run.smithi145.stdout: "pool_id": 6, 2024-05-09T04:35:02.955 INFO:teuthology.orchestra.run.smithi145.stdout: "pool_ns": "" 2024-05-09T04:35:02.955 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-05-09T04:35:02.955 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:35:02.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:35:02.960 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent_c) 2024-05-09T04:35:03.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:35:03.012 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest/mnt.0 && exec mv parent_b/alpha parent_c/alpha) 2024-05-09T04:35:03.079 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T04:35:03.281 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.281+0000 7ffad7800640 1 Processor -- start 2024-05-09T04:35:03.281 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.281+0000 7ffad7800640 1 -- start start 2024-05-09T04:35:03.283 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.282+0000 7ffad7800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ffad006e860 0x7ffad006ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:03.283 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.282+0000 7ffad7800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ffad00b4910 0x7ffad00b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:03.283 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.282+0000 7ffad7800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffad006f180 con 0x7ffad015d780 2024-05-09T04:35:03.283 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.282+0000 7ffad7800640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7ffad0161360 con 0x7ffad00b4910 2024-05-09T04:35:03.283 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.282+0000 7ffad7800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7ffad01615c0 con 0x7ffad006e860 2024-05-09T04:35:03.283 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.283+0000 7ffad5800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ffad00b4910 0x7ffad00b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:03.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.283+0000 7ffad5800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ffad00b4910 0x7ffad00b4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:53086/0 (socket says 172.21.15.145:53086) 2024-05-09T04:35:03.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.283+0000 7ffad5800640 1 -- 172.21.15.145:0/3283474628 learned_addr learned my addr 172.21.15.145:0/3283474628 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:03.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.283+0000 7ffad4e00640 1 --2- 172.21.15.145:0/3283474628 >> v2:172.21.15.192:3300/0 conn(0x7ffad006e860 0x7ffad006ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:03.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.283+0000 7ffad5800640 1 -- 172.21.15.145:0/3283474628 >> v2:172.21.15.192:3300/0 conn(0x7ffad006e860 msgr2=0x7ffad006ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:03.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.284+0000 7ffad5800640 1 --2- 172.21.15.145:0/3283474628 >> v2:172.21.15.192:3300/0 conn(0x7ffad006e860 0x7ffad006ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:03.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.284+0000 7ffad5800640 1 -- 172.21.15.145:0/3283474628 >> v1:172.21.15.145:6789/0 conn(0x7ffad015d780 legacy=0x7ffad0160c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:03.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.284+0000 7ffad5800640 1 -- 172.21.15.145:0/3283474628 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffad01618a0 con 0x7ffad00b4910 2024-05-09T04:35:03.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.284+0000 7ffad4e00640 1 --2- 172.21.15.145:0/3283474628 >> v2:172.21.15.192:3300/0 conn(0x7ffad006e860 0x7ffad006ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:35:03.285 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.285+0000 7ffad5800640 1 --2- 172.21.15.145:0/3283474628 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffad00b4910 0x7ffad00b4d50 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ffad00745a0 tx=0x7ffac802a740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28f55b37649a71db server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:03.285 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.285+0000 7ffacf400640 1 -- 172.21.15.145:0/3283474628 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac8036070 con 0x7ffad00b4910 2024-05-09T04:35:03.285 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.285+0000 7ffacf400640 1 -- 172.21.15.145:0/3283474628 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffac8031600 con 0x7ffad00b4910 2024-05-09T04:35:03.285 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.285+0000 7ffacf400640 1 -- 172.21.15.145:0/3283474628 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac80319d0 con 0x7ffad00b4910 2024-05-09T04:35:03.286 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.285+0000 7ffad7800640 1 -- 172.21.15.145:0/3283474628 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffad00b4910 msgr2=0x7ffad00b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:03.286 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.285+0000 7ffad7800640 1 --2- 172.21.15.145:0/3283474628 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffad00b4910 0x7ffad00b4d50 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ffad00745a0 tx=0x7ffac802a740 comp rx=0 tx=0).stop 2024-05-09T04:35:03.286 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.286+0000 7ffad7800640 1 -- 172.21.15.145:0/3283474628 shutdown_connections 2024-05-09T04:35:03.286 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.286+0000 7ffad7800640 1 --2- 172.21.15.145:0/3283474628 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffad00b4910 0x7ffad00b4d50 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:03.286 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.286+0000 7ffad7800640 1 --2- 172.21.15.145:0/3283474628 >> v2:172.21.15.192:3300/0 conn(0x7ffad006e860 0x7ffad006ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:03.286 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.286+0000 7ffad7800640 1 -- 172.21.15.145:0/3283474628 >> 172.21.15.145:0/3283474628 conn(0x7ffad006c870 msgr2=0x7ffad006cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:03.286 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.286+0000 7ffad7800640 1 -- 172.21.15.145:0/3283474628 shutdown_connections 2024-05-09T04:35:03.286 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.286+0000 7ffad7800640 1 -- 172.21.15.145:0/3283474628 wait complete. 2024-05-09T04:35:03.287 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.287+0000 7ffad7800640 1 Processor -- start 2024-05-09T04:35:03.287 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.288+0000 7ffad7800640 1 -- start start 2024-05-09T04:35:03.288 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.288+0000 7ffad7800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffad006e860 0x7ffad0139830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:03.288 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.289+0000 7ffad7800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffad00b4910 0x7ffad01f5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:03.289 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.289+0000 7ffad7800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffad015d780 0x7ffad00afae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:03.289 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.289+0000 7ffad7800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ffad0146c00 con 0x7ffad00b4910 2024-05-09T04:35:03.289 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.289+0000 7ffad7800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ffad00b43f0 con 0x7ffad006e860 2024-05-09T04:35:03.289 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.289+0000 7ffad7800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ffad0146d80 con 0x7ffad015d780 2024-05-09T04:35:03.289 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.289+0000 7ffacfe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffad00b4910 0x7ffad01f5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:03.289 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.289+0000 7ffacfe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffad00b4910 0x7ffad01f5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:53102/0 (socket says 172.21.15.145:53102) 2024-05-09T04:35:03.289 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.289+0000 7ffacfe00640 1 -- 172.21.15.145:0/4171145332 learned_addr learned my addr 172.21.15.145:0/4171145332 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:03.291 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.289+0000 7ffad5800640 1 --2- 172.21.15.145:0/4171145332 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffad015d780 0x7ffad00afae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:03.291 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.289+0000 7ffad4e00640 1 --2- 172.21.15.145:0/4171145332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffad006e860 0x7ffad0139830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:03.291 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.289+0000 7ffacfe00640 1 -- 172.21.15.145:0/4171145332 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffad015d780 msgr2=0x7ffad00afae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:03.291 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.289+0000 7ffacfe00640 1 --2- 172.21.15.145:0/4171145332 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffad015d780 0x7ffad00afae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:03.291 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.289+0000 7ffacfe00640 1 -- 172.21.15.145:0/4171145332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffad006e860 msgr2=0x7ffad0139830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:03.291 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.289+0000 7ffacfe00640 1 --2- 172.21.15.145:0/4171145332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffad006e860 0x7ffad0139830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:03.291 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.289+0000 7ffacfe00640 1 -- 172.21.15.145:0/4171145332 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffad00aea40 con 0x7ffad00b4910 2024-05-09T04:35:03.291 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.289+0000 7ffad4e00640 1 --2- 172.21.15.145:0/4171145332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffad006e860 0x7ffad0139830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:35:03.291 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.289+0000 7ffad5800640 1 --2- 172.21.15.145:0/4171145332 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffad015d780 0x7ffad00afae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:35:03.291 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.290+0000 7ffacfe00640 1 --2- 172.21.15.145:0/4171145332 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffad00b4910 0x7ffad01f5cb0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ffab800c3d0 tx=0x7ffab800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:03.291 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.290+0000 7ffaccc00640 1 -- 172.21.15.145:0/4171145332 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab8017070 con 0x7ffad00b4910 2024-05-09T04:35:03.291 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.290+0000 7ffaccc00640 1 -- 172.21.15.145:0/4171145332 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffab80052d0 con 0x7ffad00b4910 2024-05-09T04:35:03.292 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.290+0000 7ffaccc00640 1 -- 172.21.15.145:0/4171145332 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab8003830 con 0x7ffad00b4910 2024-05-09T04:35:03.292 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.290+0000 7ffad7800640 1 -- 172.21.15.145:0/4171145332 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffad00aed20 con 0x7ffad00b4910 2024-05-09T04:35:03.292 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.290+0000 7ffad7800640 1 -- 172.21.15.145:0/4171145332 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffad00af2e0 con 0x7ffad00b4910 2024-05-09T04:35:03.292 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.292+0000 7ffad7800640 1 -- 172.21.15.145:0/4171145332 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffad006a310 con 0x7ffad00b4910 2024-05-09T04:35:03.292 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.293+0000 7ffaccc00640 1 -- 172.21.15.145:0/4171145332 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ffab801a070 con 0x7ffad00b4910 2024-05-09T04:35:03.293 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.294+0000 7ffaccc00640 1 --2- 172.21.15.145:0/4171145332 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ffaac0762d0 0x7ffaac078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:03.293 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.294+0000 7ffaccc00640 1 -- 172.21.15.145:0/4171145332 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 6104+0+0 (secure 0 0 0) 0x7ffab8096a00 con 0x7ffad00b4910 2024-05-09T04:35:03.293 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.294+0000 7ffad4e00640 1 --2- 172.21.15.145:0/4171145332 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ffaac0762d0 0x7ffaac078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:03.294 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.294+0000 7ffad4e00640 1 --2- 172.21.15.145:0/4171145332 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ffaac0762d0 0x7ffaac078790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ffac0006fd0 tx=0x7ffac0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:03.297 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.298+0000 7ffaccc00640 1 -- 172.21.15.145:0/4171145332 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ffab8060260 con 0x7ffad00b4910 2024-05-09T04:35:03.545 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.546+0000 7ffad7800640 1 -- 172.21.15.145:0/4171145332 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffad015c3a0 con 0x7ffad00b4910 2024-05-09T04:35:03.546 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.547+0000 7ffaccc00640 1 -- 172.21.15.145:0/4171145332 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5189 (secure 0 0 0) 0x7ffab8068250 con 0x7ffad00b4910 2024-05-09T04:35:03.546 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:35:03.547 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4499,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6834/3227728059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3227728059},{"type":"v1","addr":"172.21.15.192:6834","nonce":3227728059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4502,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1603013787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6833","nonce":1603013787},{"type":"v1","addr":"172.21.15.192:6835","nonce":1603013787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4672,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/68778418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":68778418},{"type":"v1","addr":"172.21.15.145:6837","nonce":68778418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T04:34:43.133970+0000","modified":"2024-05-09T04:35:00.107166+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4761},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4761":{"gid":4761,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.145:6835/4260798843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4260798843},{"type":"v1","addr":"172.21.15.145:6835","nonce":4260798843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5,6],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4761,"qdb_cluster":[4761]},"id":2}]} 2024-05-09T04:35:03.547 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 18 2024-05-09T04:35:03.549 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.549+0000 7ffad7800640 1 -- 172.21.15.145:0/4171145332 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ffaac0762d0 msgr2=0x7ffaac078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:03.550 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.549+0000 7ffad7800640 1 --2- 172.21.15.145:0/4171145332 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ffaac0762d0 0x7ffaac078790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ffac0006fd0 tx=0x7ffac0008040 comp rx=0 tx=0).stop 2024-05-09T04:35:03.550 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.549+0000 7ffad7800640 1 -- 172.21.15.145:0/4171145332 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffad00b4910 msgr2=0x7ffad01f5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:03.550 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.549+0000 7ffad7800640 1 --2- 172.21.15.145:0/4171145332 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffad00b4910 0x7ffad01f5cb0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ffab800c3d0 tx=0x7ffab800c890 comp rx=0 tx=0).stop 2024-05-09T04:35:03.550 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.549+0000 7ffad7800640 1 -- 172.21.15.145:0/4171145332 shutdown_connections 2024-05-09T04:35:03.550 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.549+0000 7ffad7800640 1 --2- 172.21.15.145:0/4171145332 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ffaac0762d0 0x7ffaac078790 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:03.550 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.549+0000 7ffad7800640 1 --2- 172.21.15.145:0/4171145332 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffad015d780 0x7ffad00afae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:03.550 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.549+0000 7ffad7800640 1 --2- 172.21.15.145:0/4171145332 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffad00b4910 0x7ffad01f5cb0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:03.550 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.549+0000 7ffad7800640 1 --2- 172.21.15.145:0/4171145332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffad006e860 0x7ffad0139830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:03.550 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.549+0000 7ffad7800640 1 -- 172.21.15.145:0/4171145332 >> 172.21.15.145:0/4171145332 conn(0x7ffad006c870 msgr2=0x7ffad015db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:03.550 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.550+0000 7ffad7800640 1 -- 172.21.15.145:0/4171145332 shutdown_connections 2024-05-09T04:35:03.550 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.550+0000 7ffad7800640 1 -- 172.21.15.145:0/4171145332 wait complete. 2024-05-09T04:35:03.569 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-05-09T04:35:03.810 INFO:teuthology.orchestra.run.smithi145.stdout:{"message":"","return_code":0} 2024-05-09T04:35:03.810 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-09T04:35:03.810 DEBUG:teuthology.orchestra.run.smithi145:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001fb.00000000 parent 2024-05-09T04:35:03.907 DEBUG:teuthology.orchestra.run.smithi145:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T04:35:03.908 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.908+0000 7faca8391f40 1 Processor -- start 2024-05-09T04:35:03.908 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.908+0000 7faca8391f40 1 -- start start 2024-05-09T04:35:03.909 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.909+0000 7faca8391f40 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55710b64db90 0x55710b632570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:03.909 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.909+0000 7faca8391f40 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55710b64e4b0 0x55710b632ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:03.909 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.909+0000 7faca8391f40 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55710b6d30d0 0x55710b62fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:03.909 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.909+0000 7faca8391f40 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55710b64cf10 con 0x55710b6d30d0 2024-05-09T04:35:03.909 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.909+0000 7faca8391f40 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55710b5a8f50 con 0x55710b64db90 2024-05-09T04:35:03.909 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.909+0000 7faca8391f40 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x55710b64ea30 con 0x55710b64e4b0 2024-05-09T04:35:03.912 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T04:35:03.915 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.910+0000 7faca6000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55710b6d30d0 0x55710b62fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:03.915 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.910+0000 7faca6000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55710b6d30d0 0x55710b62fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:53124/0 (socket says 172.21.15.145:53124) 2024-05-09T04:35:03.915 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.910+0000 7faca6000640 1 -- 172.21.15.145:0/3302618626 learned_addr learned my addr 172.21.15.145:0/3302618626 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:03.915 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.910+0000 7faca5600640 1 --2- 172.21.15.145:0/3302618626 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55710b64db90 0x55710b632570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:03.915 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.910+0000 7fac9cc00640 1 --2- 172.21.15.145:0/3302618626 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55710b64e4b0 0x55710b632ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:03.915 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.910+0000 7faca6000640 1 -- 172.21.15.145:0/3302618626 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55710b64e4b0 msgr2=0x55710b632ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:03.915 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.910+0000 7faca6000640 1 --2- 172.21.15.145:0/3302618626 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55710b64e4b0 0x55710b632ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:03.915 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.910+0000 7faca6000640 1 -- 172.21.15.145:0/3302618626 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55710b64db90 msgr2=0x55710b632570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:03.915 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.910+0000 7faca6000640 1 --2- 172.21.15.145:0/3302618626 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55710b64db90 0x55710b632570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:03.915 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.910+0000 7faca6000640 1 -- 172.21.15.145:0/3302618626 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55710b52bb30 con 0x55710b6d30d0 2024-05-09T04:35:03.915 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.910+0000 7faca6000640 1 --2- 172.21.15.145:0/3302618626 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55710b6d30d0 0x55710b62fb10 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7faca0005fd0 tx=0x7faca0050000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=638c5919182b3a95 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:03.916 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.911+0000 7fac9fe00640 1 -- 172.21.15.145:0/3302618626 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca006a070 con 0x55710b6d30d0 2024-05-09T04:35:03.916 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.911+0000 7fac9fe00640 1 -- 172.21.15.145:0/3302618626 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faca005da00 con 0x55710b6d30d0 2024-05-09T04:35:03.916 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.911+0000 7fac9fe00640 1 -- 172.21.15.145:0/3302618626 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca005ddf0 con 0x55710b6d30d0 2024-05-09T04:35:03.916 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.911+0000 7faca8391f40 1 -- 172.21.15.145:0/3302618626 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55710b6d30d0 msgr2=0x55710b62fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:03.916 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.911+0000 7faca8391f40 1 --2- 172.21.15.145:0/3302618626 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55710b6d30d0 0x55710b62fb10 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7faca0005fd0 tx=0x7faca0050000 comp rx=0 tx=0).stop 2024-05-09T04:35:03.916 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.911+0000 7faca8391f40 1 -- 172.21.15.145:0/3302618626 shutdown_connections 2024-05-09T04:35:03.916 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.911+0000 7faca8391f40 1 --2- 172.21.15.145:0/3302618626 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55710b6d30d0 0x55710b62fb10 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:03.916 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.911+0000 7faca8391f40 1 --2- 172.21.15.145:0/3302618626 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55710b64e4b0 0x55710b632ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:03.916 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.911+0000 7faca8391f40 1 --2- 172.21.15.145:0/3302618626 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55710b64db90 0x55710b632570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:03.916 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.911+0000 7faca8391f40 1 -- 172.21.15.145:0/3302618626 >> 172.21.15.145:0/3302618626 conn(0x55710b630c80 msgr2=0x55710b6d6d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:03.916 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.911+0000 7faca8391f40 1 -- 172.21.15.145:0/3302618626 shutdown_connections 2024-05-09T04:35:03.916 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.912+0000 7faca8391f40 1 -- 172.21.15.145:0/3302618626 wait complete. 2024-05-09T04:35:03.916 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.912+0000 7faca8391f40 1 Processor -- start 2024-05-09T04:35:03.916 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.912+0000 7faca8391f40 1 -- start start 2024-05-09T04:35:03.917 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.913+0000 7faca8391f40 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55710b64db90 0x55710b76ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:03.917 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.913+0000 7faca8391f40 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55710b64e4b0 0x55710b6cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:03.917 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.913+0000 7faca8391f40 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55710b6d30d0 0x55710b6c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:03.917 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.913+0000 7faca8391f40 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55710b6d2880 con 0x55710b64db90 2024-05-09T04:35:03.917 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.913+0000 7faca8391f40 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55710b6d25a0 con 0x55710b6d30d0 2024-05-09T04:35:03.917 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.913+0000 7faca8391f40 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x55710b6d7c60 con 0x55710b64e4b0 2024-05-09T04:35:03.917 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.914+0000 7faca5600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55710b64db90 0x55710b76ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:03.917 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.914+0000 7faca5600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55710b64db90 0x55710b76ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:53136/0 (socket says 172.21.15.145:53136) 2024-05-09T04:35:03.917 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.914+0000 7faca5600640 1 -- 172.21.15.145:0/3692022565 learned_addr learned my addr 172.21.15.145:0/3692022565 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:03.917 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.914+0000 7fac9cc00640 1 --2- 172.21.15.145:0/3692022565 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55710b64e4b0 0x55710b6cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:03.917 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.914+0000 7faca6000640 1 --2- 172.21.15.145:0/3692022565 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55710b6d30d0 0x55710b6c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:03.917 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.914+0000 7faca5600640 1 -- 172.21.15.145:0/3692022565 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55710b64e4b0 msgr2=0x55710b6cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:03.917 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.914+0000 7faca5600640 1 --2- 172.21.15.145:0/3692022565 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55710b64e4b0 0x55710b6cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:03.917 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.914+0000 7faca5600640 1 -- 172.21.15.145:0/3692022565 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55710b6d30d0 msgr2=0x55710b6c70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:03.917 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.914+0000 7faca5600640 1 --2- 172.21.15.145:0/3692022565 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55710b6d30d0 0x55710b6c70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:03.918 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.914+0000 7faca5600640 1 -- 172.21.15.145:0/3692022565 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55710b6caaa0 con 0x55710b64db90 2024-05-09T04:35:03.918 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.914+0000 7faca6000640 1 --2- 172.21.15.145:0/3692022565 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55710b6d30d0 0x55710b6c70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:35:03.918 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.915+0000 7faca5600640 1 --2- 172.21.15.145:0/3692022565 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55710b64db90 0x55710b76ea50 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fac9800ce10 tx=0x7fac9800a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:03.918 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.915+0000 7fac9ea00640 1 -- 172.21.15.145:0/3692022565 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9800ae60 con 0x55710b64db90 2024-05-09T04:35:03.918 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.915+0000 7fac9ea00640 1 -- 172.21.15.145:0/3692022565 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac980087e0 con 0x55710b64db90 2024-05-09T04:35:03.918 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.915+0000 7faca8391f40 1 -- 172.21.15.145:0/3692022565 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55710b6cad80 con 0x55710b64db90 2024-05-09T04:35:03.918 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.915+0000 7fac9ea00640 1 -- 172.21.15.145:0/3692022565 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac98008bd0 con 0x55710b64db90 2024-05-09T04:35:03.918 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.915+0000 7faca8391f40 1 -- 172.21.15.145:0/3692022565 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55710b6cb340 con 0x55710b64db90 2024-05-09T04:35:03.921 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.917+0000 7fac9ea00640 1 -- 172.21.15.145:0/3692022565 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fac9801e440 con 0x55710b64db90 2024-05-09T04:35:03.921 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.918+0000 7fac9ea00640 1 --2- 172.21.15.145:0/3692022565 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fac84074cb0 0x7fac84077170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:03.921 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.918+0000 7fac9ea00640 1 -- 172.21.15.145:0/3692022565 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 6104+0+0 (secure 0 0 0) 0x7fac98096760 con 0x55710b64db90 2024-05-09T04:35:03.921 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.918+0000 7fac9cc00640 1 --2- 172.21.15.145:0/3692022565 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fac84074cb0 0x7fac84077170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:03.921 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.918+0000 7faca8391f40 1 --2- 172.21.15.145:0/3692022565 >> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] conn(0x55710b6d0910 0x55710b6d0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:03.921 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.918+0000 7faca8391f40 1 -- 172.21.15.145:0/3692022565 --> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] -- osd_op(unknown.0.0:1 5.13 5:ca8fdbba:::100000001fb.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) -- 0x55710b6d1290 con 0x55710b6d0910 2024-05-09T04:35:03.921 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.919+0000 7faca6000640 1 --2- 172.21.15.145:0/3692022565 >> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] conn(0x55710b6d0910 0x55710b6d0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:03.921 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.919+0000 7fac9cc00640 1 --2- 172.21.15.145:0/3692022565 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fac84074cb0 0x7fac84077170 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x55710b6da280 tx=0x7fac90005c50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:03.921 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.920+0000 7faca6000640 1 --2- 172.21.15.145:0/3692022565 >> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] conn(0x55710b6d0910 0x55710b6d0d50 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:03.921 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.920+0000 7faca6000640 1 -- 172.21.15.145:0/3692022565 <== osd.3 v2:172.21.15.145:6808/337009826 1 ==== osd_op_reply(1 100000001fb.00000000 [getxattr (103) out=103b] v0'0 uv3 ondisk = 0) ==== 164+0+103 (crc 0 0 0) 0x7faca006d040 con 0x55710b6d0910 2024-05-09T04:35:03.921 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.920+0000 7faca8391f40 1 -- 172.21.15.145:0/3692022565 >> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] conn(0x55710b6d0910 msgr2=0x55710b6d0d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:03.922 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.920+0000 7faca8391f40 1 --2- 172.21.15.145:0/3692022565 >> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] conn(0x55710b6d0910 0x55710b6d0d50 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:03.922 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.920+0000 7faca8391f40 1 -- 172.21.15.145:0/3692022565 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fac84074cb0 msgr2=0x7fac84077170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:03.922 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.920+0000 7faca8391f40 1 --2- 172.21.15.145:0/3692022565 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fac84074cb0 0x7fac84077170 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x55710b6da280 tx=0x7fac90005c50 comp rx=0 tx=0).stop 2024-05-09T04:35:03.922 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.920+0000 7faca8391f40 1 -- 172.21.15.145:0/3692022565 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55710b64db90 msgr2=0x55710b76ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:03.922 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.920+0000 7faca8391f40 1 --2- 172.21.15.145:0/3692022565 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55710b64db90 0x55710b76ea50 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fac9800ce10 tx=0x7fac9800a750 comp rx=0 tx=0).stop 2024-05-09T04:35:03.922 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.920+0000 7faca6000640 1 -- 172.21.15.145:0/3692022565 reap_dead start 2024-05-09T04:35:03.922 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.920+0000 7faca8391f40 1 -- 172.21.15.145:0/3692022565 shutdown_connections 2024-05-09T04:35:03.922 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.920+0000 7faca8391f40 1 -- 172.21.15.145:0/3692022565 >> 172.21.15.145:0/3692022565 conn(0x55710b630c80 msgr2=0x55710b601570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:03.922 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.921+0000 7faca8391f40 1 -- 172.21.15.145:0/3692022565 shutdown_connections 2024-05-09T04:35:03.922 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:03.921+0000 7faca8391f40 1 -- 172.21.15.145:0/3692022565 wait complete. 2024-05-09T04:35:03.925 INFO:teuthology.orchestra.run.smithi145.stdout:a��alpha parent_b$ 2024-05-09T04:35:03.925 DEBUG:teuthology.orchestra.run.smithi145:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-05-09T04:35:04.057 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-05-09T04:35:04.057 INFO:teuthology.orchestra.run.smithi145.stdout: "ino": 1099511628283, 2024-05-09T04:35:04.057 INFO:teuthology.orchestra.run.smithi145.stdout: "ancestors": [ 2024-05-09T04:35:04.057 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-05-09T04:35:04.057 INFO:teuthology.orchestra.run.smithi145.stdout: "dirino": 1099511628284, 2024-05-09T04:35:04.057 INFO:teuthology.orchestra.run.smithi145.stdout: "dname": "alpha", 2024-05-09T04:35:04.057 INFO:teuthology.orchestra.run.smithi145.stdout: "version": 9 2024-05-09T04:35:04.057 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:35:04.057 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-05-09T04:35:04.057 INFO:teuthology.orchestra.run.smithi145.stdout: "dirino": 1, 2024-05-09T04:35:04.057 INFO:teuthology.orchestra.run.smithi145.stdout: "dname": "parent_b", 2024-05-09T04:35:04.057 INFO:teuthology.orchestra.run.smithi145.stdout: "version": 36 2024-05-09T04:35:04.058 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-05-09T04:35:04.058 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2024-05-09T04:35:04.058 INFO:teuthology.orchestra.run.smithi145.stdout: "pool": 6, 2024-05-09T04:35:04.058 INFO:teuthology.orchestra.run.smithi145.stdout: "old_pools": [ 2024-05-09T04:35:04.058 INFO:teuthology.orchestra.run.smithi145.stdout: 5 2024-05-09T04:35:04.058 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-05-09T04:35:04.058 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-05-09T04:35:04.058 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:35:04.062 DEBUG:teuthology.orchestra.run.smithi145:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool data_new getxattr 100000001fb.00000000 parent 2024-05-09T04:35:04.213 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.213+0000 7f2410325f40 1 Processor -- start 2024-05-09T04:35:04.213 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.213+0000 7f2410325f40 1 -- start start 2024-05-09T04:35:04.215 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.214+0000 7f2410325f40 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x56329d7edee0 0x56329d888760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:04.215 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.214+0000 7f2410325f40 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56329d807b90 0x56329d88b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:04.215 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.214+0000 7f2410325f40 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56329d88d0d0 0x56329d7ec520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:04.216 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.214+0000 7f2410325f40 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56329d806f10 con 0x56329d807b90 2024-05-09T04:35:04.216 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.214+0000 7f2410325f40 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x56329d762f50 con 0x56329d88d0d0 2024-05-09T04:35:04.216 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.214+0000 7f2410325f40 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x56329d8940d0 con 0x56329d7edee0 2024-05-09T04:35:04.216 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.214+0000 7f240c000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56329d807b90 0x56329d88b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:04.216 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.215+0000 7f240c000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56329d807b90 0x56329d88b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:53144/0 (socket says 172.21.15.145:53144) 2024-05-09T04:35:04.216 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.215+0000 7f240c000640 1 -- 172.21.15.145:0/2648614693 learned_addr learned my addr 172.21.15.145:0/2648614693 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:04.216 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.215+0000 7f240ca00640 1 --2- 172.21.15.145:0/2648614693 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x56329d7edee0 0x56329d888760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:04.216 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.215+0000 7f240d400640 1 --2- 172.21.15.145:0/2648614693 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56329d88d0d0 0x56329d7ec520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:04.216 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.215+0000 7f240c000640 1 -- 172.21.15.145:0/2648614693 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x56329d7edee0 msgr2=0x56329d888760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:04.216 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.215+0000 7f240c000640 1 --2- 172.21.15.145:0/2648614693 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x56329d7edee0 0x56329d888760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:04.216 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.215+0000 7f240c000640 1 -- 172.21.15.145:0/2648614693 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56329d88d0d0 msgr2=0x56329d7ec520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:04.216 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.215+0000 7f240c000640 1 --2- 172.21.15.145:0/2648614693 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56329d88d0d0 0x56329d7ec520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:04.216 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.215+0000 7f240c000640 1 -- 172.21.15.145:0/2648614693 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56329d6e5b30 con 0x56329d807b90 2024-05-09T04:35:04.216 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.215+0000 7f240c000640 1 --2- 172.21.15.145:0/2648614693 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56329d807b90 0x56329d88b250 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f23f4001eb0 tx=0x7f23f400b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dd797b0b2f3357e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:04.217 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.216+0000 7f240ac00640 1 -- 172.21.15.145:0/2648614693 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f4016070 con 0x56329d807b90 2024-05-09T04:35:04.217 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.216+0000 7f240ac00640 1 -- 172.21.15.145:0/2648614693 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23f400d6d0 con 0x56329d807b90 2024-05-09T04:35:04.217 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.216+0000 7f240ac00640 1 -- 172.21.15.145:0/2648614693 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f400db00 con 0x56329d807b90 2024-05-09T04:35:04.219 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.216+0000 7f2410325f40 1 -- 172.21.15.145:0/2648614693 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56329d807b90 msgr2=0x56329d88b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:04.220 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.216+0000 7f2410325f40 1 --2- 172.21.15.145:0/2648614693 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56329d807b90 0x56329d88b250 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f23f4001eb0 tx=0x7f23f400b650 comp rx=0 tx=0).stop 2024-05-09T04:35:04.220 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.216+0000 7f2410325f40 1 -- 172.21.15.145:0/2648614693 shutdown_connections 2024-05-09T04:35:04.220 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.216+0000 7f2410325f40 1 --2- 172.21.15.145:0/2648614693 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56329d88d0d0 0x56329d7ec520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:04.220 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.216+0000 7f2410325f40 1 --2- 172.21.15.145:0/2648614693 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56329d807b90 0x56329d88b250 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:04.220 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.216+0000 7f2410325f40 1 --2- 172.21.15.145:0/2648614693 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x56329d7edee0 0x56329d888760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:04.220 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.216+0000 7f2410325f40 1 -- 172.21.15.145:0/2648614693 >> 172.21.15.145:0/2648614693 conn(0x56329d7eac80 msgr2=0x56329d892450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:04.220 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.216+0000 7f2410325f40 1 -- 172.21.15.145:0/2648614693 shutdown_connections 2024-05-09T04:35:04.220 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.216+0000 7f2410325f40 1 -- 172.21.15.145:0/2648614693 wait complete. 2024-05-09T04:35:04.220 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.217+0000 7f2410325f40 1 Processor -- start 2024-05-09T04:35:04.220 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.217+0000 7f2410325f40 1 -- start start 2024-05-09T04:35:04.220 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.218+0000 7f2410325f40 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56329d7edee0 0x56329d880d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:04.220 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.218+0000 7f2410325f40 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56329d807b90 0x56329d884eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:04.220 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.218+0000 7f2410325f40 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x56329d88d0d0 0x56329d8853f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:04.221 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.218+0000 7f2410325f40 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x56329d889710 con 0x56329d807b90 2024-05-09T04:35:04.221 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.218+0000 7f2410325f40 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x56329d889f60 con 0x56329d7edee0 2024-05-09T04:35:04.221 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.218+0000 7f2410325f40 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x56329d888e80 con 0x56329d88d0d0 2024-05-09T04:35:04.221 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.219+0000 7f240c000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56329d807b90 0x56329d884eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:04.221 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.219+0000 7f240c000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56329d807b90 0x56329d884eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:53146/0 (socket says 172.21.15.145:53146) 2024-05-09T04:35:04.221 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.219+0000 7f240c000640 1 -- 172.21.15.145:0/2545899350 learned_addr learned my addr 172.21.15.145:0/2545899350 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:04.221 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.219+0000 7f240ca00640 1 --2- 172.21.15.145:0/2545899350 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56329d7edee0 0x56329d880d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:04.221 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.219+0000 7f240d400640 1 --2- 172.21.15.145:0/2545899350 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x56329d88d0d0 0x56329d8853f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:04.221 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.219+0000 7f240c000640 1 -- 172.21.15.145:0/2545899350 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x56329d88d0d0 msgr2=0x56329d8853f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:04.221 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.219+0000 7f240c000640 1 --2- 172.21.15.145:0/2545899350 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x56329d88d0d0 0x56329d8853f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:04.221 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.219+0000 7f240c000640 1 -- 172.21.15.145:0/2545899350 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56329d7edee0 msgr2=0x56329d880d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:04.221 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.219+0000 7f240c000640 1 --2- 172.21.15.145:0/2545899350 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56329d7edee0 0x56329d880d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:04.221 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.219+0000 7f240c000640 1 -- 172.21.15.145:0/2545899350 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56329d7bbea0 con 0x56329d807b90 2024-05-09T04:35:04.221 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.219+0000 7f240ca00640 1 --2- 172.21.15.145:0/2545899350 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x56329d7edee0 0x56329d880d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:35:04.221 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.219+0000 7f240c000640 1 --2- 172.21.15.145:0/2545899350 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56329d807b90 0x56329d884eb0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f23f400b690 tx=0x7f23f4000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:04.222 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.219+0000 7f2409800640 1 -- 172.21.15.145:0/2545899350 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f4022070 con 0x56329d807b90 2024-05-09T04:35:04.222 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.220+0000 7f2409800640 1 -- 172.21.15.145:0/2545899350 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23f4010850 con 0x56329d807b90 2024-05-09T04:35:04.222 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.220+0000 7f2409800640 1 -- 172.21.15.145:0/2545899350 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f4016070 con 0x56329d807b90 2024-05-09T04:35:04.222 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.220+0000 7f2410325f40 1 -- 172.21.15.145:0/2545899350 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56329d883260 con 0x56329d807b90 2024-05-09T04:35:04.222 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.220+0000 7f2410325f40 1 -- 172.21.15.145:0/2545899350 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56329d883820 con 0x56329d807b90 2024-05-09T04:35:04.223 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.222+0000 7f2409800640 1 -- 172.21.15.145:0/2545899350 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f23f4025070 con 0x56329d807b90 2024-05-09T04:35:04.223 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.223+0000 7f2409800640 1 --2- 172.21.15.145:0/2545899350 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f23e8074cb0 0x7f23e8077170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:04.223 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.223+0000 7f2409800640 1 -- 172.21.15.145:0/2545899350 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 6104+0+0 (secure 0 0 0) 0x7f23f4097390 con 0x56329d807b90 2024-05-09T04:35:04.223 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.223+0000 7f240ca00640 1 --2- 172.21.15.145:0/2545899350 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f23e8074cb0 0x7f23e8077170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:04.224 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.223+0000 7f2410325f40 1 --2- 172.21.15.145:0/2545899350 >> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] conn(0x56329d881ce0 0x56329d886710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:04.224 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.223+0000 7f2410325f40 1 -- 172.21.15.145:0/2545899350 --> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] -- osd_op(unknown.0.0:1 6.13 6:ca8fdbba:::100000001fb.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) -- 0x56329d886c50 con 0x56329d881ce0 2024-05-09T04:35:04.224 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.224+0000 7f240d400640 1 --2- 172.21.15.145:0/2545899350 >> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] conn(0x56329d881ce0 0x56329d886710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:04.224 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.224+0000 7f240ca00640 1 --2- 172.21.15.145:0/2545899350 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f23e8074cb0 0x7f23e8077170 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f23fc009640 tx=0x7f23fc007040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:04.225 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.224+0000 7f240d400640 1 --2- 172.21.15.145:0/2545899350 >> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] conn(0x56329d881ce0 0x56329d886710 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:04.225 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.225+0000 7f240d400640 1 -- 172.21.15.145:0/2545899350 <== osd.3 v2:172.21.15.145:6808/337009826 1 ==== osd_op_reply(1 100000001fb.00000000 [getxattr (103) out=103b] v0'0 uv2 ondisk = 0) ==== 164+0+103 (crc 0 0 0) 0x7f2404057030 con 0x56329d881ce0 2024-05-09T04:35:04.225 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.225+0000 7f2410325f40 1 -- 172.21.15.145:0/2545899350 >> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] conn(0x56329d881ce0 msgr2=0x56329d886710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:04.225 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.225+0000 7f2410325f40 1 --2- 172.21.15.145:0/2545899350 >> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] conn(0x56329d881ce0 0x56329d886710 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:04.225 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.225+0000 7f2410325f40 1 -- 172.21.15.145:0/2545899350 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f23e8074cb0 msgr2=0x7f23e8077170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:04.226 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.225+0000 7f2410325f40 1 --2- 172.21.15.145:0/2545899350 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f23e8074cb0 0x7f23e8077170 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f23fc009640 tx=0x7f23fc007040 comp rx=0 tx=0).stop 2024-05-09T04:35:04.226 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.225+0000 7f2410325f40 1 -- 172.21.15.145:0/2545899350 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56329d807b90 msgr2=0x56329d884eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:04.226 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.225+0000 7f2410325f40 1 --2- 172.21.15.145:0/2545899350 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x56329d807b90 0x56329d884eb0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f23f400b690 tx=0x7f23f4000ed0 comp rx=0 tx=0).stop 2024-05-09T04:35:04.226 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.225+0000 7f240d400640 1 -- 172.21.15.145:0/2545899350 reap_dead start 2024-05-09T04:35:04.226 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.225+0000 7f2410325f40 1 -- 172.21.15.145:0/2545899350 shutdown_connections 2024-05-09T04:35:04.226 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.225+0000 7f2410325f40 1 -- 172.21.15.145:0/2545899350 >> 172.21.15.145:0/2545899350 conn(0x56329d7eac80 msgr2=0x56329d808240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:04.226 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.225+0000 7f2410325f40 1 -- 172.21.15.145:0/2545899350 shutdown_connections 2024-05-09T04:35:04.226 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.225+0000 7f2410325f40 1 -- 172.21.15.145:0/2545899350 wait complete. 2024-05-09T04:35:04.227 INFO:teuthology.orchestra.run.smithi145.stdout:a��alpha 2024-05-09T04:35:04.230 INFO:teuthology.orchestra.run.smithi145.stdout:parent_c+ 2024-05-09T04:35:04.230 DEBUG:teuthology.orchestra.run.smithi145:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-05-09T04:35:04.333 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-05-09T04:35:04.333 INFO:teuthology.orchestra.run.smithi145.stdout: "ino": 1099511628283, 2024-05-09T04:35:04.333 INFO:teuthology.orchestra.run.smithi145.stdout: "ancestors": [ 2024-05-09T04:35:04.333 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-05-09T04:35:04.333 INFO:teuthology.orchestra.run.smithi145.stdout: "dirino": 1099511628285, 2024-05-09T04:35:04.333 INFO:teuthology.orchestra.run.smithi145.stdout: "dname": "alpha", 2024-05-09T04:35:04.334 INFO:teuthology.orchestra.run.smithi145.stdout: "version": 10 2024-05-09T04:35:04.334 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:35:04.334 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-05-09T04:35:04.334 INFO:teuthology.orchestra.run.smithi145.stdout: "dirino": 1, 2024-05-09T04:35:04.334 INFO:teuthology.orchestra.run.smithi145.stdout: "dname": "parent_c", 2024-05-09T04:35:04.334 INFO:teuthology.orchestra.run.smithi145.stdout: "version": 43 2024-05-09T04:35:04.334 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-05-09T04:35:04.334 INFO:teuthology.orchestra.run.smithi145.stdout: ], 2024-05-09T04:35:04.334 INFO:teuthology.orchestra.run.smithi145.stdout: "pool": 6, 2024-05-09T04:35:04.334 INFO:teuthology.orchestra.run.smithi145.stdout: "old_pools": [ 2024-05-09T04:35:04.334 INFO:teuthology.orchestra.run.smithi145.stdout: 5 2024-05-09T04:35:04.334 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-05-09T04:35:04.334 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-05-09T04:35:04.335 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:35:04.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:35:04.339 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.file.layout.object_size -v 8388608 ./parent_c/alpha) 2024-05-09T04:35:04.397 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T04:35:04.595 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.596+0000 7f626da00640 1 Processor -- start 2024-05-09T04:35:04.596 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.596+0000 7f626da00640 1 -- start start 2024-05-09T04:35:04.597 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.597+0000 7f626da00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f626815fbc0 0x7f6268142200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:04.597 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.597+0000 7f626da00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f626815ffa0 0x7f62680cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:04.597 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.597+0000 7f626da00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f626813f340 con 0x7f62680cfe10 2024-05-09T04:35:04.597 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.597+0000 7f626da00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f6268072220 con 0x7f626815fbc0 2024-05-09T04:35:04.597 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.597+0000 7f626da00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f62680d02b0 con 0x7f626815ffa0 2024-05-09T04:35:04.598 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.597+0000 7f6266a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f626815fbc0 0x7f6268142200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:04.598 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.598+0000 7f6267400640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f62680cfe10 0x7f626815c5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:45208/0 (socket says 172.21.15.145:45208) 2024-05-09T04:35:04.598 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.598+0000 7f6267400640 1 -- 172.21.15.145:0/876653138 learned_addr learned my addr 172.21.15.145:0/876653138 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:35:04.598 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.598+0000 7f6266000640 1 --2- 172.21.15.145:0/876653138 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f626815ffa0 0x7f62680cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:04.598 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.598+0000 7f6266a00640 1 -- 172.21.15.145:0/876653138 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f626815ffa0 msgr2=0x7f62680cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:04.598 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.598+0000 7f6266a00640 1 --2- 172.21.15.145:0/876653138 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f626815ffa0 0x7f62680cf8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:04.599 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.598+0000 7f6266a00640 1 -- 172.21.15.145:0/876653138 >> v1:172.21.15.145:6789/0 conn(0x7f62680cfe10 legacy=0x7f626815c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:04.599 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.598+0000 7f6266a00640 1 -- 172.21.15.145:0/876653138 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62680d0590 con 0x7f626815fbc0 2024-05-09T04:35:04.599 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.598+0000 7f6265600640 1 -- 172.21.15.145:0/876653138 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 997817280 0 0) 0x7f6260001490 con 0x7f62680cfe10 2024-05-09T04:35:04.599 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.599+0000 7f6266a00640 1 --2- 172.21.15.145:0/876653138 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f626815fbc0 0x7f6268142200 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f62580097a0 tx=0x7f625802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7517ce8b07949588 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:04.599 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.599+0000 7f6265600640 1 -- 172.21.15.145:0/876653138 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6258037070 con 0x7f626815fbc0 2024-05-09T04:35:04.599 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.600+0000 7f6265600640 1 -- 172.21.15.145:0/876653138 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f625802ae50 con 0x7f626815fbc0 2024-05-09T04:35:04.599 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.600+0000 7f6265600640 1 -- 172.21.15.145:0/876653138 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6258032660 con 0x7f626815fbc0 2024-05-09T04:35:04.600 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.600+0000 7f626da00640 1 -- 172.21.15.145:0/876653138 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f626815fbc0 msgr2=0x7f6268142200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:04.600 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.600+0000 7f626da00640 1 --2- 172.21.15.145:0/876653138 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f626815fbc0 0x7f6268142200 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f62580097a0 tx=0x7f625802a100 comp rx=0 tx=0).stop 2024-05-09T04:35:04.600 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.600+0000 7f626da00640 1 -- 172.21.15.145:0/876653138 shutdown_connections 2024-05-09T04:35:04.600 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.600+0000 7f626da00640 1 --2- 172.21.15.145:0/876653138 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f626815ffa0 0x7f62680cf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:04.600 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.600+0000 7f626da00640 1 --2- 172.21.15.145:0/876653138 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f626815fbc0 0x7f6268142200 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:04.601 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.600+0000 7f626da00640 1 -- 172.21.15.145:0/876653138 >> 172.21.15.145:0/876653138 conn(0x7f626806c870 msgr2=0x7f626806cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:04.601 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.600+0000 7f626da00640 1 -- 172.21.15.145:0/876653138 shutdown_connections 2024-05-09T04:35:04.601 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.600+0000 7f626da00640 1 -- 172.21.15.145:0/876653138 wait complete. 2024-05-09T04:35:04.602 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.602+0000 7f626da00640 1 Processor -- start 2024-05-09T04:35:04.602 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.602+0000 7f626da00640 1 -- start start 2024-05-09T04:35:04.603 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.603+0000 7f626da00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62680cfe10 0x7f6268140e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:04.603 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.603+0000 7f626da00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f626815fbc0 0x7f6268151510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:04.603 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.603+0000 7f626da00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f626815ffa0 0x7f62681551a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:04.603 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.603+0000 7f626da00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6268160400 con 0x7f62680cfe10 2024-05-09T04:35:04.603 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.603+0000 7f626da00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f62680d0b50 con 0x7f626815fbc0 2024-05-09T04:35:04.603 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.603+0000 7f626da00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f62680736c0 con 0x7f626815ffa0 2024-05-09T04:35:04.603 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.603+0000 7f6266a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62680cfe10 0x7f6268140e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:04.603 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.603+0000 7f6266a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62680cfe10 0x7f6268140e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:53156/0 (socket says 172.21.15.145:53156) 2024-05-09T04:35:04.603 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.603+0000 7f6266a00640 1 -- 172.21.15.145:0/1079195737 learned_addr learned my addr 172.21.15.145:0/1079195737 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:04.604 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.604+0000 7f6266a00640 1 -- 172.21.15.145:0/1079195737 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f626815ffa0 msgr2=0x7f62681551a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:04.604 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.604+0000 7f6266a00640 1 --2- 172.21.15.145:0/1079195737 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f626815ffa0 0x7f62681551a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:04.605 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.604+0000 7f6266a00640 1 -- 172.21.15.145:0/1079195737 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f626815fbc0 msgr2=0x7f6268151510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:04.605 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.604+0000 7f6266a00640 1 --2- 172.21.15.145:0/1079195737 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f626815fbc0 0x7f6268151510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:04.605 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.604+0000 7f6266a00640 1 -- 172.21.15.145:0/1079195737 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6268141390 con 0x7f62680cfe10 2024-05-09T04:35:04.605 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.604+0000 7f6266a00640 1 --2- 172.21.15.145:0/1079195737 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62680cfe10 0x7f6268140e50 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f62580098c0 tx=0x7f62580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:04.605 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.604+0000 7f625ea00640 1 -- 172.21.15.145:0/1079195737 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6258041020 con 0x7f62680cfe10 2024-05-09T04:35:04.605 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.604+0000 7f625ea00640 1 -- 172.21.15.145:0/1079195737 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f625803c670 con 0x7f62680cfe10 2024-05-09T04:35:04.605 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.604+0000 7f625ea00640 1 -- 172.21.15.145:0/1079195737 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6258037040 con 0x7f62680cfe10 2024-05-09T04:35:04.605 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.604+0000 7f626da00640 1 -- 172.21.15.145:0/1079195737 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6268154460 con 0x7f62680cfe10 2024-05-09T04:35:04.605 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.605+0000 7f626da00640 1 -- 172.21.15.145:0/1079195737 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6268140370 con 0x7f62680cfe10 2024-05-09T04:35:04.605 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.606+0000 7f626da00640 1 -- 172.21.15.145:0/1079195737 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62680ce160 con 0x7f62680cfe10 2024-05-09T04:35:04.606 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.607+0000 7f625ea00640 1 -- 172.21.15.145:0/1079195737 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6258044020 con 0x7f62680cfe10 2024-05-09T04:35:04.608 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.608+0000 7f625ea00640 1 --2- 172.21.15.145:0/1079195737 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f62480762d0 0x7f6248078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:04.608 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.608+0000 7f625ea00640 1 -- 172.21.15.145:0/1079195737 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 6104+0+0 (secure 0 0 0) 0x7f62580b7de0 con 0x7f62680cfe10 2024-05-09T04:35:04.608 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.608+0000 7f6266000640 1 --2- 172.21.15.145:0/1079195737 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f62480762d0 0x7f6248078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:04.608 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.609+0000 7f6266000640 1 --2- 172.21.15.145:0/1079195737 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f62480762d0 0x7f6248078790 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6254006fd0 tx=0x7f6254008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:04.612 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.612+0000 7f625ea00640 1 -- 172.21.15.145:0/1079195737 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f625804b070 con 0x7f62680cfe10 2024-05-09T04:35:04.853 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.853+0000 7f626da00640 1 -- 172.21.15.145:0/1079195737 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f62680b4070 con 0x7f62680cfe10 2024-05-09T04:35:04.854 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.854+0000 7f625ea00640 1 -- 172.21.15.145:0/1079195737 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5189 (secure 0 0 0) 0x7f6258081510 con 0x7f62680cfe10 2024-05-09T04:35:04.854 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:35:04.854 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4499,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6834/3227728059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3227728059},{"type":"v1","addr":"172.21.15.192:6834","nonce":3227728059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4502,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1603013787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6833","nonce":1603013787},{"type":"v1","addr":"172.21.15.192:6835","nonce":1603013787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4672,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/68778418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":68778418},{"type":"v1","addr":"172.21.15.145:6837","nonce":68778418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T04:34:43.133970+0000","modified":"2024-05-09T04:35:00.107166+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4761},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4761":{"gid":4761,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.145:6835/4260798843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4260798843},{"type":"v1","addr":"172.21.15.145:6835","nonce":4260798843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5,6],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4761,"qdb_cluster":[4761]},"id":2}]} 2024-05-09T04:35:04.855 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 18 2024-05-09T04:35:04.856 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.856+0000 7f626da00640 1 -- 172.21.15.145:0/1079195737 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f62480762d0 msgr2=0x7f6248078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:04.857 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.856+0000 7f626da00640 1 --2- 172.21.15.145:0/1079195737 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f62480762d0 0x7f6248078790 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6254006fd0 tx=0x7f6254008040 comp rx=0 tx=0).stop 2024-05-09T04:35:04.857 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.856+0000 7f626da00640 1 -- 172.21.15.145:0/1079195737 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62680cfe10 msgr2=0x7f6268140e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:04.857 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.856+0000 7f626da00640 1 --2- 172.21.15.145:0/1079195737 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62680cfe10 0x7f6268140e50 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f62580098c0 tx=0x7f62580046c0 comp rx=0 tx=0).stop 2024-05-09T04:35:04.857 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.857+0000 7f626da00640 1 -- 172.21.15.145:0/1079195737 shutdown_connections 2024-05-09T04:35:04.857 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.857+0000 7f626da00640 1 --2- 172.21.15.145:0/1079195737 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f62480762d0 0x7f6248078790 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:04.857 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.857+0000 7f626da00640 1 --2- 172.21.15.145:0/1079195737 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f626815ffa0 0x7f62681551a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:04.857 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.857+0000 7f626da00640 1 --2- 172.21.15.145:0/1079195737 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f626815fbc0 0x7f6268151510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:04.857 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.857+0000 7f626da00640 1 --2- 172.21.15.145:0/1079195737 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f62680cfe10 0x7f6268140e50 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:04.857 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.857+0000 7f626da00640 1 -- 172.21.15.145:0/1079195737 >> 172.21.15.145:0/1079195737 conn(0x7f626806c870 msgr2=0x7f62680b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:04.857 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.857+0000 7f626da00640 1 -- 172.21.15.145:0/1079195737 shutdown_connections 2024-05-09T04:35:04.857 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:04.857+0000 7f626da00640 1 -- 172.21.15.145:0/1079195737 wait complete. 2024-05-09T04:35:04.874 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-05-09T04:35:05.099 INFO:teuthology.orchestra.run.smithi145.stdout:{"message":"","return_code":0} 2024-05-09T04:35:05.100 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-05-09T04:35:05.100 DEBUG:teuthology.orchestra.run.smithi145:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool data_new getxattr 100000001fb.00000000 layout 2024-05-09T04:35:05.204 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.204+0000 7fb62f487f40 1 Processor -- start 2024-05-09T04:35:05.204 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.204+0000 7fb62f487f40 1 -- start start 2024-05-09T04:35:05.205 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.205+0000 7fb62f487f40 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fd44b3dee0 0x55fd44be1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:05.206 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.205+0000 7fb62f487f40 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55fd44b57b90 0x55fd44b3c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:05.206 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.205+0000 7fb62f487f40 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fd44bdd0d0 0x55fd44bdba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:05.206 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.205+0000 7fb62f487f40 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55fd44b56f10 con 0x55fd44bdd0d0 2024-05-09T04:35:05.206 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.205+0000 7fb62f487f40 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55fd44ab2f50 con 0x55fd44b3dee0 2024-05-09T04:35:05.206 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.205+0000 7fb62f487f40 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x55fd44be40d0 con 0x55fd44b57b90 2024-05-09T04:35:05.206 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.205+0000 7fb62d000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fd44bdd0d0 0x55fd44bdba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:05.206 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.205+0000 7fb62d000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fd44bdd0d0 0x55fd44bdba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:53174/0 (socket says 172.21.15.145:53174) 2024-05-09T04:35:05.206 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.205+0000 7fb62d000640 1 -- 172.21.15.145:0/807191407 learned_addr learned my addr 172.21.15.145:0/807191407 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:05.206 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.206+0000 7fb62d000640 1 -- 172.21.15.145:0/807191407 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55fd44b57b90 msgr2=0x55fd44b3c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:05.206 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.206+0000 7fb62d000640 1 --2- 172.21.15.145:0/807191407 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55fd44b57b90 0x55fd44b3c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:05.206 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.206+0000 7fb62d000640 1 -- 172.21.15.145:0/807191407 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fd44b3dee0 msgr2=0x55fd44be1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:05.206 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.206+0000 7fb62d000640 1 --2- 172.21.15.145:0/807191407 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fd44b3dee0 0x55fd44be1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:05.206 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.206+0000 7fb62d000640 1 -- 172.21.15.145:0/807191407 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd44a35b30 con 0x55fd44bdd0d0 2024-05-09T04:35:05.206 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.206+0000 7fb62d000640 1 --2- 172.21.15.145:0/807191407 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fd44bdd0d0 0x55fd44bdba30 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb62400a5a0 tx=0x7fb624051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95fbfe59ce01f968 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:05.208 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.206+0000 7fb62b200640 1 -- 172.21.15.145:0/807191407 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb624068070 con 0x55fd44bdd0d0 2024-05-09T04:35:05.208 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.206+0000 7fb62b200640 1 -- 172.21.15.145:0/807191407 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb624051820 con 0x55fd44bdd0d0 2024-05-09T04:35:05.208 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.206+0000 7fb62b200640 1 -- 172.21.15.145:0/807191407 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb624055070 con 0x55fd44bdd0d0 2024-05-09T04:35:05.208 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.207+0000 7fb62f487f40 1 -- 172.21.15.145:0/807191407 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fd44bdd0d0 msgr2=0x55fd44bdba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:05.208 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.207+0000 7fb62f487f40 1 --2- 172.21.15.145:0/807191407 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fd44bdd0d0 0x55fd44bdba30 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb62400a5a0 tx=0x7fb624051000 comp rx=0 tx=0).stop 2024-05-09T04:35:05.208 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.207+0000 7fb62f487f40 1 -- 172.21.15.145:0/807191407 shutdown_connections 2024-05-09T04:35:05.208 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.207+0000 7fb62f487f40 1 --2- 172.21.15.145:0/807191407 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fd44bdd0d0 0x55fd44bdba30 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:05.208 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.207+0000 7fb62f487f40 1 --2- 172.21.15.145:0/807191407 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55fd44b57b90 0x55fd44b3c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:05.208 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.207+0000 7fb62f487f40 1 --2- 172.21.15.145:0/807191407 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fd44b3dee0 0x55fd44be1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:05.208 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.207+0000 7fb62f487f40 1 -- 172.21.15.145:0/807191407 >> 172.21.15.145:0/807191407 conn(0x55fd44b3ac80 msgr2=0x55fd44be23c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:05.208 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.207+0000 7fb62f487f40 1 -- 172.21.15.145:0/807191407 shutdown_connections 2024-05-09T04:35:05.208 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.207+0000 7fb62f487f40 1 -- 172.21.15.145:0/807191407 wait complete. 2024-05-09T04:35:05.209 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.208+0000 7fb62f487f40 1 Processor -- start 2024-05-09T04:35:05.209 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.208+0000 7fb62f487f40 1 -- start start 2024-05-09T04:35:05.210 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.209+0000 7fb62f487f40 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fd44b3dee0 0x55fd44bdc570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:05.210 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.209+0000 7fb62f487f40 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55fd44b57b90 0x55fd44bde160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:05.211 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.209+0000 7fb62f487f40 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fd44bdd0d0 0x55fd44bc56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:05.211 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.209+0000 7fb62f487f40 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55fd44bdaea0 con 0x55fd44bdd0d0 2024-05-09T04:35:05.211 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.209+0000 7fb62f487f40 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x55fd44b3a590 con 0x55fd44b3dee0 2024-05-09T04:35:05.211 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.209+0000 7fb62f487f40 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x55fd44b39f40 con 0x55fd44b57b90 2024-05-09T04:35:05.211 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.209+0000 7fb62d000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fd44bdd0d0 0x55fd44bc56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:05.211 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.209+0000 7fb62d000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fd44bdd0d0 0x55fd44bc56c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:53186/0 (socket says 172.21.15.145:53186) 2024-05-09T04:35:05.211 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.209+0000 7fb62d000640 1 -- 172.21.15.145:0/3539238935 learned_addr learned my addr 172.21.15.145:0/3539238935 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:05.211 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.209+0000 7fb62d000640 1 -- 172.21.15.145:0/3539238935 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55fd44b57b90 msgr2=0x55fd44bde160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:05.211 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.209+0000 7fb62d000640 1 --2- 172.21.15.145:0/3539238935 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55fd44b57b90 0x55fd44bde160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:05.211 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.209+0000 7fb62d000640 1 -- 172.21.15.145:0/3539238935 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fd44b3dee0 msgr2=0x55fd44bdc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:05.211 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.209+0000 7fb62d000640 1 --2- 172.21.15.145:0/3539238935 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55fd44b3dee0 0x55fd44bdc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:05.211 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.209+0000 7fb62d000640 1 -- 172.21.15.145:0/3539238935 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd44bde840 con 0x55fd44bdd0d0 2024-05-09T04:35:05.211 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.210+0000 7fb62d000640 1 --2- 172.21.15.145:0/3539238935 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fd44bdd0d0 0x55fd44bc56c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fb624025e20 tx=0x7fb624002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:05.211 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.210+0000 7fb629e00640 1 -- 172.21.15.145:0/3539238935 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb624070020 con 0x55fd44bdd0d0 2024-05-09T04:35:05.211 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.210+0000 7fb629e00640 1 -- 172.21.15.145:0/3539238935 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb624051df0 con 0x55fd44bdd0d0 2024-05-09T04:35:05.212 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.210+0000 7fb62f487f40 1 -- 172.21.15.145:0/3539238935 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55fd44bcf4c0 con 0x55fd44bdd0d0 2024-05-09T04:35:05.212 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.210+0000 7fb629e00640 1 -- 172.21.15.145:0/3539238935 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb624068040 con 0x55fd44bdd0d0 2024-05-09T04:35:05.212 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.210+0000 7fb62f487f40 1 -- 172.21.15.145:0/3539238935 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd44bcfa80 con 0x55fd44bdd0d0 2024-05-09T04:35:05.212 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.212+0000 7fb629e00640 1 -- 172.21.15.145:0/3539238935 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb624073020 con 0x55fd44bdd0d0 2024-05-09T04:35:05.213 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.213+0000 7fb629e00640 1 --2- 172.21.15.145:0/3539238935 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fb608074cb0 0x7fb608077170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:05.213 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.213+0000 7fb629e00640 1 -- 172.21.15.145:0/3539238935 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 6104+0+0 (secure 0 0 0) 0x7fb6240db850 con 0x55fd44bdd0d0 2024-05-09T04:35:05.213 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.213+0000 7fb62c600640 1 --2- 172.21.15.145:0/3539238935 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fb608074cb0 0x7fb608077170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:05.214 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.214+0000 7fb62f487f40 1 --2- 172.21.15.145:0/3539238935 >> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] conn(0x55fd44bcffa0 0x55fd44bd0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:05.214 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.214+0000 7fb62f487f40 1 -- 172.21.15.145:0/3539238935 --> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] -- osd_op(unknown.0.0:1 6.13 6:ca8fdbba:::100000001fb.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) -- 0x55fd44bd0990 con 0x55fd44bcffa0 2024-05-09T04:35:05.214 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.214+0000 7fb623c00640 1 --2- 172.21.15.145:0/3539238935 >> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] conn(0x55fd44bcffa0 0x55fd44bd0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:05.215 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.214+0000 7fb62c600640 1 --2- 172.21.15.145:0/3539238935 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fb608074cb0 0x7fb608077170 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb61c005fd0 tx=0x7fb61c007040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:05.215 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.215+0000 7fb623c00640 1 --2- 172.21.15.145:0/3539238935 >> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] conn(0x55fd44bcffa0 0x55fd44bd0450 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:05.215 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.215+0000 7fb623c00640 1 -- 172.21.15.145:0/3539238935 <== osd.3 v2:172.21.15.145:6808/337009826 1 ==== osd_op_reply(1 100000001fb.00000000 [getxattr (30) out=30b] v0'0 uv3 ondisk = 0) ==== 164+0+30 (crc 0 0 0) 0x7fb61400d2f0 con 0x55fd44bcffa0 2024-05-09T04:35:05.215 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.215+0000 7fb62f487f40 1 -- 172.21.15.145:0/3539238935 >> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] conn(0x55fd44bcffa0 msgr2=0x55fd44bd0450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:05.215 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.215+0000 7fb62f487f40 1 --2- 172.21.15.145:0/3539238935 >> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] conn(0x55fd44bcffa0 0x55fd44bd0450 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:05.215 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.215+0000 7fb62f487f40 1 -- 172.21.15.145:0/3539238935 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fb608074cb0 msgr2=0x7fb608077170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:05.216 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.215+0000 7fb62f487f40 1 --2- 172.21.15.145:0/3539238935 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fb608074cb0 0x7fb608077170 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb61c005fd0 tx=0x7fb61c007040 comp rx=0 tx=0).stop 2024-05-09T04:35:05.216 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.215+0000 7fb62f487f40 1 -- 172.21.15.145:0/3539238935 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fd44bdd0d0 msgr2=0x55fd44bc56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:05.216 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.215+0000 7fb62f487f40 1 --2- 172.21.15.145:0/3539238935 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x55fd44bdd0d0 0x55fd44bc56c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fb624025e20 tx=0x7fb624002fb0 comp rx=0 tx=0).stop 2024-05-09T04:35:05.216 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.215+0000 7fb62d000640 1 -- 172.21.15.145:0/3539238935 reap_dead start 2024-05-09T04:35:05.216 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.215+0000 7fb62f487f40 1 -- 172.21.15.145:0/3539238935 shutdown_connections 2024-05-09T04:35:05.216 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.215+0000 7fb62f487f40 1 -- 172.21.15.145:0/3539238935 >> 172.21.15.145:0/3539238935 conn(0x55fd44b3ac80 msgr2=0x55fd44be2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:05.216 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.215+0000 7fb62f487f40 1 -- 172.21.15.145:0/3539238935 shutdown_connections 2024-05-09T04:35:05.216 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.215+0000 7fb62f487f40 1 -- 172.21.15.145:0/3539238935 wait complete. 2024-05-09T04:35:05.219 INFO:teuthology.orchestra.run.smithi145.stdout:@� 2024-05-09T04:35:05.219 DEBUG:teuthology.orchestra.run.smithi145:> ceph-dencoder type file_layout_t import - decode dump_json 2024-05-09T04:35:05.308 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-05-09T04:35:05.308 INFO:teuthology.orchestra.run.smithi145.stdout: "stripe_unit": 4194304, 2024-05-09T04:35:05.308 INFO:teuthology.orchestra.run.smithi145.stdout: "stripe_count": 1, 2024-05-09T04:35:05.308 INFO:teuthology.orchestra.run.smithi145.stdout: "object_size": 8388608, 2024-05-09T04:35:05.309 INFO:teuthology.orchestra.run.smithi145.stdout: "pool_id": 6, 2024-05-09T04:35:05.309 INFO:teuthology.orchestra.run.smithi145.stdout: "pool_ns": "" 2024-05-09T04:35:05.309 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-05-09T04:35:05.309 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:35:05.314 DEBUG:teuthology.orchestra.run.smithi145:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001fb.00000000 layout 2024-05-09T04:35:05.420 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.420+0000 7f943a7e9f40 1 Processor -- start 2024-05-09T04:35:05.420 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.420+0000 7f943a7e9f40 1 -- start start 2024-05-09T04:35:05.421 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.421+0000 7f943a7e9f40 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5573645525d0 0x557364551bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:05.421 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.421+0000 7f943a7e9f40 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5573645dd790 0x5573645d3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:05.421 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.421+0000 7f943a7e9f40 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5573645dc120 0x5573645d3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:05.422 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.421+0000 7f943a7e9f40 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5573644aff50 con 0x5573645dd790 2024-05-09T04:35:05.422 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.421+0000 7f943a7e9f40 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5573644b3f90 con 0x5573645dc120 2024-05-09T04:35:05.422 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.421+0000 7f943a7e9f40 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x557364539160 con 0x5573645525d0 2024-05-09T04:35:05.422 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.421+0000 7f9437000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5573645dd790 0x5573645d3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:05.422 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.421+0000 7f9437000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5573645dd790 0x5573645d3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:53202/0 (socket says 172.21.15.145:53202) 2024-05-09T04:35:05.422 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.421+0000 7f9437000640 1 -- 172.21.15.145:0/2651998672 learned_addr learned my addr 172.21.15.145:0/2651998672 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:05.422 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.422+0000 7f9438400640 1 --2- 172.21.15.145:0/2651998672 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5573645dc120 0x5573645d3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:05.422 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.422+0000 7f9437a00640 1 --2- 172.21.15.145:0/2651998672 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5573645525d0 0x557364551bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:05.422 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.422+0000 7f9437000640 1 -- 172.21.15.145:0/2651998672 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5573645525d0 msgr2=0x557364551bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:05.422 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.422+0000 7f9437000640 1 --2- 172.21.15.145:0/2651998672 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5573645525d0 0x557364551bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:05.422 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.422+0000 7f9437000640 1 -- 172.21.15.145:0/2651998672 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5573645dc120 msgr2=0x5573645d3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:05.422 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.422+0000 7f9437000640 1 --2- 172.21.15.145:0/2651998672 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5573645dc120 0x5573645d3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:05.422 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.422+0000 7f9437000640 1 -- 172.21.15.145:0/2651998672 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557364506990 con 0x5573645dd790 2024-05-09T04:35:05.424 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.422+0000 7f9437000640 1 --2- 172.21.15.145:0/2651998672 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5573645dd790 0x5573645d3a50 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f942400a460 tx=0x7f942400a920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c35db991ef107171 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:05.424 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.422+0000 7f9435c00640 1 -- 172.21.15.145:0/2651998672 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f942400b920 con 0x5573645dd790 2024-05-09T04:35:05.424 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.422+0000 7f9435c00640 1 -- 172.21.15.145:0/2651998672 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f942400f070 con 0x5573645dd790 2024-05-09T04:35:05.424 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.422+0000 7f9435c00640 1 -- 172.21.15.145:0/2651998672 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f942400bd20 con 0x5573645dd790 2024-05-09T04:35:05.424 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.423+0000 7f943a7e9f40 1 -- 172.21.15.145:0/2651998672 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5573645dd790 msgr2=0x5573645d3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:05.424 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.423+0000 7f943a7e9f40 1 --2- 172.21.15.145:0/2651998672 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5573645dd790 0x5573645d3a50 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f942400a460 tx=0x7f942400a920 comp rx=0 tx=0).stop 2024-05-09T04:35:05.424 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.423+0000 7f943a7e9f40 1 -- 172.21.15.145:0/2651998672 shutdown_connections 2024-05-09T04:35:05.424 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.423+0000 7f943a7e9f40 1 --2- 172.21.15.145:0/2651998672 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5573645dc120 0x5573645d3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:05.424 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.423+0000 7f943a7e9f40 1 --2- 172.21.15.145:0/2651998672 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5573645dd790 0x5573645d3a50 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:05.424 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.423+0000 7f943a7e9f40 1 --2- 172.21.15.145:0/2651998672 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5573645525d0 0x557364551bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:05.425 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.423+0000 7f943a7e9f40 1 -- 172.21.15.145:0/2651998672 >> 172.21.15.145:0/2651998672 conn(0x557364553df0 msgr2=0x5573645541d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:05.425 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.423+0000 7f943a7e9f40 1 -- 172.21.15.145:0/2651998672 shutdown_connections 2024-05-09T04:35:05.425 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.423+0000 7f943a7e9f40 1 -- 172.21.15.145:0/2651998672 wait complete. 2024-05-09T04:35:05.425 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.424+0000 7f943a7e9f40 1 Processor -- start 2024-05-09T04:35:05.425 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.424+0000 7f943a7e9f40 1 -- start start 2024-05-09T04:35:05.427 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.425+0000 7f943a7e9f40 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5573645525d0 0x557364509960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:05.427 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.425+0000 7f943a7e9f40 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5573645dc120 0x5573645cc920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:05.427 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.425+0000 7f943a7e9f40 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5573645dd790 0x55736454d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:05.427 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.425+0000 7f943a7e9f40 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55736450a060 con 0x5573645dd790 2024-05-09T04:35:05.427 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.425+0000 7f943a7e9f40 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x557364538e80 con 0x5573645525d0 2024-05-09T04:35:05.427 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.425+0000 7f943a7e9f40 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x55736450a340 con 0x5573645dc120 2024-05-09T04:35:05.427 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.425+0000 7f9438400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5573645dd790 0x55736454d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:05.427 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.425+0000 7f9438400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5573645dd790 0x55736454d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:53210/0 (socket says 172.21.15.145:53210) 2024-05-09T04:35:05.427 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.425+0000 7f9438400640 1 -- 172.21.15.145:0/2385151953 learned_addr learned my addr 172.21.15.145:0/2385151953 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:05.427 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.425+0000 7f9438400640 1 -- 172.21.15.145:0/2385151953 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5573645dc120 msgr2=0x5573645cc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:05.427 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.425+0000 7f9438400640 1 --2- 172.21.15.145:0/2385151953 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5573645dc120 0x5573645cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:05.427 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.426+0000 7f9438400640 1 -- 172.21.15.145:0/2385151953 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5573645525d0 msgr2=0x557364509960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:05.428 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.426+0000 7f9437a00640 1 --2- 172.21.15.145:0/2385151953 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5573645525d0 0x557364509960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:05.428 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.426+0000 7f9438400640 1 --2- 172.21.15.145:0/2385151953 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5573645525d0 0x557364509960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:05.428 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.426+0000 7f9438400640 1 -- 172.21.15.145:0/2385151953 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573645d04e0 con 0x5573645dd790 2024-05-09T04:35:05.428 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.426+0000 7f9438400640 1 --2- 172.21.15.145:0/2385151953 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5573645dd790 0x55736454d100 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9430005fd0 tx=0x7f9430003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:05.428 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.426+0000 7f942fe00640 1 -- 172.21.15.145:0/2385151953 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f943006a070 con 0x5573645dd790 2024-05-09T04:35:05.428 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.426+0000 7f942fe00640 1 -- 172.21.15.145:0/2385151953 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f943005a450 con 0x5573645dd790 2024-05-09T04:35:05.428 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.426+0000 7f943a7e9f40 1 -- 172.21.15.145:0/2385151953 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557364538260 con 0x5573645dd790 2024-05-09T04:35:05.428 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.426+0000 7f942fe00640 1 -- 172.21.15.145:0/2385151953 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9430055040 con 0x5573645dd790 2024-05-09T04:35:05.428 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.426+0000 7f943a7e9f40 1 -- 172.21.15.145:0/2385151953 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557364509160 con 0x5573645dd790 2024-05-09T04:35:05.428 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.428+0000 7f942fe00640 1 -- 172.21.15.145:0/2385151953 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f943006d070 con 0x5573645dd790 2024-05-09T04:35:05.429 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.429+0000 7f942fe00640 1 --2- 172.21.15.145:0/2385151953 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f9418074cb0 0x7f9418077170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:05.429 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.429+0000 7f942fe00640 1 -- 172.21.15.145:0/2385151953 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 6104+0+0 (secure 0 0 0) 0x7f94300da260 con 0x5573645dd790 2024-05-09T04:35:05.429 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.430+0000 7f943a7e9f40 1 --2- 172.21.15.145:0/2385151953 >> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] conn(0x55736454e3f0 0x557364676b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:05.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.430+0000 7f943a7e9f40 1 -- 172.21.15.145:0/2385151953 --> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] -- osd_op(unknown.0.0:1 5.13 5:ca8fdbba:::100000001fb.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) -- 0x5573645de870 con 0x55736454e3f0 2024-05-09T04:35:05.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.430+0000 7f9437a00640 1 --2- 172.21.15.145:0/2385151953 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f9418074cb0 0x7f9418077170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:05.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.430+0000 7f9437000640 1 --2- 172.21.15.145:0/2385151953 >> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] conn(0x55736454e3f0 0x557364676b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:05.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.431+0000 7f9437a00640 1 --2- 172.21.15.145:0/2385151953 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f9418074cb0 0x7f9418077170 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9428000c00 tx=0x7f9428001430 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:05.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.431+0000 7f9437000640 1 --2- 172.21.15.145:0/2385151953 >> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] conn(0x55736454e3f0 0x557364676b70 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:05.431 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.431+0000 7f9437000640 1 -- 172.21.15.145:0/2385151953 <== osd.3 v2:172.21.15.145:6808/337009826 1 ==== osd_op_reply(1 100000001fb.00000000 [getxattr (30) out=30b] v0'0 uv3 ondisk = 0) ==== 164+0+30 (crc 0 0 0) 0x7f942400c7b0 con 0x55736454e3f0 2024-05-09T04:35:05.431 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.431+0000 7f943a7e9f40 1 -- 172.21.15.145:0/2385151953 >> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] conn(0x55736454e3f0 msgr2=0x557364676b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:05.431 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.431+0000 7f943a7e9f40 1 --2- 172.21.15.145:0/2385151953 >> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] conn(0x55736454e3f0 0x557364676b70 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:05.432 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.431+0000 7f943a7e9f40 1 -- 172.21.15.145:0/2385151953 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f9418074cb0 msgr2=0x7f9418077170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:05.432 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.431+0000 7f943a7e9f40 1 --2- 172.21.15.145:0/2385151953 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f9418074cb0 0x7f9418077170 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9428000c00 tx=0x7f9428001430 comp rx=0 tx=0).stop 2024-05-09T04:35:05.432 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.431+0000 7f943a7e9f40 1 -- 172.21.15.145:0/2385151953 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5573645dd790 msgr2=0x55736454d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:05.432 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.431+0000 7f943a7e9f40 1 --2- 172.21.15.145:0/2385151953 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5573645dd790 0x55736454d100 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9430005fd0 tx=0x7f9430003330 comp rx=0 tx=0).stop 2024-05-09T04:35:05.432 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.431+0000 7f9438400640 1 -- 172.21.15.145:0/2385151953 reap_dead start 2024-05-09T04:35:05.432 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.431+0000 7f943a7e9f40 1 -- 172.21.15.145:0/2385151953 shutdown_connections 2024-05-09T04:35:05.432 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.431+0000 7f943a7e9f40 1 -- 172.21.15.145:0/2385151953 >> 172.21.15.145:0/2385151953 conn(0x557364553df0 msgr2=0x5573645541d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:05.432 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.432+0000 7f943a7e9f40 1 -- 172.21.15.145:0/2385151953 shutdown_connections 2024-05-09T04:35:05.432 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:05.432+0000 7f943a7e9f40 1 -- 172.21.15.145:0/2385151953 wait complete. 2024-05-09T04:35:05.436 INFO:teuthology.orchestra.run.smithi145.stdout:@@ 2024-05-09T04:35:05.436 DEBUG:teuthology.orchestra.run.smithi145:> ceph-dencoder type file_layout_t import - decode dump_json 2024-05-09T04:35:05.528 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-05-09T04:35:05.528 INFO:teuthology.orchestra.run.smithi145.stdout: "stripe_unit": 4194304, 2024-05-09T04:35:05.528 INFO:teuthology.orchestra.run.smithi145.stdout: "stripe_count": 1, 2024-05-09T04:35:05.528 INFO:teuthology.orchestra.run.smithi145.stdout: "object_size": 4194304, 2024-05-09T04:35:05.529 INFO:teuthology.orchestra.run.smithi145.stdout: "pool_id": 5, 2024-05-09T04:35:05.529 INFO:teuthology.orchestra.run.smithi145.stdout: "pool_ns": "" 2024-05-09T04:35:05.529 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-05-09T04:35:05.529 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:35:05.534 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-09T04:35:05.534 DEBUG:teuthology.orchestra.run.smithi145:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-09T04:35:05.645 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-09T04:35:05.753 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-09T04:35:05.754 INFO:tasks.cephfs.mount:Terminating background process 2024-05-09T04:35:05.754 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-05-09T04:35:05.754 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T04:35:05.812 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-05-09T04:35:05.812 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T04:35:05.866 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T04:35:06.066 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.066+0000 7f921e000640 1 Processor -- start 2024-05-09T04:35:06.067 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.067+0000 7f921e000640 1 -- start start 2024-05-09T04:35:06.068 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.068+0000 7f921e000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f92180d3cd0 0x7f92180d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:06.068 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.068+0000 7f921e000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f921815f4e0 0x7f92180d1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:06.068 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.068+0000 7f921e000640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92180cfda0 con 0x7f921815f100 2024-05-09T04:35:06.069 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.068+0000 7f921e000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f92180b8e70 con 0x7f92180d3cd0 2024-05-09T04:35:06.069 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.068+0000 7f921e000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f92180d0080 con 0x7f921815f4e0 2024-05-09T04:35:06.069 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.069+0000 7f920fe00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f92180d3cd0 0x7f92180d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:06.069 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.069+0000 7f920fe00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f92180d3cd0 0x7f92180d1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:53218/0 (socket says 172.21.15.145:53218) 2024-05-09T04:35:06.069 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.069+0000 7f920fe00640 1 -- 172.21.15.145:0/2241855898 learned_addr learned my addr 172.21.15.145:0/2241855898 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:06.069 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.069+0000 7f921d600640 1 --2- 172.21.15.145:0/2241855898 >> v2:172.21.15.192:3300/0 conn(0x7f921815f4e0 0x7f92180d1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:06.069 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.069+0000 7f920fe00640 1 -- 172.21.15.145:0/2241855898 >> v2:172.21.15.192:3300/0 conn(0x7f921815f4e0 msgr2=0x7f92180d1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:06.070 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.069+0000 7f920fe00640 1 --2- 172.21.15.145:0/2241855898 >> v2:172.21.15.192:3300/0 conn(0x7f921815f4e0 0x7f92180d1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:06.070 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.069+0000 7f920fe00640 1 -- 172.21.15.145:0/2241855898 >> v1:172.21.15.192:6789/0 conn(0x7f921815f100 legacy=0x7f92180d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:06.070 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.069+0000 7f920fe00640 1 -- 172.21.15.145:0/2241855898 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92180d0360 con 0x7f92180d3cd0 2024-05-09T04:35:06.070 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.070+0000 7f920fe00640 1 --2- 172.21.15.145:0/2241855898 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f92180d3cd0 0x7f92180d1950 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9200009870 tx=0x7f920002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13057bbf4de91905 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:06.070 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.070+0000 7f9217e00640 1 -- 172.21.15.145:0/2241855898 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9200037070 con 0x7f92180d3cd0 2024-05-09T04:35:06.070 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.070+0000 7f9217e00640 1 -- 172.21.15.145:0/2241855898 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9200032440 con 0x7f92180d3cd0 2024-05-09T04:35:06.070 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.070+0000 7f9217e00640 1 -- 172.21.15.145:0/2241855898 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9200032810 con 0x7f92180d3cd0 2024-05-09T04:35:06.071 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.071+0000 7f921e000640 1 -- 172.21.15.145:0/2241855898 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f92180d3cd0 msgr2=0x7f92180d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:06.071 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.071+0000 7f921e000640 1 --2- 172.21.15.145:0/2241855898 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f92180d3cd0 0x7f92180d1950 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9200009870 tx=0x7f920002a210 comp rx=0 tx=0).stop 2024-05-09T04:35:06.071 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.071+0000 7f921e000640 1 -- 172.21.15.145:0/2241855898 shutdown_connections 2024-05-09T04:35:06.071 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.071+0000 7f921e000640 1 --2- 172.21.15.145:0/2241855898 >> v2:172.21.15.192:3300/0 conn(0x7f921815f4e0 0x7f92180d1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:06.072 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.071+0000 7f921e000640 1 --2- 172.21.15.145:0/2241855898 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f92180d3cd0 0x7f92180d1950 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:06.072 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.071+0000 7f921e000640 1 -- 172.21.15.145:0/2241855898 >> 172.21.15.145:0/2241855898 conn(0x7f921809e2e0 msgr2=0x7f92180d5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:06.072 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.071+0000 7f921e000640 1 -- 172.21.15.145:0/2241855898 shutdown_connections 2024-05-09T04:35:06.072 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.071+0000 7f921e000640 1 -- 172.21.15.145:0/2241855898 wait complete. 2024-05-09T04:35:06.073 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.073+0000 7f921e000640 1 Processor -- start 2024-05-09T04:35:06.073 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.073+0000 7f921e000640 1 -- start start 2024-05-09T04:35:06.076 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.074+0000 7f921e000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92180d3cd0 0x7f92180cb390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:06.076 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.074+0000 7f921e000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f921815f100 0x7f9218158b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:06.076 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.074+0000 7f921e000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f921815f4e0 0x7f921815b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:06.076 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.074+0000 7f921e000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f92180d0e40 con 0x7f921815f4e0 2024-05-09T04:35:06.077 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.074+0000 7f921e000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f92180d0920 con 0x7f92180d3cd0 2024-05-09T04:35:06.077 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.074+0000 7f921e000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f92180cb8d0 con 0x7f921815f100 2024-05-09T04:35:06.077 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.075+0000 7f921d600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f921815f4e0 0x7f921815b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:06.077 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.075+0000 7f921d600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f921815f4e0 0x7f921815b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:53224/0 (socket says 172.21.15.145:53224) 2024-05-09T04:35:06.077 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.075+0000 7f921d600640 1 -- 172.21.15.145:0/1314077252 learned_addr learned my addr 172.21.15.145:0/1314077252 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:06.077 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.075+0000 7f921cc00640 1 --2- 172.21.15.145:0/1314077252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92180d3cd0 0x7f92180cb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:06.077 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.075+0000 7f920fe00640 1 --2- 172.21.15.145:0/1314077252 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f921815f100 0x7f9218158b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:06.077 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.075+0000 7f921d600640 1 -- 172.21.15.145:0/1314077252 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f921815f100 msgr2=0x7f9218158b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:06.077 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.075+0000 7f921d600640 1 --2- 172.21.15.145:0/1314077252 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f921815f100 0x7f9218158b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:06.077 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.075+0000 7f921d600640 1 -- 172.21.15.145:0/1314077252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92180d3cd0 msgr2=0x7f92180cb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:06.077 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.075+0000 7f921d600640 1 --2- 172.21.15.145:0/1314077252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92180d3cd0 0x7f92180cb390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:06.077 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.075+0000 7f921d600640 1 -- 172.21.15.145:0/1314077252 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f921815a1f0 con 0x7f921815f4e0 2024-05-09T04:35:06.077 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.075+0000 7f921cc00640 1 --2- 172.21.15.145:0/1314077252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92180d3cd0 0x7f92180cb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:35:06.077 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.075+0000 7f921d600640 1 --2- 172.21.15.145:0/1314077252 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f921815f4e0 0x7f921815b080 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9210009fd0 tx=0x7f921000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:06.077 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.076+0000 7f9215600640 1 -- 172.21.15.145:0/1314077252 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9210014420 con 0x7f921815f4e0 2024-05-09T04:35:06.077 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.076+0000 7f9215600640 1 -- 172.21.15.145:0/1314077252 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9210019070 con 0x7f921815f4e0 2024-05-09T04:35:06.078 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.076+0000 7f9215600640 1 -- 172.21.15.145:0/1314077252 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9210014800 con 0x7f921815f4e0 2024-05-09T04:35:06.078 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.076+0000 7f921e000640 1 -- 172.21.15.145:0/1314077252 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92181596d0 con 0x7f921815f4e0 2024-05-09T04:35:06.078 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.076+0000 7f921e000640 1 -- 172.21.15.145:0/1314077252 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9218159f10 con 0x7f921815f4e0 2024-05-09T04:35:06.078 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.077+0000 7f921e000640 1 -- 172.21.15.145:0/1314077252 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f921808f090 con 0x7f921815f4e0 2024-05-09T04:35:06.078 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.078+0000 7f9215600640 1 -- 172.21.15.145:0/1314077252 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f92100149a0 con 0x7f921815f4e0 2024-05-09T04:35:06.078 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.079+0000 7f9215600640 1 --2- 172.21.15.145:0/1314077252 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f91f40762d0 0x7f91f4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:06.079 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.079+0000 7f9215600640 1 -- 172.21.15.145:0/1314077252 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 6104+0+0 (secure 0 0 0) 0x7f9210097fa0 con 0x7f921815f4e0 2024-05-09T04:35:06.079 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.079+0000 7f921cc00640 1 --2- 172.21.15.145:0/1314077252 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f91f40762d0 0x7f91f4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:06.079 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.080+0000 7f921cc00640 1 --2- 172.21.15.145:0/1314077252 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f91f40762d0 0x7f91f4078790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9208003eb0 tx=0x7f9208009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:06.083 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.084+0000 7f9215600640 1 -- 172.21.15.145:0/1314077252 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f92181596d0 con 0x7f921815f4e0 2024-05-09T04:35:06.306 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.306+0000 7f921e000640 1 -- 172.21.15.145:0/1314077252 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9218076760 con 0x7f921815f4e0 2024-05-09T04:35:06.306 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.307+0000 7f9215600640 1 -- 172.21.15.145:0/1314077252 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v35) ==== 80+0+0 (secure 0 0 0) 0x7f9210061770 con 0x7f921815f4e0 2024-05-09T04:35:06.307 INFO:teuthology.orchestra.run.smithi145.stderr:listed 0 entries 2024-05-09T04:35:06.310 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.309+0000 7f921e000640 1 -- 172.21.15.145:0/1314077252 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f91f40762d0 msgr2=0x7f91f4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:06.310 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.309+0000 7f921e000640 1 --2- 172.21.15.145:0/1314077252 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f91f40762d0 0x7f91f4078790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9208003eb0 tx=0x7f9208009040 comp rx=0 tx=0).stop 2024-05-09T04:35:06.310 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.309+0000 7f921e000640 1 -- 172.21.15.145:0/1314077252 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f921815f4e0 msgr2=0x7f921815b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:06.310 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.309+0000 7f921e000640 1 --2- 172.21.15.145:0/1314077252 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f921815f4e0 0x7f921815b080 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9210009fd0 tx=0x7f921000e4b0 comp rx=0 tx=0).stop 2024-05-09T04:35:06.310 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.309+0000 7f921e000640 1 -- 172.21.15.145:0/1314077252 shutdown_connections 2024-05-09T04:35:06.310 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.310+0000 7f921e000640 1 --2- 172.21.15.145:0/1314077252 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f91f40762d0 0x7f91f4078790 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:06.310 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.310+0000 7f921e000640 1 --2- 172.21.15.145:0/1314077252 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f921815f4e0 0x7f921815b080 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:06.310 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.310+0000 7f921e000640 1 --2- 172.21.15.145:0/1314077252 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f921815f100 0x7f9218158b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:06.311 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.310+0000 7f921e000640 1 --2- 172.21.15.145:0/1314077252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f92180d3cd0 0x7f92180cb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:06.311 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.310+0000 7f921e000640 1 -- 172.21.15.145:0/1314077252 >> 172.21.15.145:0/1314077252 conn(0x7f921809e2e0 msgr2=0x7f92180ba1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:06.311 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.310+0000 7f921e000640 1 -- 172.21.15.145:0/1314077252 shutdown_connections 2024-05-09T04:35:06.311 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:06.310+0000 7f921e000640 1 -- 172.21.15.145:0/1314077252 wait complete. 2024-05-09T04:35:06.329 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi145.front.sepia.ceph.com... 2024-05-09T04:35:06.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:35:06.329 DEBUG:teuthology.orchestra.run.smithi145:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-09T04:35:06.409 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi145.stderr:ceph-fuse[46320]: fuse finished with error 0 and tester_r 0 2024-05-09T04:35:06.415 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi145.front.sepia.ceph.com 2024-05-09T04:35:06.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:35:06.415 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-09T04:35:06.624 INFO:tasks.cephfs.mount:Terminating background process 2024-05-09T04:35:06.624 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-09T04:35:06.624 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T04:35:06.677 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-09T04:35:06.678 INFO:tasks.cephfs.mount:Terminating background process 2024-05-09T04:35:06.678 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-05-09T04:35:06.678 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T04:35:06.733 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-09T04:35:06.733 INFO:tasks.cephfs.mount:Terminating background process 2024-05-09T04:35:06.733 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-09T04:35:06.733 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T04:35:06.788 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-09T04:35:06.789 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T04:35:07.003 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.003+0000 7f0aca400640 1 Processor -- start 2024-05-09T04:35:07.003 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.003+0000 7f0aca400640 1 -- start start 2024-05-09T04:35:07.005 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.005+0000 7f0aca400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f0ac406e860 0x7f0ac406ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:07.005 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.005+0000 7f0aca400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f0ac40b4910 0x7f0ac40b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:07.005 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.005+0000 7f0aca400640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ac406f180 con 0x7f0ac415d780 2024-05-09T04:35:07.005 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.005+0000 7f0aca400640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f0ac4161360 con 0x7f0ac406e860 2024-05-09T04:35:07.005 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.005+0000 7f0aca400640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f0ac41615c0 con 0x7f0ac40b4910 2024-05-09T04:35:07.005 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.005+0000 7f0ac3400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f0ac406e860 0x7f0ac406ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:07.005 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.005+0000 7f0ac3400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f0ac406e860 0x7f0ac406ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:53232/0 (socket says 172.21.15.145:53232) 2024-05-09T04:35:07.005 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.005+0000 7f0ac3400640 1 -- 172.21.15.145:0/3830103207 learned_addr learned my addr 172.21.15.145:0/3830103207 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:07.006 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.005+0000 7f0ac3e00640 1 --2- 172.21.15.145:0/3830103207 >> v2:172.21.15.192:3300/0 conn(0x7f0ac40b4910 0x7f0ac40b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:07.006 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.005+0000 7f0ac3400640 1 -- 172.21.15.145:0/3830103207 >> v2:172.21.15.192:3300/0 conn(0x7f0ac40b4910 msgr2=0x7f0ac40b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:07.006 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.005+0000 7f0ac3400640 1 --2- 172.21.15.145:0/3830103207 >> v2:172.21.15.192:3300/0 conn(0x7f0ac40b4910 0x7f0ac40b4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:07.006 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.005+0000 7f0ac3400640 1 -- 172.21.15.145:0/3830103207 >> v1:172.21.15.192:6789/0 conn(0x7f0ac415d780 legacy=0x7f0ac4160c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:07.006 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.006+0000 7f0ac3400640 1 -- 172.21.15.145:0/3830103207 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ac41618a0 con 0x7f0ac406e860 2024-05-09T04:35:07.007 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.007+0000 7f0ac3400640 1 --2- 172.21.15.145:0/3830103207 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ac406e860 0x7f0ac406ec40 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0ac406f8c0 tx=0x7f0ab402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3d78749668acf79 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:07.007 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.007+0000 7f0ac2000640 1 -- 172.21.15.145:0/3830103207 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab4037070 con 0x7f0ac406e860 2024-05-09T04:35:07.007 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.007+0000 7f0ac2000640 1 -- 172.21.15.145:0/3830103207 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ab4032440 con 0x7f0ac406e860 2024-05-09T04:35:07.007 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.007+0000 7f0ac2000640 1 -- 172.21.15.145:0/3830103207 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab4032810 con 0x7f0ac406e860 2024-05-09T04:35:07.008 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.008+0000 7f0aca400640 1 -- 172.21.15.145:0/3830103207 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ac406e860 msgr2=0x7f0ac406ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:07.008 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.008+0000 7f0aca400640 1 --2- 172.21.15.145:0/3830103207 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ac406e860 0x7f0ac406ec40 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0ac406f8c0 tx=0x7f0ab402a1e0 comp rx=0 tx=0).stop 2024-05-09T04:35:07.008 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.008+0000 7f0aca400640 1 -- 172.21.15.145:0/3830103207 shutdown_connections 2024-05-09T04:35:07.008 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.008+0000 7f0aca400640 1 --2- 172.21.15.145:0/3830103207 >> v2:172.21.15.192:3300/0 conn(0x7f0ac40b4910 0x7f0ac40b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:07.008 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.008+0000 7f0aca400640 1 --2- 172.21.15.145:0/3830103207 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ac406e860 0x7f0ac406ec40 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:07.008 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.008+0000 7f0aca400640 1 -- 172.21.15.145:0/3830103207 >> 172.21.15.145:0/3830103207 conn(0x7f0ac406c870 msgr2=0x7f0ac406cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:07.008 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.008+0000 7f0aca400640 1 -- 172.21.15.145:0/3830103207 shutdown_connections 2024-05-09T04:35:07.008 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.008+0000 7f0aca400640 1 -- 172.21.15.145:0/3830103207 wait complete. 2024-05-09T04:35:07.009 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.010+0000 7f0aca400640 1 Processor -- start 2024-05-09T04:35:07.009 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.010+0000 7f0aca400640 1 -- start start 2024-05-09T04:35:07.011 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.011+0000 7f0aca400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0ac406e860 0x7f0ac414bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:07.011 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.011+0000 7f0aca400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ac40b4910 0x7f0ac40b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:07.011 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.011+0000 7f0aca400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ac415d780 0x7f0ac413c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:07.011 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.011+0000 7f0aca400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac4146c00 con 0x7f0ac40b4910 2024-05-09T04:35:07.011 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.011+0000 7f0aca400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac40b43f0 con 0x7f0ac415d780 2024-05-09T04:35:07.011 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.011+0000 7f0aca400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0ac4146d80 con 0x7f0ac406e860 2024-05-09T04:35:07.011 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.011+0000 7f0ac2a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ac40b4910 0x7f0ac40b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:07.011 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.011+0000 7f0ac2a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ac40b4910 0x7f0ac40b12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:53242/0 (socket says 172.21.15.145:53242) 2024-05-09T04:35:07.011 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.011+0000 7f0ac2a00640 1 -- 172.21.15.145:0/3147804649 learned_addr learned my addr 172.21.15.145:0/3147804649 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:07.011 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.011+0000 7f0ac2a00640 1 -- 172.21.15.145:0/3147804649 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0ac406e860 msgr2=0x7f0ac414bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:07.012 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.011+0000 7f0ac2a00640 1 --2- 172.21.15.145:0/3147804649 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0ac406e860 0x7f0ac414bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:07.012 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.011+0000 7f0ac2a00640 1 -- 172.21.15.145:0/3147804649 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ac415d780 msgr2=0x7f0ac413c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:07.012 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.011+0000 7f0ac2a00640 1 --2- 172.21.15.145:0/3147804649 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ac415d780 0x7f0ac413c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:07.012 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.011+0000 7f0ac2a00640 1 -- 172.21.15.145:0/3147804649 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ac414c090 con 0x7f0ac40b4910 2024-05-09T04:35:07.012 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.012+0000 7f0ac2a00640 1 --2- 172.21.15.145:0/3147804649 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ac40b4910 0x7f0ac40b12d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0ab000b3e0 tx=0x7f0ab000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:07.013 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.012+0000 7f0abb400640 1 -- 172.21.15.145:0/3147804649 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab0015020 con 0x7f0ac40b4910 2024-05-09T04:35:07.013 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.012+0000 7f0abb400640 1 -- 172.21.15.145:0/3147804649 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ab0011850 con 0x7f0ac40b4910 2024-05-09T04:35:07.013 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.012+0000 7f0abb400640 1 -- 172.21.15.145:0/3147804649 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab0011c40 con 0x7f0ac40b4910 2024-05-09T04:35:07.013 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.012+0000 7f0aca400640 1 -- 172.21.15.145:0/3147804649 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ac41e80b0 con 0x7f0ac40b4910 2024-05-09T04:35:07.013 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.012+0000 7f0aca400640 1 -- 172.21.15.145:0/3147804649 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ac41e72f0 con 0x7f0ac40b4910 2024-05-09T04:35:07.013 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.013+0000 7f0aca400640 1 -- 172.21.15.145:0/3147804649 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ac41e7cf0 con 0x7f0ac40b4910 2024-05-09T04:35:07.014 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.014+0000 7f0abb400640 1 -- 172.21.15.145:0/3147804649 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0ab0018020 con 0x7f0ac40b4910 2024-05-09T04:35:07.015 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.015+0000 7f0abb400640 1 --2- 172.21.15.145:0/3147804649 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f0aa40762d0 0x7f0aa4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:07.015 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.016+0000 7f0abb400640 1 -- 172.21.15.145:0/3147804649 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 6104+0+0 (secure 0 0 0) 0x7f0ab00976f0 con 0x7f0ac40b4910 2024-05-09T04:35:07.015 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.016+0000 7f0ac3400640 1 --2- 172.21.15.145:0/3147804649 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f0aa40762d0 0x7f0aa4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:07.016 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.017+0000 7f0ac3400640 1 --2- 172.21.15.145:0/3147804649 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f0aa40762d0 0x7f0aa4078790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4034000 tx=0x7f0ab402c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:07.020 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.021+0000 7f0abb400640 1 -- 172.21.15.145:0/3147804649 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0ab001f020 con 0x7f0ac40b4910 2024-05-09T04:35:07.264 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.264+0000 7f0aca400640 1 -- 172.21.15.145:0/3147804649 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0ac4141f10 con 0x7f0ac40b4910 2024-05-09T04:35:07.264 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.265+0000 7f0abb400640 1 -- 172.21.15.145:0/3147804649 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5189 (secure 0 0 0) 0x7f0ac414c090 con 0x7f0ac40b4910 2024-05-09T04:35:07.265 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:35:07.265 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4499,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6834/3227728059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3227728059},{"type":"v1","addr":"172.21.15.192:6834","nonce":3227728059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4502,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1603013787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6833","nonce":1603013787},{"type":"v1","addr":"172.21.15.192:6835","nonce":1603013787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4672,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/68778418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":68778418},{"type":"v1","addr":"172.21.15.145:6837","nonce":68778418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T04:34:43.133970+0000","modified":"2024-05-09T04:35:00.107166+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4761},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4761":{"gid":4761,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.145:6835/4260798843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4260798843},{"type":"v1","addr":"172.21.15.145:6835","nonce":4260798843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5,6],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4761,"qdb_cluster":[4761]},"id":2}]} 2024-05-09T04:35:07.265 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 18 2024-05-09T04:35:07.268 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.268+0000 7f0aca400640 1 -- 172.21.15.145:0/3147804649 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f0aa40762d0 msgr2=0x7f0aa4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:07.268 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.268+0000 7f0aca400640 1 --2- 172.21.15.145:0/3147804649 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f0aa40762d0 0x7f0aa4078790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4034000 tx=0x7f0ab402c000 comp rx=0 tx=0).stop 2024-05-09T04:35:07.268 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.268+0000 7f0aca400640 1 -- 172.21.15.145:0/3147804649 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ac40b4910 msgr2=0x7f0ac40b12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:07.268 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.268+0000 7f0aca400640 1 --2- 172.21.15.145:0/3147804649 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ac40b4910 0x7f0ac40b12d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0ab000b3e0 tx=0x7f0ab000b8a0 comp rx=0 tx=0).stop 2024-05-09T04:35:07.268 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.268+0000 7f0aca400640 1 -- 172.21.15.145:0/3147804649 shutdown_connections 2024-05-09T04:35:07.268 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.268+0000 7f0aca400640 1 --2- 172.21.15.145:0/3147804649 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f0aa40762d0 0x7f0aa4078790 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:07.268 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.268+0000 7f0aca400640 1 --2- 172.21.15.145:0/3147804649 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ac415d780 0x7f0ac413c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:07.268 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.268+0000 7f0aca400640 1 --2- 172.21.15.145:0/3147804649 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0ac40b4910 0x7f0ac40b12d0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:07.268 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.268+0000 7f0aca400640 1 --2- 172.21.15.145:0/3147804649 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0ac406e860 0x7f0ac414bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:07.268 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.268+0000 7f0aca400640 1 -- 172.21.15.145:0/3147804649 >> 172.21.15.145:0/3147804649 conn(0x7f0ac406c870 msgr2=0x7f0ac415db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:07.269 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.268+0000 7f0aca400640 1 -- 172.21.15.145:0/3147804649 shutdown_connections 2024-05-09T04:35:07.269 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.268+0000 7f0aca400640 1 -- 172.21.15.145:0/3147804649 wait complete. 2024-05-09T04:35:07.288 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T04:35:07.490 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.490+0000 7f2835800640 1 Processor -- start 2024-05-09T04:35:07.490 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.490+0000 7f2835800640 1 -- start start 2024-05-09T04:35:07.492 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.492+0000 7f2835800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f283006ef70 0x7f28300b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:07.492 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.492+0000 7f2835800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f28300b5cb0 0x7f28300b44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:07.492 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.492+0000 7f2835800640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28300b4c10 con 0x7f283014d210 2024-05-09T04:35:07.492 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.492+0000 7f2835800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f28300b49f0 con 0x7f28300b5cb0 2024-05-09T04:35:07.492 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.492+0000 7f2835800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f28300b5820 con 0x7f283006ef70 2024-05-09T04:35:07.493 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.493+0000 7f282f400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f28300b5cb0 0x7f28300b44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:07.493 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.493+0000 7f2826000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f283006ef70 0x7f28300b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:07.493 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.493+0000 7f282f400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f28300b5cb0 0x7f28300b44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.145:51988/0 (socket says 172.21.15.145:51988) 2024-05-09T04:35:07.493 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.493+0000 7f282f400640 1 -- 172.21.15.145:0/2304758167 learned_addr learned my addr 172.21.15.145:0/2304758167 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:07.493 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.494+0000 7f282f400640 1 -- 172.21.15.145:0/2304758167 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f283006ef70 msgr2=0x7f28300b3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:07.493 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.494+0000 7f282f400640 1 --2- 172.21.15.145:0/2304758167 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f283006ef70 0x7f28300b3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:07.494 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.494+0000 7f282f400640 1 -- 172.21.15.145:0/2304758167 >> v1:172.21.15.192:6789/0 conn(0x7f283014d210 legacy=0x7f283006e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:07.494 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.494+0000 7f282f400640 1 -- 172.21.15.145:0/2304758167 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28300cf730 con 0x7f28300b5cb0 2024-05-09T04:35:07.494 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.494+0000 7f2826000640 1 --2- 172.21.15.145:0/2304758167 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f283006ef70 0x7f28300b3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:35:07.494 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.495+0000 7f282f400640 1 --2- 172.21.15.145:0/2304758167 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28300b5cb0 0x7f28300b44b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2828009e50 tx=0x7f282802a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c20e32ab5c0b7df8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:07.495 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.495+0000 7f282e000640 1 -- 172.21.15.145:0/2304758167 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2828036070 con 0x7f28300b5cb0 2024-05-09T04:35:07.495 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.496+0000 7f282e000640 1 -- 172.21.15.145:0/2304758167 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2828031600 con 0x7f28300b5cb0 2024-05-09T04:35:07.495 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.496+0000 7f282e000640 1 -- 172.21.15.145:0/2304758167 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28280319d0 con 0x7f28300b5cb0 2024-05-09T04:35:07.496 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.496+0000 7f2835800640 1 -- 172.21.15.145:0/2304758167 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28300b5cb0 msgr2=0x7f28300b44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:07.496 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.496+0000 7f2835800640 1 --2- 172.21.15.145:0/2304758167 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28300b5cb0 0x7f28300b44b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2828009e50 tx=0x7f282802a710 comp rx=0 tx=0).stop 2024-05-09T04:35:07.496 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.496+0000 7f2835800640 1 -- 172.21.15.145:0/2304758167 shutdown_connections 2024-05-09T04:35:07.496 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.496+0000 7f2835800640 1 --2- 172.21.15.145:0/2304758167 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28300b5cb0 0x7f28300b44b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:07.496 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.496+0000 7f2835800640 1 --2- 172.21.15.145:0/2304758167 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f283006ef70 0x7f28300b3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:07.496 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.496+0000 7f2835800640 1 -- 172.21.15.145:0/2304758167 >> 172.21.15.145:0/2304758167 conn(0x7f283006c870 msgr2=0x7f283006cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:07.496 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.496+0000 7f2835800640 1 -- 172.21.15.145:0/2304758167 shutdown_connections 2024-05-09T04:35:07.496 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.496+0000 7f2835800640 1 -- 172.21.15.145:0/2304758167 wait complete. 2024-05-09T04:35:07.498 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.498+0000 7f2835800640 1 Processor -- start 2024-05-09T04:35:07.498 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.498+0000 7f2835800640 1 -- start start 2024-05-09T04:35:07.499 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.499+0000 7f2835800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f283006ef70 0x7f28300cd450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:07.499 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.499+0000 7f2835800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28300b5cb0 0x7f28300ab7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:07.499 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.499+0000 7f2835800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f283014d210 0x7f2830149c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:07.499 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.499+0000 7f2835800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f28300b54b0 con 0x7f283014d210 2024-05-09T04:35:07.499 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.499+0000 7f2835800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f28300b51d0 con 0x7f28300b5cb0 2024-05-09T04:35:07.499 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.499+0000 7f2835800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f28300ce720 con 0x7f283006ef70 2024-05-09T04:35:07.500 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.500+0000 7f282f400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f283014d210 0x7f2830149c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:07.500 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.500+0000 7f282f400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f283014d210 0x7f2830149c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:53254/0 (socket says 172.21.15.145:53254) 2024-05-09T04:35:07.500 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.500+0000 7f282f400640 1 -- 172.21.15.145:0/3917090198 learned_addr learned my addr 172.21.15.145:0/3917090198 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:07.501 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.500+0000 7f282ea00640 1 --2- 172.21.15.145:0/3917090198 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f283006ef70 0x7f28300cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:07.501 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.500+0000 7f2826000640 1 --2- 172.21.15.145:0/3917090198 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28300b5cb0 0x7f28300ab7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:07.501 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.500+0000 7f282f400640 1 -- 172.21.15.145:0/3917090198 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f283006ef70 msgr2=0x7f28300cd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:07.501 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.500+0000 7f282f400640 1 --2- 172.21.15.145:0/3917090198 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f283006ef70 0x7f28300cd450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:07.501 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.500+0000 7f282f400640 1 -- 172.21.15.145:0/3917090198 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28300b5cb0 msgr2=0x7f28300ab7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:07.501 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.500+0000 7f282f400640 1 --2- 172.21.15.145:0/3917090198 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28300b5cb0 0x7f28300ab7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:07.501 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.500+0000 7f282f400640 1 -- 172.21.15.145:0/3917090198 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28300aa470 con 0x7f283014d210 2024-05-09T04:35:07.501 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.501+0000 7f282f400640 1 --2- 172.21.15.145:0/3917090198 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f283014d210 0x7f2830149c70 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2828009f70 tx=0x7f2828004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:07.501 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.501+0000 7f2827400640 1 -- 172.21.15.145:0/3917090198 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2828041020 con 0x7f283014d210 2024-05-09T04:35:07.502 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.501+0000 7f2827400640 1 -- 172.21.15.145:0/3917090198 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2828008730 con 0x7f283014d210 2024-05-09T04:35:07.502 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.501+0000 7f2827400640 1 -- 172.21.15.145:0/3917090198 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2828036040 con 0x7f283014d210 2024-05-09T04:35:07.502 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.501+0000 7f2835800640 1 -- 172.21.15.145:0/3917090198 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28300aa750 con 0x7f283014d210 2024-05-09T04:35:07.502 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.501+0000 7f2835800640 1 -- 172.21.15.145:0/3917090198 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28300aad10 con 0x7f283014d210 2024-05-09T04:35:07.502 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.502+0000 7f2835800640 1 -- 172.21.15.145:0/3917090198 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28300ab4f0 con 0x7f283014d210 2024-05-09T04:35:07.503 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.503+0000 7f2827400640 1 -- 172.21.15.145:0/3917090198 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2828044020 con 0x7f283014d210 2024-05-09T04:35:07.504 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.504+0000 7f2827400640 1 --2- 172.21.15.145:0/3917090198 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f280c0762d0 0x7f280c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:07.504 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.504+0000 7f2827400640 1 -- 172.21.15.145:0/3917090198 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 6104+0+0 (secure 0 0 0) 0x7f28280b8320 con 0x7f283014d210 2024-05-09T04:35:07.504 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.505+0000 7f282ea00640 1 --2- 172.21.15.145:0/3917090198 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f280c0762d0 0x7f280c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:07.505 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.505+0000 7f282ea00640 1 --2- 172.21.15.145:0/3917090198 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f280c0762d0 0x7f280c078790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f28200015e0 tx=0x7f2820009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:07.508 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.509+0000 7f2827400640 1 -- 172.21.15.145:0/3917090198 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f28300ab4f0 con 0x7f283014d210 2024-05-09T04:35:07.761 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.761+0000 7f2835800640 1 -- 172.21.15.145:0/3917090198 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f283006a310 con 0x7f283014d210 2024-05-09T04:35:07.762 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.762+0000 7f2827400640 1 -- 172.21.15.145:0/3917090198 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5189 (secure 0 0 0) 0x7f282804b030 con 0x7f283014d210 2024-05-09T04:35:07.762 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:35:07.762 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4499,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6834/3227728059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3227728059},{"type":"v1","addr":"172.21.15.192:6834","nonce":3227728059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4502,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1603013787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6833","nonce":1603013787},{"type":"v1","addr":"172.21.15.192:6835","nonce":1603013787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4672,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/68778418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":68778418},{"type":"v1","addr":"172.21.15.145:6837","nonce":68778418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T04:34:43.133970+0000","modified":"2024-05-09T04:35:00.107166+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4761},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4761":{"gid":4761,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.145:6835/4260798843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4260798843},{"type":"v1","addr":"172.21.15.145:6835","nonce":4260798843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5,6],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4761,"qdb_cluster":[4761]},"id":2}]} 2024-05-09T04:35:07.763 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 18 2024-05-09T04:35:07.765 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.765+0000 7f2835800640 1 -- 172.21.15.145:0/3917090198 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f280c0762d0 msgr2=0x7f280c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:07.765 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.765+0000 7f2835800640 1 --2- 172.21.15.145:0/3917090198 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f280c0762d0 0x7f280c078790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f28200015e0 tx=0x7f2820009040 comp rx=0 tx=0).stop 2024-05-09T04:35:07.765 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.765+0000 7f2835800640 1 -- 172.21.15.145:0/3917090198 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f283014d210 msgr2=0x7f2830149c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:07.765 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.765+0000 7f2835800640 1 --2- 172.21.15.145:0/3917090198 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f283014d210 0x7f2830149c70 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2828009f70 tx=0x7f2828004a40 comp rx=0 tx=0).stop 2024-05-09T04:35:07.765 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.765+0000 7f2835800640 1 -- 172.21.15.145:0/3917090198 shutdown_connections 2024-05-09T04:35:07.765 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.765+0000 7f2835800640 1 --2- 172.21.15.145:0/3917090198 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f280c0762d0 0x7f280c078790 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:07.765 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.765+0000 7f2835800640 1 --2- 172.21.15.145:0/3917090198 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f283014d210 0x7f2830149c70 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:07.765 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.765+0000 7f2835800640 1 --2- 172.21.15.145:0/3917090198 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f28300b5cb0 0x7f28300ab7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:07.765 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.765+0000 7f2835800640 1 --2- 172.21.15.145:0/3917090198 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f283006ef70 0x7f28300cd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:07.766 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.765+0000 7f2835800640 1 -- 172.21.15.145:0/3917090198 >> 172.21.15.145:0/3917090198 conn(0x7f283006c870 msgr2=0x7f28300ceb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:07.766 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.765+0000 7f2835800640 1 -- 172.21.15.145:0/3917090198 shutdown_connections 2024-05-09T04:35:07.766 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.766+0000 7f2835800640 1 -- 172.21.15.145:0/3917090198 wait complete. 2024-05-09T04:35:07.783 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T04:35:07.974 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.973+0000 7efd87200640 1 Processor -- start 2024-05-09T04:35:07.974 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.974+0000 7efd87200640 1 -- start start 2024-05-09T04:35:07.975 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.975+0000 7efd87200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd880d3180 0x7efd880d5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:07.975 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.975+0000 7efd87200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7efd880d22f0 0x7efd880d3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:07.975 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.975+0000 7efd87200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7efd880b4760 0x7efd880b4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:07.975 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.975+0000 7efd87200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7efd880d4150 con 0x7efd880d22f0 2024-05-09T04:35:07.975 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.975+0000 7efd87200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7efd880b6200 con 0x7efd880b4760 2024-05-09T04:35:07.975 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.975+0000 7efd87200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7efd880d1e60 con 0x7efd880d3180 2024-05-09T04:35:07.975 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.975+0000 7efd85400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7efd880d22f0 0x7efd880d3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:07.976 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.975+0000 7efd85400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7efd880d22f0 0x7efd880d3560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:53260/0 (socket says 172.21.15.145:53260) 2024-05-09T04:35:07.976 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.975+0000 7efd85400640 1 -- 172.21.15.145:0/1153947842 learned_addr learned my addr 172.21.15.145:0/1153947842 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:07.976 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.976+0000 7efd85400640 1 -- 172.21.15.145:0/1153947842 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd880d3180 msgr2=0x7efd880d5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:07.976 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.976+0000 7efd85400640 1 --2- 172.21.15.145:0/1153947842 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd880d3180 0x7efd880d5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:07.977 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.976+0000 7efd85400640 1 -- 172.21.15.145:0/1153947842 >> v2:172.21.15.192:3300/0 conn(0x7efd880b4760 msgr2=0x7efd880b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:07.977 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.976+0000 7efd86800640 1 --2- 172.21.15.145:0/1153947842 >> v2:172.21.15.192:3300/0 conn(0x7efd880b4760 0x7efd880b4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:07.977 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.976+0000 7efd85400640 1 --2- 172.21.15.145:0/1153947842 >> v2:172.21.15.192:3300/0 conn(0x7efd880b4760 0x7efd880b4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:07.977 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.976+0000 7efd85400640 1 -- 172.21.15.145:0/1153947842 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd880d3e70 con 0x7efd880d22f0 2024-05-09T04:35:07.977 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.977+0000 7efd85400640 1 --2- 172.21.15.145:0/1153947842 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efd880d22f0 0x7efd880d3560 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7efd880cf7b0 tx=0x7efd7402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef789897eef5abb6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:07.978 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.977+0000 7efd84a00640 1 -- 172.21.15.145:0/1153947842 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd74037070 con 0x7efd880d22f0 2024-05-09T04:35:07.978 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.977+0000 7efd84a00640 1 -- 172.21.15.145:0/1153947842 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd74032440 con 0x7efd880d22f0 2024-05-09T04:35:07.978 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.977+0000 7efd84a00640 1 -- 172.21.15.145:0/1153947842 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd74032810 con 0x7efd880d22f0 2024-05-09T04:35:07.978 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.978+0000 7efd87200640 1 -- 172.21.15.145:0/1153947842 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efd880d22f0 msgr2=0x7efd880d3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:07.978 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.978+0000 7efd87200640 1 --2- 172.21.15.145:0/1153947842 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efd880d22f0 0x7efd880d3560 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7efd880cf7b0 tx=0x7efd7402a140 comp rx=0 tx=0).stop 2024-05-09T04:35:07.978 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.978+0000 7efd87200640 1 -- 172.21.15.145:0/1153947842 shutdown_connections 2024-05-09T04:35:07.978 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.978+0000 7efd87200640 1 --2- 172.21.15.145:0/1153947842 >> v2:172.21.15.192:3300/0 conn(0x7efd880b4760 0x7efd880b4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:07.978 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.978+0000 7efd87200640 1 --2- 172.21.15.145:0/1153947842 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efd880d22f0 0x7efd880d3560 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:07.978 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.978+0000 7efd87200640 1 --2- 172.21.15.145:0/1153947842 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd880d3180 0x7efd880d5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:07.978 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.978+0000 7efd87200640 1 -- 172.21.15.145:0/1153947842 >> 172.21.15.145:0/1153947842 conn(0x7efd880d4fb0 msgr2=0x7efd880d53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:07.978 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.978+0000 7efd87200640 1 -- 172.21.15.145:0/1153947842 shutdown_connections 2024-05-09T04:35:07.979 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.978+0000 7efd87200640 1 -- 172.21.15.145:0/1153947842 wait complete. 2024-05-09T04:35:07.979 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.980+0000 7efd87200640 1 Processor -- start 2024-05-09T04:35:07.979 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.980+0000 7efd87200640 1 -- start start 2024-05-09T04:35:07.981 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.981+0000 7efd87200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efd880b4760 0x7efd8814a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:07.981 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.981+0000 7efd87200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd880d22f0 0x7efd8815ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:07.981 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.981+0000 7efd87200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd880d3180 0x7efd880cb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:07.981 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.981+0000 7efd87200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7efd880d3aa0 con 0x7efd880b4760 2024-05-09T04:35:07.981 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.981+0000 7efd87200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7efd880d2750 con 0x7efd880d22f0 2024-05-09T04:35:07.981 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.981+0000 7efd87200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7efd880d3c20 con 0x7efd880d3180 2024-05-09T04:35:07.981 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.981+0000 7efd85e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efd880b4760 0x7efd8814a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:07.981 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.981+0000 7efd85e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efd880b4760 0x7efd8814a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:53274/0 (socket says 172.21.15.145:53274) 2024-05-09T04:35:07.981 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.981+0000 7efd85e00640 1 -- 172.21.15.145:0/2994923808 learned_addr learned my addr 172.21.15.145:0/2994923808 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:07.981 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.981+0000 7efd85400640 1 --2- 172.21.15.145:0/2994923808 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd880d22f0 0x7efd8815ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:07.982 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.981+0000 7efd86800640 1 --2- 172.21.15.145:0/2994923808 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd880d3180 0x7efd880cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:07.982 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.981+0000 7efd85e00640 1 -- 172.21.15.145:0/2994923808 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd880d3180 msgr2=0x7efd880cb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:07.982 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.981+0000 7efd85e00640 1 --2- 172.21.15.145:0/2994923808 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd880d3180 0x7efd880cb890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:07.982 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.981+0000 7efd85e00640 1 -- 172.21.15.145:0/2994923808 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd880d22f0 msgr2=0x7efd8815ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:07.982 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.981+0000 7efd85e00640 1 --2- 172.21.15.145:0/2994923808 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd880d22f0 0x7efd8815ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:07.982 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.981+0000 7efd85e00640 1 -- 172.21.15.145:0/2994923808 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd8814acf0 con 0x7efd880b4760 2024-05-09T04:35:07.983 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.982+0000 7efd85e00640 1 --2- 172.21.15.145:0/2994923808 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efd880b4760 0x7efd8814a710 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7efd7800b4a0 tx=0x7efd7800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:07.983 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.982+0000 7efd7e000640 1 -- 172.21.15.145:0/2994923808 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd78014070 con 0x7efd880b4760 2024-05-09T04:35:07.983 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.982+0000 7efd7e000640 1 -- 172.21.15.145:0/2994923808 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd78010610 con 0x7efd880b4760 2024-05-09T04:35:07.983 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.982+0000 7efd7e000640 1 -- 172.21.15.145:0/2994923808 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd78010a00 con 0x7efd880b4760 2024-05-09T04:35:07.983 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.982+0000 7efd87200640 1 -- 172.21.15.145:0/2994923808 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd8815a590 con 0x7efd880b4760 2024-05-09T04:35:07.983 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.982+0000 7efd87200640 1 -- 172.21.15.145:0/2994923808 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd8815ab50 con 0x7efd880b4760 2024-05-09T04:35:07.983 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.984+0000 7efd87200640 1 -- 172.21.15.145:0/2994923808 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd8806a310 con 0x7efd880b4760 2024-05-09T04:35:07.984 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.984+0000 7efd7e000640 1 -- 172.21.15.145:0/2994923808 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7efd78017070 con 0x7efd880b4760 2024-05-09T04:35:07.985 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.985+0000 7efd7e000640 1 --2- 172.21.15.145:0/2994923808 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7efd680762d0 0x7efd68078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:07.985 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.986+0000 7efd7e000640 1 -- 172.21.15.145:0/2994923808 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 6104+0+0 (secure 0 0 0) 0x7efd78097120 con 0x7efd880b4760 2024-05-09T04:35:07.985 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.986+0000 7efd85400640 1 --2- 172.21.15.145:0/2994923808 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7efd680762d0 0x7efd68078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:07.986 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.987+0000 7efd85400640 1 --2- 172.21.15.145:0/2994923808 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7efd680762d0 0x7efd68078790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7efd74034000 tx=0x7efd7402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:07.990 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:07.990+0000 7efd7e000640 1 -- 172.21.15.145:0/2994923808 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7efd78060980 con 0x7efd880b4760 2024-05-09T04:35:08.197 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.198+0000 7efd87200640 1 -- 172.21.15.145:0/2994923808 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efd880cbe30 con 0x7efd880b4760 2024-05-09T04:35:08.199 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.200+0000 7efd7e000640 1 -- 172.21.15.145:0/2994923808 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+18529 (secure 0 0 0) 0x7efd78060280 con 0x7efd880b4760 2024-05-09T04:35:08.200 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:35:08.200 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":35,"fsid":"957ee6c1-073b-44cf-9b93-9491b3fdfc1d","created":"2024-05-09T04:33:26.544367+0000","modified":"2024-05-09T04:35:01.635404+0000","last_up_change":"2024-05-09T04:33:39.039318+0000","last_in_change":"2024-05-09T04:33:29.902361+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T04:33:39.795344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-09T04:34:40.995268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-09T04:34:42.107773+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"data_new","create_time":"2024-05-09T04:34:59.021572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22e550ad-847b-4952-80e6-63d859ed1a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6801","nonce":3182572912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6803","nonce":3182572912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6807","nonce":3182572912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6805","nonce":3182572912}]},"public_addr":"172.21.15.145:6801/3182572912","cluster_addr":"172.21.15.145:6803/3182572912","heartbeat_back_addr":"172.21.15.145:6807/3182572912","heartbeat_front_addr":"172.21.15.145:6805/3182572912","state":["exists","up"]},{"osd":1,"uuid":"0f6b9fca-0c41-4145-8d74-598ddb42f2a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6825","nonce":1845225555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6827","nonce":1845225555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6831","nonce":1845225555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6829","nonce":1845225555}]},"public_addr":"172.21.15.145:6825/1845225555","cluster_addr":"172.21.15.145:6827/1845225555","heartbeat_back_addr":"172.21.15.145:6831/1845225555","heartbeat_front_addr":"172.21.15.145:6829/1845225555","state":["exists","up"]},{"osd":2,"uuid":"b3b1f67e-5790-483f-8610-e05a5bce4c30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6817","nonce":2168958993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6819","nonce":2168958993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6823","nonce":2168958993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6821","nonce":2168958993}]},"public_addr":"172.21.15.145:6817/2168958993","cluster_addr":"172.21.15.145:6819/2168958993","heartbeat_back_addr":"172.21.15.145:6823/2168958993","heartbeat_front_addr":"172.21.15.145:6821/2168958993","state":["exists","up"]},{"osd":3,"uuid":"884787a5-e037-4d31-9aae-743030eba4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6809","nonce":337009826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6811","nonce":337009826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6815","nonce":337009826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6813","nonce":337009826}]},"public_addr":"172.21.15.145:6809/337009826","cluster_addr":"172.21.15.145:6811/337009826","heartbeat_back_addr":"172.21.15.145:6815/337009826","heartbeat_front_addr":"172.21.15.145:6813/337009826","state":["exists","up"]},{"osd":4,"uuid":"0759fc1e-c5fc-4a2f-86bb-e8eccd83f833","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6818","nonce":3117700995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6821","nonce":3117700995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6827","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6829","nonce":3117700995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6823","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6825","nonce":3117700995}]},"public_addr":"172.21.15.192:6818/3117700995","cluster_addr":"172.21.15.192:6821/3117700995","heartbeat_back_addr":"172.21.15.192:6829/3117700995","heartbeat_front_addr":"172.21.15.192:6825/3117700995","state":["exists","up"]},{"osd":5,"uuid":"9443cc22-9379-46e3-9173-98b592bf2c16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6801","nonce":1328373268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6803","nonce":1328373268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6807","nonce":1328373268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6805","nonce":1328373268}]},"public_addr":"172.21.15.192:6801/1328373268","cluster_addr":"172.21.15.192:6803/1328373268","heartbeat_back_addr":"172.21.15.192:6807/1328373268","heartbeat_front_addr":"172.21.15.192:6805/1328373268","state":["exists","up"]},{"osd":6,"uuid":"d05a3311-505a-45b5-85b9-aa3113e96987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6817","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6820","nonce":1868011057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6824","nonce":1868011057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6831","nonce":1868011057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6828","nonce":1868011057}]},"public_addr":"172.21.15.192:6820/1868011057","cluster_addr":"172.21.15.192:6824/1868011057","heartbeat_back_addr":"172.21.15.192:6831/1868011057","heartbeat_front_addr":"172.21.15.192:6828/1868011057","state":["exists","up"]},{"osd":7,"uuid":"c7c23dbc-2029-43ec-9f53-25f3d58019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6809","nonce":3368512157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6811","nonce":3368512157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6815","nonce":3368512157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6813","nonce":3368512157}]},"public_addr":"172.21.15.192:6809/3368512157","cluster_addr":"172.21.15.192:6811/3368512157","heartbeat_back_addr":"172.21.15.192:6815/3368512157","heartbeat_front_addr":"172.21.15.192:6813/3368512157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.091188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.032567+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:34.797468+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.228839+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.622400+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.473103+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.566092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.349395+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T04:35:08.202 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.202+0000 7efd87200640 1 -- 172.21.15.145:0/2994923808 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7efd680762d0 msgr2=0x7efd68078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:08.202 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.202+0000 7efd87200640 1 --2- 172.21.15.145:0/2994923808 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7efd680762d0 0x7efd68078790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7efd74034000 tx=0x7efd7402e000 comp rx=0 tx=0).stop 2024-05-09T04:35:08.203 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.202+0000 7efd87200640 1 -- 172.21.15.145:0/2994923808 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efd880b4760 msgr2=0x7efd8814a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:08.203 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.202+0000 7efd87200640 1 --2- 172.21.15.145:0/2994923808 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efd880b4760 0x7efd8814a710 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7efd7800b4a0 tx=0x7efd7800b960 comp rx=0 tx=0).stop 2024-05-09T04:35:08.203 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.202+0000 7efd87200640 1 -- 172.21.15.145:0/2994923808 shutdown_connections 2024-05-09T04:35:08.203 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.202+0000 7efd87200640 1 --2- 172.21.15.145:0/2994923808 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7efd680762d0 0x7efd68078790 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:08.203 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.202+0000 7efd87200640 1 --2- 172.21.15.145:0/2994923808 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd880d3180 0x7efd880cb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:08.203 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.202+0000 7efd87200640 1 --2- 172.21.15.145:0/2994923808 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd880d22f0 0x7efd8815ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:08.203 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.203+0000 7efd87200640 1 --2- 172.21.15.145:0/2994923808 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efd880b4760 0x7efd8814a710 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:08.203 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.203+0000 7efd87200640 1 -- 172.21.15.145:0/2994923808 >> 172.21.15.145:0/2994923808 conn(0x7efd880d4fb0 msgr2=0x7efd880b2e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:08.203 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.203+0000 7efd87200640 1 -- 172.21.15.145:0/2994923808 shutdown_connections 2024-05-09T04:35:08.203 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.203+0000 7efd87200640 1 -- 172.21.15.145:0/2994923808 wait complete. 2024-05-09T04:35:08.221 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-09T04:35:08.221 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T04:35:08.421 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.421+0000 7f2b42000640 1 Processor -- start 2024-05-09T04:35:08.421 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.421+0000 7f2b42000640 1 -- start start 2024-05-09T04:35:08.423 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.423+0000 7f2b42000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f2b3c15f4a0 0x7f2b3c0bae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:08.423 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.423+0000 7f2b42000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2b3c0d4b40 0x7f2b3c0d32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:08.423 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.423+0000 7f2b42000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b3c0d3800 con 0x7f2b3c0b72a0 2024-05-09T04:35:08.423 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.423+0000 7f2b42000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f2b3c0d4fa0 con 0x7f2b3c15f4a0 2024-05-09T04:35:08.423 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.423+0000 7f2b42000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f2b3c0d3aa0 con 0x7f2b3c0d4b40 2024-05-09T04:35:08.423 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.423+0000 7f2b3be00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f2b3c0b72a0 0x7f2b3c0d51d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:45244/0 (socket says 172.21.15.145:45244) 2024-05-09T04:35:08.423 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.423+0000 7f2b3b400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f2b3c15f4a0 0x7f2b3c0bae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:08.423 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.423+0000 7f2b3be00640 1 -- 172.21.15.145:0/2144836018 learned_addr learned my addr 172.21.15.145:0/2144836018 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:35:08.424 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.424+0000 7f2b33e00640 1 --2- 172.21.15.145:0/2144836018 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2b3c0d4b40 0x7f2b3c0d32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:08.424 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.424+0000 7f2b3aa00640 1 -- 172.21.15.145:0/2144836018 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3270390082 0 0) 0x7f2b3c0d3800 con 0x7f2b3c0b72a0 2024-05-09T04:35:08.424 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.424+0000 7f2b3aa00640 1 -- 172.21.15.145:0/2144836018 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2b20003520 con 0x7f2b3c0b72a0 2024-05-09T04:35:08.424 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.424+0000 7f2b3b400640 1 -- 172.21.15.145:0/2144836018 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2b3c0d4b40 msgr2=0x7f2b3c0d32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:08.424 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.424+0000 7f2b3b400640 1 --2- 172.21.15.145:0/2144836018 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2b3c0d4b40 0x7f2b3c0d32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:08.424 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.424+0000 7f2b3b400640 1 -- 172.21.15.145:0/2144836018 >> v1:172.21.15.145:6789/0 conn(0x7f2b3c0b72a0 legacy=0x7f2b3c0d51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:08.425 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.424+0000 7f2b3b400640 1 -- 172.21.15.145:0/2144836018 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b3c0d3d80 con 0x7f2b3c15f4a0 2024-05-09T04:35:08.425 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.425+0000 7f2b3b400640 1 --2- 172.21.15.145:0/2144836018 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2b3c15f4a0 0x7f2b3c0bae00 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c0d39d0 tx=0x7f2b2c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acad2934862684fd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:08.425 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.425+0000 7f2b3aa00640 1 -- 172.21.15.145:0/2144836018 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c037070 con 0x7f2b3c15f4a0 2024-05-09T04:35:08.425 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.425+0000 7f2b3aa00640 1 -- 172.21.15.145:0/2144836018 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b2c032440 con 0x7f2b3c15f4a0 2024-05-09T04:35:08.425 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.425+0000 7f2b3aa00640 1 -- 172.21.15.145:0/2144836018 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c032810 con 0x7f2b3c15f4a0 2024-05-09T04:35:08.426 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.426+0000 7f2b42000640 1 -- 172.21.15.145:0/2144836018 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2b3c15f4a0 msgr2=0x7f2b3c0bae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:08.427 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.426+0000 7f2b42000640 1 --2- 172.21.15.145:0/2144836018 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2b3c15f4a0 0x7f2b3c0bae00 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c0d39d0 tx=0x7f2b2c02a180 comp rx=0 tx=0).stop 2024-05-09T04:35:08.427 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.426+0000 7f2b42000640 1 -- 172.21.15.145:0/2144836018 shutdown_connections 2024-05-09T04:35:08.427 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.426+0000 7f2b42000640 1 --2- 172.21.15.145:0/2144836018 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2b3c0d4b40 0x7f2b3c0d32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:08.427 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.426+0000 7f2b42000640 1 --2- 172.21.15.145:0/2144836018 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2b3c15f4a0 0x7f2b3c0bae00 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:08.427 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.426+0000 7f2b42000640 1 -- 172.21.15.145:0/2144836018 >> 172.21.15.145:0/2144836018 conn(0x7f2b3c0685e0 msgr2=0x7f2b3c0689f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:08.427 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.426+0000 7f2b42000640 1 -- 172.21.15.145:0/2144836018 shutdown_connections 2024-05-09T04:35:08.427 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.426+0000 7f2b42000640 1 -- 172.21.15.145:0/2144836018 wait complete. 2024-05-09T04:35:08.427 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.428+0000 7f2b42000640 1 Processor -- start 2024-05-09T04:35:08.428 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.428+0000 7f2b42000640 1 -- start start 2024-05-09T04:35:08.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.429+0000 7f2b42000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b3c0b72a0 0x7f2b3c1f8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:08.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.429+0000 7f2b42000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2b3c0d4b40 0x7f2b3c1f5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:08.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.429+0000 7f2b42000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2b3c15f4a0 0x7f2b3c15d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:08.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.429+0000 7f2b42000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3c0d4620 con 0x7f2b3c15f4a0 2024-05-09T04:35:08.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.429+0000 7f2b42000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3c0d4340 con 0x7f2b3c0b72a0 2024-05-09T04:35:08.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.429+0000 7f2b42000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f2b3c0b76f0 con 0x7f2b3c0d4b40 2024-05-09T04:35:08.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.429+0000 7f2b3be00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2b3c15f4a0 0x7f2b3c15d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:08.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.429+0000 7f2b3be00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2b3c15f4a0 0x7f2b3c15d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:53294/0 (socket says 172.21.15.145:53294) 2024-05-09T04:35:08.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.429+0000 7f2b3be00640 1 -- 172.21.15.145:0/4134317151 learned_addr learned my addr 172.21.15.145:0/4134317151 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:08.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.430+0000 7f2b3be00640 1 -- 172.21.15.145:0/4134317151 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2b3c0d4b40 msgr2=0x7f2b3c1f5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:08.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.430+0000 7f2b3be00640 1 --2- 172.21.15.145:0/4134317151 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2b3c0d4b40 0x7f2b3c1f5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:08.431 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.430+0000 7f2b3be00640 1 -- 172.21.15.145:0/4134317151 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b3c0b72a0 msgr2=0x7f2b3c1f8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:08.431 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.430+0000 7f2b3be00640 1 --2- 172.21.15.145:0/4134317151 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b3c0b72a0 0x7f2b3c1f8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:08.431 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.430+0000 7f2b3be00640 1 -- 172.21.15.145:0/4134317151 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b3c1f8f90 con 0x7f2b3c15f4a0 2024-05-09T04:35:08.431 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.430+0000 7f2b3be00640 1 --2- 172.21.15.145:0/4134317151 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2b3c15f4a0 0x7f2b3c15d190 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2b3400cb40 tx=0x7f2b3400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:08.431 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.430+0000 7f2b33400640 1 -- 172.21.15.145:0/4134317151 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3400aca0 con 0x7f2b3c15f4a0 2024-05-09T04:35:08.431 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.430+0000 7f2b33400640 1 -- 172.21.15.145:0/4134317151 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b3400ae40 con 0x7f2b3c15f4a0 2024-05-09T04:35:08.431 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.430+0000 7f2b42000640 1 -- 172.21.15.145:0/4134317151 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b3c150ce0 con 0x7f2b3c15f4a0 2024-05-09T04:35:08.431 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.430+0000 7f2b33400640 1 -- 172.21.15.145:0/4134317151 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b34012600 con 0x7f2b3c15f4a0 2024-05-09T04:35:08.431 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.430+0000 7f2b42000640 1 -- 172.21.15.145:0/4134317151 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b3c1512a0 con 0x7f2b3c15f4a0 2024-05-09T04:35:08.431 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.432+0000 7f2b42000640 1 -- 172.21.15.145:0/4134317151 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b3c1519c0 con 0x7f2b3c15f4a0 2024-05-09T04:35:08.432 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.433+0000 7f2b33400640 1 -- 172.21.15.145:0/4134317151 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2b340127a0 con 0x7f2b3c15f4a0 2024-05-09T04:35:08.433 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.433+0000 7f2b33400640 1 --2- 172.21.15.145:0/4134317151 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f2b180762d0 0x7f2b18078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:08.433 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.434+0000 7f2b33400640 1 -- 172.21.15.145:0/4134317151 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 6104+0+0 (secure 0 0 0) 0x7f2b34097e90 con 0x7f2b3c15f4a0 2024-05-09T04:35:08.433 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.434+0000 7f2b3b400640 1 --2- 172.21.15.145:0/4134317151 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f2b180762d0 0x7f2b18078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:08.434 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.435+0000 7f2b3b400640 1 --2- 172.21.15.145:0/4134317151 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f2b180762d0 0x7f2b18078790 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c034000 tx=0x7f2b2c02c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:08.438 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.438+0000 7f2b33400640 1 -- 172.21.15.145:0/4134317151 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2b3c1f8f90 con 0x7f2b3c15f4a0 2024-05-09T04:35:08.684 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.684+0000 7f2b42000640 1 -- 172.21.15.145:0/4134317151 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2b3c06a310 con 0x7f2b3c15f4a0 2024-05-09T04:35:08.686 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.686+0000 7f2b33400640 1 -- 172.21.15.145:0/4134317151 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5189 (secure 0 0 0) 0x7f2b340615c0 con 0x7f2b3c15f4a0 2024-05-09T04:35:08.686 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:35:08.686 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4499,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6834/3227728059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3227728059},{"type":"v1","addr":"172.21.15.192:6834","nonce":3227728059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4502,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1603013787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6833","nonce":1603013787},{"type":"v1","addr":"172.21.15.192:6835","nonce":1603013787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4672,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/68778418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":68778418},{"type":"v1","addr":"172.21.15.145:6837","nonce":68778418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T04:34:43.133970+0000","modified":"2024-05-09T04:35:00.107166+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4761},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4761":{"gid":4761,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.145:6835/4260798843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4260798843},{"type":"v1","addr":"172.21.15.145:6835","nonce":4260798843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5,6],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4761,"qdb_cluster":[4761]},"id":2}]} 2024-05-09T04:35:08.687 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 18 2024-05-09T04:35:08.689 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.689+0000 7f2b42000640 1 -- 172.21.15.145:0/4134317151 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f2b180762d0 msgr2=0x7f2b18078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:08.689 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.689+0000 7f2b42000640 1 --2- 172.21.15.145:0/4134317151 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f2b180762d0 0x7f2b18078790 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c034000 tx=0x7f2b2c02c000 comp rx=0 tx=0).stop 2024-05-09T04:35:08.689 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.689+0000 7f2b42000640 1 -- 172.21.15.145:0/4134317151 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2b3c15f4a0 msgr2=0x7f2b3c15d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:08.689 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.689+0000 7f2b42000640 1 --2- 172.21.15.145:0/4134317151 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2b3c15f4a0 0x7f2b3c15d190 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2b3400cb40 tx=0x7f2b3400a060 comp rx=0 tx=0).stop 2024-05-09T04:35:08.689 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.689+0000 7f2b42000640 1 -- 172.21.15.145:0/4134317151 shutdown_connections 2024-05-09T04:35:08.689 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.689+0000 7f2b42000640 1 --2- 172.21.15.145:0/4134317151 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f2b180762d0 0x7f2b18078790 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:08.689 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.689+0000 7f2b42000640 1 --2- 172.21.15.145:0/4134317151 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2b3c15f4a0 0x7f2b3c15d190 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:08.689 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.689+0000 7f2b42000640 1 --2- 172.21.15.145:0/4134317151 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2b3c0d4b40 0x7f2b3c1f5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:08.689 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.689+0000 7f2b42000640 1 --2- 172.21.15.145:0/4134317151 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2b3c0b72a0 0x7f2b3c1f8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:08.690 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.689+0000 7f2b42000640 1 -- 172.21.15.145:0/4134317151 >> 172.21.15.145:0/4134317151 conn(0x7f2b3c0685e0 msgr2=0x7f2b3c0b9540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:08.690 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.689+0000 7f2b42000640 1 -- 172.21.15.145:0/4134317151 shutdown_connections 2024-05-09T04:35:08.690 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.689+0000 7f2b42000640 1 -- 172.21.15.145:0/4134317151 wait complete. 2024-05-09T04:35:08.707 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T04:35:08.903 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.903+0000 7f2298400640 1 Processor -- start 2024-05-09T04:35:08.903 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.903+0000 7f2298400640 1 -- start start 2024-05-09T04:35:08.905 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.905+0000 7f2298400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f229015fbc0 0x7f2290142200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:08.905 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.905+0000 7f2298400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22900726b0 0x7f2290072b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:08.905 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.905+0000 7f2298400640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f229013f340 con 0x7f229015ffa0 2024-05-09T04:35:08.905 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.905+0000 7f2298400640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f2290072220 con 0x7f229015fbc0 2024-05-09T04:35:08.905 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.905+0000 7f2298400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f22900cffe0 con 0x7f22900726b0 2024-05-09T04:35:08.905 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.905+0000 7f2295000640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f229015ffa0 0x7f22900cf8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:45256/0 (socket says 172.21.15.145:45256) 2024-05-09T04:35:08.905 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.905+0000 7f2295000640 1 -- 172.21.15.145:0/3309534629 learned_addr learned my addr 172.21.15.145:0/3309534629 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:35:08.907 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.906+0000 7f2295a00640 1 --2- 172.21.15.145:0/3309534629 >> v2:172.21.15.192:3300/0 conn(0x7f229015fbc0 0x7f2290142200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:08.907 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.906+0000 7f2296400640 1 --2- 172.21.15.145:0/3309534629 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22900726b0 0x7f2290072b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:08.907 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.906+0000 7f228fe00640 1 -- 172.21.15.145:0/3309534629 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1394125063 0 0) 0x7f227c001110 con 0x7f229015ffa0 2024-05-09T04:35:08.907 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.906+0000 7f228fe00640 1 -- 172.21.15.145:0/3309534629 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2278003520 con 0x7f229015ffa0 2024-05-09T04:35:08.907 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.906+0000 7f228fe00640 1 -- 172.21.15.145:0/3309534629 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3090063673 0 0) 0x7f2278003520 con 0x7f229015ffa0 2024-05-09T04:35:08.907 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.906+0000 7f228fe00640 1 -- 172.21.15.145:0/3309534629 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22900726b0 msgr2=0x7f2290072b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:08.907 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.906+0000 7f228fe00640 1 --2- 172.21.15.145:0/3309534629 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22900726b0 0x7f2290072b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:08.907 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.906+0000 7f228fe00640 1 -- 172.21.15.145:0/3309534629 >> v2:172.21.15.192:3300/0 conn(0x7f229015fbc0 msgr2=0x7f2290142200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:08.907 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.906+0000 7f228fe00640 1 --2- 172.21.15.145:0/3309534629 >> v2:172.21.15.192:3300/0 conn(0x7f229015fbc0 0x7f2290142200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:08.907 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.906+0000 7f228fe00640 1 -- 172.21.15.145:0/3309534629 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22900d02b0 con 0x7f229015ffa0 2024-05-09T04:35:08.908 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.906+0000 7f228fe00640 1 -- 172.21.15.145:0/3309534629 <== mon.0 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3474413261 0 0) 0x7f227c0018c0 con 0x7f229015ffa0 2024-05-09T04:35:08.908 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.906+0000 7f228fe00640 1 -- 172.21.15.145:0/3309534629 >> v1:172.21.15.145:6789/0 conn(0x7f229015ffa0 legacy=0x7f22900cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:08.908 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.906+0000 7f228fe00640 1 --2- 172.21.15.145:0/3309534629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2278003a00 0x7f2278003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:08.908 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.906+0000 7f228fe00640 1 --2- 172.21.15.145:0/3309534629 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2278004800 0x7f2278004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:08.908 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.906+0000 7f228fe00640 1 --2- 172.21.15.145:0/3309534629 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22900726b0 0x7f2278005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:08.908 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.906+0000 7f228fe00640 1 -- 172.21.15.145:0/3309534629 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2290072220 con 0x7f2278004800 2024-05-09T04:35:08.908 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.906+0000 7f228fe00640 1 -- 172.21.15.145:0/3309534629 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f22900cffe0 con 0x7f2278003a00 2024-05-09T04:35:08.908 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.906+0000 7f228fe00640 1 -- 172.21.15.145:0/3309534629 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f227800ddb0 con 0x7f22900726b0 2024-05-09T04:35:08.908 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.906+0000 7f2295a00640 1 --2- 172.21.15.145:0/3309534629 >> v2:172.21.15.192:3300/0 conn(0x7f229015fbc0 0x7f2290142200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:35:08.908 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.907+0000 7f2295000640 1 --2- 172.21.15.145:0/3309534629 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2278004800 0x7f2278004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:08.908 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.907+0000 7f2295000640 1 -- 172.21.15.145:0/3309534629 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22900726b0 msgr2=0x7f2278005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:08.908 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.907+0000 7f2295000640 1 --2- 172.21.15.145:0/3309534629 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22900726b0 0x7f2278005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:08.908 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.907+0000 7f2295000640 1 -- 172.21.15.145:0/3309534629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2278003a00 msgr2=0x7f2278003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:08.908 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.907+0000 7f2295000640 1 --2- 172.21.15.145:0/3309534629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2278003a00 0x7f2278003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:08.909 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.907+0000 7f2295000640 1 -- 172.21.15.145:0/3309534629 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2278003520 con 0x7f2278004800 2024-05-09T04:35:08.909 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.908+0000 7f2295000640 1 --2- 172.21.15.145:0/3309534629 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2278004800 0x7f2278004c80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f227c00c3e0 tx=0x7f2278005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c7393467dc75d5e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:08.910 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.908+0000 7f228fe00640 1 -- 172.21.15.145:0/3309534629 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227c00c0e0 con 0x7f2278004800 2024-05-09T04:35:08.910 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.908+0000 7f228fe00640 1 -- 172.21.15.145:0/3309534629 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f227c034070 con 0x7f2278004800 2024-05-09T04:35:08.910 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.908+0000 7f228fe00640 1 -- 172.21.15.145:0/3309534629 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227c005d10 con 0x7f2278004800 2024-05-09T04:35:08.910 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.909+0000 7f2298400640 1 -- 172.21.15.145:0/3309534629 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2278004800 msgr2=0x7f2278004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:08.910 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.909+0000 7f2298400640 1 --2- 172.21.15.145:0/3309534629 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2278004800 0x7f2278004c80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f227c00c3e0 tx=0x7f2278005830 comp rx=0 tx=0).stop 2024-05-09T04:35:08.910 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.909+0000 7f2296400640 1 -- 172.21.15.145:0/3309534629 reap_dead start 2024-05-09T04:35:08.910 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.909+0000 7f2298400640 1 -- 172.21.15.145:0/3309534629 shutdown_connections 2024-05-09T04:35:08.910 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.909+0000 7f2298400640 1 -- 172.21.15.145:0/3309534629 >> 172.21.15.145:0/3309534629 conn(0x7f229006c870 msgr2=0x7f229006cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:08.910 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.909+0000 7f2298400640 1 -- 172.21.15.145:0/3309534629 shutdown_connections 2024-05-09T04:35:08.910 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.909+0000 7f2298400640 1 -- 172.21.15.145:0/3309534629 wait complete. 2024-05-09T04:35:08.912 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.910+0000 7f2298400640 1 Processor -- start 2024-05-09T04:35:08.913 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.911+0000 7f2298400640 1 -- start start 2024-05-09T04:35:08.913 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.911+0000 7f2298400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2278004800 0x7f229015c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:08.913 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.911+0000 7f2298400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22901405a0 0x7f22901396f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:08.913 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.912+0000 7f2298400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2290072a90 0x7f2290072f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:08.913 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.912+0000 7f2298400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f22900d0b50 con 0x7f2278004800 2024-05-09T04:35:08.913 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.912+0000 7f2298400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f22900d0870 con 0x7f22901405a0 2024-05-09T04:35:08.913 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.912+0000 7f2298400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f2290160430 con 0x7f2290072a90 2024-05-09T04:35:08.913 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.912+0000 7f2295a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2278004800 0x7f229015c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:08.913 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.912+0000 7f2295a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2278004800 0x7f229015c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:53304/0 (socket says 172.21.15.145:53304) 2024-05-09T04:35:08.913 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.912+0000 7f2295a00640 1 -- 172.21.15.145:0/2898042540 learned_addr learned my addr 172.21.15.145:0/2898042540 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:08.913 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.912+0000 7f2295a00640 1 -- 172.21.15.145:0/2898042540 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2290072a90 msgr2=0x7f2290072f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:08.913 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.912+0000 7f2295a00640 1 --2- 172.21.15.145:0/2898042540 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2290072a90 0x7f2290072f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:08.913 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.912+0000 7f2295a00640 1 -- 172.21.15.145:0/2898042540 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22901405a0 msgr2=0x7f22901396f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:08.914 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.912+0000 7f2295a00640 1 --2- 172.21.15.145:0/2898042540 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22901405a0 0x7f22901396f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:08.914 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.912+0000 7f2295a00640 1 -- 172.21.15.145:0/2898042540 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2290155d30 con 0x7f2278004800 2024-05-09T04:35:08.914 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.913+0000 7f2295a00640 1 --2- 172.21.15.145:0/2898042540 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2278004800 0x7f229015c460 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f228400e810 tx=0x7f228400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:08.914 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.913+0000 7f228d600640 1 -- 172.21.15.145:0/2898042540 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f228400f040 con 0x7f2278004800 2024-05-09T04:35:08.914 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.913+0000 7f228d600640 1 -- 172.21.15.145:0/2898042540 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2284004030 con 0x7f2278004800 2024-05-09T04:35:08.914 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.913+0000 7f228d600640 1 -- 172.21.15.145:0/2898042540 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2284004420 con 0x7f2278004800 2024-05-09T04:35:08.914 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.913+0000 7f2298400640 1 -- 172.21.15.145:0/2898042540 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2290156010 con 0x7f2278004800 2024-05-09T04:35:08.914 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.913+0000 7f2298400640 1 -- 172.21.15.145:0/2898042540 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22901532f0 con 0x7f2278004800 2024-05-09T04:35:08.914 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.914+0000 7f2298400640 1 -- 172.21.15.145:0/2898042540 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f229006a310 con 0x7f2278004800 2024-05-09T04:35:08.915 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.915+0000 7f228d600640 1 -- 172.21.15.145:0/2898042540 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f228400c670 con 0x7f2278004800 2024-05-09T04:35:08.916 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.916+0000 7f228d600640 1 --2- 172.21.15.145:0/2898042540 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f2270076240 0x7f2270078700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:08.916 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.916+0000 7f228d600640 1 -- 172.21.15.145:0/2898042540 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 6104+0+0 (secure 0 0 0) 0x7f2284097510 con 0x7f2278004800 2024-05-09T04:35:08.916 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.916+0000 7f2295000640 1 --2- 172.21.15.145:0/2898042540 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f2270076240 0x7f2270078700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:08.916 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.917+0000 7f2295000640 1 --2- 172.21.15.145:0/2898042540 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f2270076240 0x7f2270078700 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f227c033000 tx=0x7f227c009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:08.920 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:08.921+0000 7f228d600640 1 -- 172.21.15.145:0/2898042540 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2284017020 con 0x7f2278004800 2024-05-09T04:35:09.145 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.145+0000 7f2298400640 1 -- 172.21.15.145:0/2898042540 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f229015bea0 con 0x7f2278004800 2024-05-09T04:35:09.146 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.147+0000 7f228d600640 1 -- 172.21.15.145:0/2898042540 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5189 (secure 0 0 0) 0x7f2284060c40 con 0x7f2278004800 2024-05-09T04:35:09.147 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:35:09.147 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4499,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6834/3227728059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3227728059},{"type":"v1","addr":"172.21.15.192:6834","nonce":3227728059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4502,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1603013787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6833","nonce":1603013787},{"type":"v1","addr":"172.21.15.192:6835","nonce":1603013787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4672,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/68778418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":68778418},{"type":"v1","addr":"172.21.15.145:6837","nonce":68778418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T04:34:43.133970+0000","modified":"2024-05-09T04:35:00.107166+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4761},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4761":{"gid":4761,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.145:6835/4260798843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":4260798843},{"type":"v1","addr":"172.21.15.145:6835","nonce":4260798843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5,6],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4761,"qdb_cluster":[4761]},"id":2}]} 2024-05-09T04:35:09.147 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 18 2024-05-09T04:35:09.149 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.149+0000 7f2298400640 1 -- 172.21.15.145:0/2898042540 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f2270076240 msgr2=0x7f2270078700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:09.149 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.149+0000 7f2298400640 1 --2- 172.21.15.145:0/2898042540 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f2270076240 0x7f2270078700 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f227c033000 tx=0x7f227c009030 comp rx=0 tx=0).stop 2024-05-09T04:35:09.149 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.149+0000 7f2298400640 1 -- 172.21.15.145:0/2898042540 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2278004800 msgr2=0x7f229015c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:09.149 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.149+0000 7f2298400640 1 --2- 172.21.15.145:0/2898042540 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2278004800 0x7f229015c460 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f228400e810 tx=0x7f228400ecd0 comp rx=0 tx=0).stop 2024-05-09T04:35:09.149 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.149+0000 7f2298400640 1 -- 172.21.15.145:0/2898042540 shutdown_connections 2024-05-09T04:35:09.149 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.149+0000 7f2298400640 1 --2- 172.21.15.145:0/2898042540 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f2270076240 0x7f2270078700 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:09.149 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.149+0000 7f2298400640 1 --2- 172.21.15.145:0/2898042540 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2290072a90 0x7f2290072f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:09.149 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.149+0000 7f2298400640 1 --2- 172.21.15.145:0/2898042540 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22901405a0 0x7f22901396f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:09.149 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.149+0000 7f2298400640 1 --2- 172.21.15.145:0/2898042540 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2278004800 0x7f229015c460 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:09.149 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.149+0000 7f2298400640 1 -- 172.21.15.145:0/2898042540 >> 172.21.15.145:0/2898042540 conn(0x7f229006c870 msgr2=0x7f2290140b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:09.150 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.149+0000 7f2298400640 1 -- 172.21.15.145:0/2898042540 shutdown_connections 2024-05-09T04:35:09.150 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.149+0000 7f2298400640 1 -- 172.21.15.145:0/2898042540 wait complete. 2024-05-09T04:35:09.167 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T04:35:09.365 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.365+0000 7fe2bac00640 1 Processor -- start 2024-05-09T04:35:09.365 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.365+0000 7fe2bac00640 1 -- start start 2024-05-09T04:35:09.366 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.367+0000 7fe2bac00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe2b40d3180 0x7fe2b40d5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:09.367 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.367+0000 7fe2bac00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe2b40d22f0 0x7fe2b40d3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:09.367 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.367+0000 7fe2bac00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2b40d3e70 con 0x7fe2b40b4760 2024-05-09T04:35:09.367 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.367+0000 7fe2bac00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fe2b40d4150 con 0x7fe2b40d22f0 2024-05-09T04:35:09.367 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.367+0000 7fe2bac00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe2b40b6200 con 0x7fe2b40d3180 2024-05-09T04:35:09.367 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.367+0000 7fe2ba200640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7fe2b40b4760 0x7fe2b40b4bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:45270/0 (socket says 172.21.15.145:45270) 2024-05-09T04:35:09.367 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.367+0000 7fe2ba200640 1 -- 172.21.15.145:0/191129482 learned_addr learned my addr 172.21.15.145:0/191129482 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:35:09.368 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.367+0000 7fe2b9800640 1 --2- 172.21.15.145:0/191129482 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe2b40d3180 0x7fe2b40d5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:09.368 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.367+0000 7fe2b8e00640 1 --2- 172.21.15.145:0/191129482 >> v2:172.21.15.192:3300/0 conn(0x7fe2b40d22f0 0x7fe2b40d3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:09.368 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.367+0000 7fe2b3e00640 1 -- 172.21.15.145:0/191129482 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 630784565 0 0) 0x7fe2b40d3e70 con 0x7fe2b40b4760 2024-05-09T04:35:09.369 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.368+0000 7fe2b3e00640 1 -- 172.21.15.145:0/191129482 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe298003520 con 0x7fe2b40b4760 2024-05-09T04:35:09.369 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.368+0000 7fe2b3e00640 1 -- 172.21.15.145:0/191129482 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3522881005 0 0) 0x7fe298003520 con 0x7fe2b40b4760 2024-05-09T04:35:09.369 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.368+0000 7fe2b3e00640 1 -- 172.21.15.145:0/191129482 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe2b40d3180 msgr2=0x7fe2b40d5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:09.369 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.368+0000 7fe2b3e00640 1 --2- 172.21.15.145:0/191129482 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe2b40d3180 0x7fe2b40d5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:09.369 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.368+0000 7fe2b3e00640 1 -- 172.21.15.145:0/191129482 >> v2:172.21.15.192:3300/0 conn(0x7fe2b40d22f0 msgr2=0x7fe2b40d3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:09.369 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.368+0000 7fe2b3e00640 1 --2- 172.21.15.145:0/191129482 >> v2:172.21.15.192:3300/0 conn(0x7fe2b40d22f0 0x7fe2b40d3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:09.369 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.368+0000 7fe2b3e00640 1 -- 172.21.15.145:0/191129482 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2b40b6450 con 0x7fe2b40b4760 2024-05-09T04:35:09.369 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.368+0000 7fe2b3e00640 1 -- 172.21.15.145:0/191129482 <== mon.0 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3474413261 0 0) 0x7fe2ac001a70 con 0x7fe2b40b4760 2024-05-09T04:35:09.369 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.368+0000 7fe2b3e00640 1 -- 172.21.15.145:0/191129482 >> v1:172.21.15.145:6789/0 conn(0x7fe2b40b4760 legacy=0x7fe2b40b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:09.369 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.368+0000 7fe2b3e00640 1 --2- 172.21.15.145:0/191129482 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe298003a00 0x7fe298003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:09.369 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.368+0000 7fe2b3e00640 1 --2- 172.21.15.145:0/191129482 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe298004800 0x7fe298004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:09.369 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.368+0000 7fe2b3e00640 1 --2- 172.21.15.145:0/191129482 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe2b40d3180 0x7fe2980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:09.369 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.368+0000 7fe2b3e00640 1 -- 172.21.15.145:0/191129482 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b40b6200 con 0x7fe298003a00 2024-05-09T04:35:09.369 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.368+0000 7fe2b3e00640 1 -- 172.21.15.145:0/191129482 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe29800ba70 con 0x7fe298004800 2024-05-09T04:35:09.369 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.368+0000 7fe2b3e00640 1 -- 172.21.15.145:0/191129482 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe29800bc70 con 0x7fe2b40d3180 2024-05-09T04:35:09.369 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.368+0000 7fe2b9800640 1 --2- 172.21.15.145:0/191129482 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe298003a00 0x7fe298003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:09.370 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.369+0000 7fe2b9800640 1 -- 172.21.15.145:0/191129482 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe2b40d3180 msgr2=0x7fe2980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:09.370 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.369+0000 7fe2b9800640 1 --2- 172.21.15.145:0/191129482 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe2b40d3180 0x7fe2980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:09.370 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.369+0000 7fe2b9800640 1 -- 172.21.15.145:0/191129482 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe298004800 msgr2=0x7fe298004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:09.370 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.369+0000 7fe2b9800640 1 --2- 172.21.15.145:0/191129482 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe298004800 0x7fe298004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:09.370 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.369+0000 7fe2b9800640 1 -- 172.21.15.145:0/191129482 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe298003520 con 0x7fe298003a00 2024-05-09T04:35:09.371 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.370+0000 7fe2b9800640 1 --2- 172.21.15.145:0/191129482 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe298003a00 0x7fe298003e20 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe2a400ba30 tx=0x7fe298005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f33cf69001cec143 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:09.371 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.370+0000 7fe2b3e00640 1 -- 172.21.15.145:0/191129482 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a4030070 con 0x7fe298003a00 2024-05-09T04:35:09.371 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.370+0000 7fe2b3e00640 1 -- 172.21.15.145:0/191129482 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2a402ae10 con 0x7fe298003a00 2024-05-09T04:35:09.371 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.370+0000 7fe2b3e00640 1 -- 172.21.15.145:0/191129482 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a400b600 con 0x7fe298003a00 2024-05-09T04:35:09.371 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.370+0000 7fe2bac00640 1 -- 172.21.15.145:0/191129482 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe298003a00 msgr2=0x7fe298003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:09.371 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.371+0000 7fe2bac00640 1 --2- 172.21.15.145:0/191129482 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe298003a00 0x7fe298003e20 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe2a400ba30 tx=0x7fe298005830 comp rx=0 tx=0).stop 2024-05-09T04:35:09.371 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.371+0000 7fe2ba200640 1 -- 172.21.15.145:0/191129482 reap_dead start 2024-05-09T04:35:09.371 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.371+0000 7fe2bac00640 1 -- 172.21.15.145:0/191129482 shutdown_connections 2024-05-09T04:35:09.372 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.371+0000 7fe2bac00640 1 -- 172.21.15.145:0/191129482 >> 172.21.15.145:0/191129482 conn(0x7fe2b40d4fb0 msgr2=0x7fe2b40d53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:09.372 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.371+0000 7fe2bac00640 1 -- 172.21.15.145:0/191129482 shutdown_connections 2024-05-09T04:35:09.372 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.371+0000 7fe2bac00640 1 -- 172.21.15.145:0/191129482 wait complete. 2024-05-09T04:35:09.372 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.372+0000 7fe2bac00640 1 Processor -- start 2024-05-09T04:35:09.372 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.373+0000 7fe2bac00640 1 -- start start 2024-05-09T04:35:09.374 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.374+0000 7fe2bac00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe298003a00 0x7fe2b415c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:09.374 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.374+0000 7fe2bac00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe2b4158520 0x7fe2b414f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:09.374 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.374+0000 7fe2bac00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe2b40b2510 0x7fe2b40b2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:09.374 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.374+0000 7fe2bac00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b40b52e0 con 0x7fe2b4158520 2024-05-09T04:35:09.374 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.374+0000 7fe2bac00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b40d2750 con 0x7fe298003a00 2024-05-09T04:35:09.374 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.374+0000 7fe2bac00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe2b40d3aa0 con 0x7fe2b40b2510 2024-05-09T04:35:09.374 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.374+0000 7fe2b8e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe2b4158520 0x7fe2b414f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:09.374 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.374+0000 7fe2b8e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe2b4158520 0x7fe2b414f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:53330/0 (socket says 172.21.15.145:53330) 2024-05-09T04:35:09.374 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.374+0000 7fe2b8e00640 1 -- 172.21.15.145:0/2880119574 learned_addr learned my addr 172.21.15.145:0/2880119574 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:09.375 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.374+0000 7fe2b8e00640 1 -- 172.21.15.145:0/2880119574 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe2b40b2510 msgr2=0x7fe2b40b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:09.375 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.374+0000 7fe2b8e00640 1 --2- 172.21.15.145:0/2880119574 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe2b40b2510 0x7fe2b40b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:09.375 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.375+0000 7fe2b8e00640 1 -- 172.21.15.145:0/2880119574 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe298003a00 msgr2=0x7fe2b415c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:09.376 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.375+0000 7fe2b8e00640 1 --2- 172.21.15.145:0/2880119574 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe298003a00 0x7fe2b415c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:09.376 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.375+0000 7fe2b8e00640 1 -- 172.21.15.145:0/2880119574 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2b4159040 con 0x7fe2b4158520 2024-05-09T04:35:09.376 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.375+0000 7fe2b8e00640 1 --2- 172.21.15.145:0/2880119574 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe2b4158520 0x7fe2b414f2c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe2a000e860 tx=0x7fe2a000ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:09.376 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.375+0000 7fe2b1600640 1 -- 172.21.15.145:0/2880119574 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a0017070 con 0x7fe2b4158520 2024-05-09T04:35:09.376 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.375+0000 7fe2b1600640 1 -- 172.21.15.145:0/2880119574 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2a000c8e0 con 0x7fe2b4158520 2024-05-09T04:35:09.376 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.375+0000 7fe2b1600640 1 -- 172.21.15.145:0/2880119574 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a000ccb0 con 0x7fe2b4158520 2024-05-09T04:35:09.376 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.375+0000 7fe2bac00640 1 -- 172.21.15.145:0/2880119574 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2b4159320 con 0x7fe2b4158520 2024-05-09T04:35:09.376 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.375+0000 7fe2bac00640 1 -- 172.21.15.145:0/2880119574 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2b41598e0 con 0x7fe2b4158520 2024-05-09T04:35:09.376 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.376+0000 7fe2bac00640 1 -- 172.21.15.145:0/2880119574 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2b406a310 con 0x7fe2b4158520 2024-05-09T04:35:09.377 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.378+0000 7fe2b1600640 1 -- 172.21.15.145:0/2880119574 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe2a001a070 con 0x7fe2b4158520 2024-05-09T04:35:09.378 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.378+0000 7fe2b1600640 1 --2- 172.21.15.145:0/2880119574 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fe294076300 0x7fe2940787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:09.378 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.379+0000 7fe2b1600640 1 -- 172.21.15.145:0/2880119574 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 6104+0+0 (secure 0 0 0) 0x7fe2a0097a30 con 0x7fe2b4158520 2024-05-09T04:35:09.378 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.379+0000 7fe2b9800640 1 --2- 172.21.15.145:0/2880119574 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fe294076300 0x7fe2940787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:09.379 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.379+0000 7fe2b9800640 1 --2- 172.21.15.145:0/2880119574 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fe294076300 0x7fe2940787c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe2a400ba00 tx=0x7fe2a402c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:09.382 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.383+0000 7fe2b1600640 1 -- 172.21.15.145:0/2880119574 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe2a0061160 con 0x7fe2b4158520 2024-05-09T04:35:09.591 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.591+0000 7fe2bac00640 1 -- 172.21.15.145:0/2880119574 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe2b40d0d60 con 0x7fe2b4158520 2024-05-09T04:35:09.593 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.593+0000 7fe2b1600640 1 -- 172.21.15.145:0/2880119574 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+18529 (secure 0 0 0) 0x7fe2a0060a60 con 0x7fe2b4158520 2024-05-09T04:35:09.594 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:35:09.594 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":35,"fsid":"957ee6c1-073b-44cf-9b93-9491b3fdfc1d","created":"2024-05-09T04:33:26.544367+0000","modified":"2024-05-09T04:35:01.635404+0000","last_up_change":"2024-05-09T04:33:39.039318+0000","last_in_change":"2024-05-09T04:33:29.902361+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T04:33:39.795344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-05-09T04:34:40.995268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-05-09T04:34:42.107773+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"data_new","create_time":"2024-05-09T04:34:59.021572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22e550ad-847b-4952-80e6-63d859ed1a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6801","nonce":3182572912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6803","nonce":3182572912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6807","nonce":3182572912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6805","nonce":3182572912}]},"public_addr":"172.21.15.145:6801/3182572912","cluster_addr":"172.21.15.145:6803/3182572912","heartbeat_back_addr":"172.21.15.145:6807/3182572912","heartbeat_front_addr":"172.21.15.145:6805/3182572912","state":["exists","up"]},{"osd":1,"uuid":"0f6b9fca-0c41-4145-8d74-598ddb42f2a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6825","nonce":1845225555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6827","nonce":1845225555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6831","nonce":1845225555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6829","nonce":1845225555}]},"public_addr":"172.21.15.145:6825/1845225555","cluster_addr":"172.21.15.145:6827/1845225555","heartbeat_back_addr":"172.21.15.145:6831/1845225555","heartbeat_front_addr":"172.21.15.145:6829/1845225555","state":["exists","up"]},{"osd":2,"uuid":"b3b1f67e-5790-483f-8610-e05a5bce4c30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6817","nonce":2168958993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6819","nonce":2168958993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6823","nonce":2168958993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6821","nonce":2168958993}]},"public_addr":"172.21.15.145:6817/2168958993","cluster_addr":"172.21.15.145:6819/2168958993","heartbeat_back_addr":"172.21.15.145:6823/2168958993","heartbeat_front_addr":"172.21.15.145:6821/2168958993","state":["exists","up"]},{"osd":3,"uuid":"884787a5-e037-4d31-9aae-743030eba4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6809","nonce":337009826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6811","nonce":337009826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6815","nonce":337009826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6813","nonce":337009826}]},"public_addr":"172.21.15.145:6809/337009826","cluster_addr":"172.21.15.145:6811/337009826","heartbeat_back_addr":"172.21.15.145:6815/337009826","heartbeat_front_addr":"172.21.15.145:6813/337009826","state":["exists","up"]},{"osd":4,"uuid":"0759fc1e-c5fc-4a2f-86bb-e8eccd83f833","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6818","nonce":3117700995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6821","nonce":3117700995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6827","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6829","nonce":3117700995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6823","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6825","nonce":3117700995}]},"public_addr":"172.21.15.192:6818/3117700995","cluster_addr":"172.21.15.192:6821/3117700995","heartbeat_back_addr":"172.21.15.192:6829/3117700995","heartbeat_front_addr":"172.21.15.192:6825/3117700995","state":["exists","up"]},{"osd":5,"uuid":"9443cc22-9379-46e3-9173-98b592bf2c16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6801","nonce":1328373268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6803","nonce":1328373268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6807","nonce":1328373268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6805","nonce":1328373268}]},"public_addr":"172.21.15.192:6801/1328373268","cluster_addr":"172.21.15.192:6803/1328373268","heartbeat_back_addr":"172.21.15.192:6807/1328373268","heartbeat_front_addr":"172.21.15.192:6805/1328373268","state":["exists","up"]},{"osd":6,"uuid":"d05a3311-505a-45b5-85b9-aa3113e96987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6817","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6820","nonce":1868011057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6824","nonce":1868011057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6831","nonce":1868011057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6828","nonce":1868011057}]},"public_addr":"172.21.15.192:6820/1868011057","cluster_addr":"172.21.15.192:6824/1868011057","heartbeat_back_addr":"172.21.15.192:6831/1868011057","heartbeat_front_addr":"172.21.15.192:6828/1868011057","state":["exists","up"]},{"osd":7,"uuid":"c7c23dbc-2029-43ec-9f53-25f3d58019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6809","nonce":3368512157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6811","nonce":3368512157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6815","nonce":3368512157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6813","nonce":3368512157}]},"public_addr":"172.21.15.192:6809/3368512157","cluster_addr":"172.21.15.192:6811/3368512157","heartbeat_back_addr":"172.21.15.192:6815/3368512157","heartbeat_front_addr":"172.21.15.192:6813/3368512157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.091188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.032567+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:34.797468+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.228839+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.622400+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.473103+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.566092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.349395+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T04:35:09.597 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.597+0000 7fe2bac00640 1 -- 172.21.15.145:0/2880119574 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fe294076300 msgr2=0x7fe2940787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:09.597 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.597+0000 7fe2bac00640 1 --2- 172.21.15.145:0/2880119574 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fe294076300 0x7fe2940787c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe2a400ba00 tx=0x7fe2a402c000 comp rx=0 tx=0).stop 2024-05-09T04:35:09.597 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.597+0000 7fe2bac00640 1 -- 172.21.15.145:0/2880119574 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe2b4158520 msgr2=0x7fe2b414f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:09.597 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.597+0000 7fe2bac00640 1 --2- 172.21.15.145:0/2880119574 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe2b4158520 0x7fe2b414f2c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe2a000e860 tx=0x7fe2a000ed20 comp rx=0 tx=0).stop 2024-05-09T04:35:09.597 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.597+0000 7fe2bac00640 1 -- 172.21.15.145:0/2880119574 shutdown_connections 2024-05-09T04:35:09.597 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.597+0000 7fe2bac00640 1 --2- 172.21.15.145:0/2880119574 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fe294076300 0x7fe2940787c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:09.598 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.597+0000 7fe2bac00640 1 --2- 172.21.15.145:0/2880119574 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe2b40b2510 0x7fe2b40b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:09.598 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.597+0000 7fe2bac00640 1 --2- 172.21.15.145:0/2880119574 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe2b4158520 0x7fe2b414f2c0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:09.598 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.597+0000 7fe2bac00640 1 --2- 172.21.15.145:0/2880119574 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe298003a00 0x7fe2b415c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:09.598 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.597+0000 7fe2bac00640 1 -- 172.21.15.145:0/2880119574 >> 172.21.15.145:0/2880119574 conn(0x7fe2b40d4fb0 msgr2=0x7fe2b40b3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:09.598 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.597+0000 7fe2bac00640 1 -- 172.21.15.145:0/2880119574 shutdown_connections 2024-05-09T04:35:09.598 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.597+0000 7fe2bac00640 1 -- 172.21.15.145:0/2880119574 wait complete. 2024-05-09T04:35:09.616 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-09T04:35:09.801 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.801+0000 7fa95a800640 1 Processor -- start 2024-05-09T04:35:09.801 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.801+0000 7fa95a800640 1 -- start start 2024-05-09T04:35:09.803 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.802+0000 7fa95a800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fa954140070 0x7fa9540cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:09.803 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.802+0000 7fa95a800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9540ce330 0x7fa954142ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:09.803 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.803+0000 7fa95a800640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa95415d780 con 0x7fa9541430e0 2024-05-09T04:35:09.803 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.803+0000 7fa95a800640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fa9540ce790 con 0x7fa954140070 2024-05-09T04:35:09.803 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.803+0000 7fa95a800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa95415e330 con 0x7fa9540ce330 2024-05-09T04:35:09.803 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.803+0000 7fa953400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fa954140070 0x7fa9540cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:09.803 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.803+0000 7fa953400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fa954140070 0x7fa9540cddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:53336/0 (socket says 172.21.15.145:53336) 2024-05-09T04:35:09.803 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.803+0000 7fa953400640 1 -- 172.21.15.145:0/3946702601 learned_addr learned my addr 172.21.15.145:0/3946702601 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:09.804 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.803+0000 7fa953400640 1 -- 172.21.15.145:0/3946702601 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9540ce330 msgr2=0x7fa954142ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:09.804 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.803+0000 7fa952a00640 1 --2- 172.21.15.145:0/3946702601 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9540ce330 0x7fa954142ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:09.804 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.803+0000 7fa953400640 1 --2- 172.21.15.145:0/3946702601 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9540ce330 0x7fa954142ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:09.804 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.803+0000 7fa953400640 1 -- 172.21.15.145:0/3946702601 >> v1:172.21.15.192:6789/0 conn(0x7fa9541430e0 legacy=0x7fa954072270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:09.804 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.803+0000 7fa953400640 1 -- 172.21.15.145:0/3946702601 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa95415da00 con 0x7fa954140070 2024-05-09T04:35:09.804 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.805+0000 7fa953400640 1 --2- 172.21.15.145:0/3946702601 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa954140070 0x7fa9540cddf0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa944009870 tx=0x7fa94402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae94d23b9e9bf7f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:09.804 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.805+0000 7fa952000640 1 -- 172.21.15.145:0/3946702601 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa944037070 con 0x7fa954140070 2024-05-09T04:35:09.805 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.805+0000 7fa952000640 1 -- 172.21.15.145:0/3946702601 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa944032440 con 0x7fa954140070 2024-05-09T04:35:09.805 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.805+0000 7fa952000640 1 -- 172.21.15.145:0/3946702601 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa944032810 con 0x7fa954140070 2024-05-09T04:35:09.805 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.806+0000 7fa95a800640 1 -- 172.21.15.145:0/3946702601 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa954140070 msgr2=0x7fa9540cddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:09.806 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.806+0000 7fa95a800640 1 --2- 172.21.15.145:0/3946702601 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa954140070 0x7fa9540cddf0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa944009870 tx=0x7fa94402a1c0 comp rx=0 tx=0).stop 2024-05-09T04:35:09.806 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.806+0000 7fa95a800640 1 -- 172.21.15.145:0/3946702601 shutdown_connections 2024-05-09T04:35:09.806 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.806+0000 7fa95a800640 1 --2- 172.21.15.145:0/3946702601 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9540ce330 0x7fa954142ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:09.806 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.806+0000 7fa95a800640 1 --2- 172.21.15.145:0/3946702601 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa954140070 0x7fa9540cddf0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:09.806 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.806+0000 7fa95a800640 1 -- 172.21.15.145:0/3946702601 >> 172.21.15.145:0/3946702601 conn(0x7fa95406c8d0 msgr2=0x7fa95406cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:09.806 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.806+0000 7fa95a800640 1 -- 172.21.15.145:0/3946702601 shutdown_connections 2024-05-09T04:35:09.806 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.806+0000 7fa95a800640 1 -- 172.21.15.145:0/3946702601 wait complete. 2024-05-09T04:35:09.807 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.808+0000 7fa95a800640 1 Processor -- start 2024-05-09T04:35:09.807 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.808+0000 7fa95a800640 1 -- start start 2024-05-09T04:35:09.808 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.809+0000 7fa95a800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa9540ce330 0x7fa9541572a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:09.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.809+0000 7fa95a800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa954140070 0x7fa95415c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:09.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.809+0000 7fa95a800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9541430e0 0x7fa95415c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:09.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.809+0000 7fa95a800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa954140850 con 0x7fa9540ce330 2024-05-09T04:35:09.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.809+0000 7fa95a800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa95415dfc0 con 0x7fa9541430e0 2024-05-09T04:35:09.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.809+0000 7fa95a800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa954140450 con 0x7fa954140070 2024-05-09T04:35:09.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.809+0000 7fa953400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa9540ce330 0x7fa9541572a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:09.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.809+0000 7fa953400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa9540ce330 0x7fa9541572a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:53340/0 (socket says 172.21.15.145:53340) 2024-05-09T04:35:09.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.809+0000 7fa953400640 1 -- 172.21.15.145:0/889400506 learned_addr learned my addr 172.21.15.145:0/889400506 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:09.810 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.809+0000 7fa953400640 1 -- 172.21.15.145:0/889400506 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa954140070 msgr2=0x7fa95415c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:09.810 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.809+0000 7fa953400640 1 --2- 172.21.15.145:0/889400506 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa954140070 0x7fa95415c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:09.810 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.809+0000 7fa953400640 1 -- 172.21.15.145:0/889400506 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9541430e0 msgr2=0x7fa95415c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:09.810 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.809+0000 7fa953400640 1 --2- 172.21.15.145:0/889400506 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9541430e0 0x7fa95415c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:09.811 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.809+0000 7fa953400640 1 -- 172.21.15.145:0/889400506 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9541f5070 con 0x7fa9540ce330 2024-05-09T04:35:09.811 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.810+0000 7fa953400640 1 --2- 172.21.15.145:0/889400506 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa9540ce330 0x7fa9541572a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fa944009870 tx=0x7fa9440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:09.811 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.810+0000 7fa94b400640 1 -- 172.21.15.145:0/889400506 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa944041020 con 0x7fa9540ce330 2024-05-09T04:35:09.811 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.810+0000 7fa94b400640 1 -- 172.21.15.145:0/889400506 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa944030970 con 0x7fa9540ce330 2024-05-09T04:35:09.811 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.810+0000 7fa94b400640 1 -- 172.21.15.145:0/889400506 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa944037040 con 0x7fa9540ce330 2024-05-09T04:35:09.811 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.810+0000 7fa95a800640 1 -- 172.21.15.145:0/889400506 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9541f5330 con 0x7fa9540ce330 2024-05-09T04:35:09.811 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.810+0000 7fa95a800640 1 -- 172.21.15.145:0/889400506 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9541f58f0 con 0x7fa9540ce330 2024-05-09T04:35:09.811 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.811+0000 7fa95a800640 1 -- 172.21.15.145:0/889400506 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9541f5fb0 con 0x7fa9540ce330 2024-05-09T04:35:09.813 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.814+0000 7fa94b400640 1 -- 172.21.15.145:0/889400506 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa944044020 con 0x7fa9540ce330 2024-05-09T04:35:09.815 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.815+0000 7fa94b400640 1 --2- 172.21.15.145:0/889400506 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fa9340762d0 0x7fa934078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:09.815 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.815+0000 7fa94b400640 1 -- 172.21.15.145:0/889400506 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 6104+0+0 (secure 0 0 0) 0x7fa9440b83e0 con 0x7fa9540ce330 2024-05-09T04:35:09.815 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.815+0000 7fa952a00640 1 --2- 172.21.15.145:0/889400506 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fa9340762d0 0x7fa934078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:09.815 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.816+0000 7fa952a00640 1 --2- 172.21.15.145:0/889400506 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fa9340762d0 0x7fa934078790 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa95406ce20 tx=0x7fa940008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:09.818 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:09.818+0000 7fa94b400640 1 -- 172.21.15.145:0/889400506 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa9541f5fb0 con 0x7fa9540ce330 2024-05-09T04:35:10.064 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.064+0000 7fa95a800640 1 -- 172.21.15.145:0/889400506 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fa95406a310 con 0x7fa9540ce330 2024-05-09T04:35:10.065 INFO:tasks.ceph.mon.a.smithi145.stderr:2024-05-09T04:35:10.065+0000 7f50dee00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T04:35:10.069 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.069+0000 7fa94b400640 1 -- 172.21.15.145:0/889400506 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v19) ==== 161+0+0 (secure 0 0 0) 0x7fa94404b070 con 0x7fa9540ce330 2024-05-09T04:35:10.069 INFO:teuthology.orchestra.run.smithi145.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-09T04:35:10.071 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.072+0000 7fa95a800640 1 -- 172.21.15.145:0/889400506 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fa9340762d0 msgr2=0x7fa934078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:10.072 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.072+0000 7fa95a800640 1 --2- 172.21.15.145:0/889400506 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fa9340762d0 0x7fa934078790 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa95406ce20 tx=0x7fa940008040 comp rx=0 tx=0).stop 2024-05-09T04:35:10.072 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.072+0000 7fa95a800640 1 -- 172.21.15.145:0/889400506 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa9540ce330 msgr2=0x7fa9541572a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:10.072 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.072+0000 7fa95a800640 1 --2- 172.21.15.145:0/889400506 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa9540ce330 0x7fa9541572a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fa944009870 tx=0x7fa9440046c0 comp rx=0 tx=0).stop 2024-05-09T04:35:10.072 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.072+0000 7fa95a800640 1 -- 172.21.15.145:0/889400506 shutdown_connections 2024-05-09T04:35:10.072 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.072+0000 7fa95a800640 1 --2- 172.21.15.145:0/889400506 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fa9340762d0 0x7fa934078790 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:10.072 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.072+0000 7fa95a800640 1 --2- 172.21.15.145:0/889400506 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9541430e0 0x7fa95415c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:10.073 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.072+0000 7fa95a800640 1 --2- 172.21.15.145:0/889400506 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa954140070 0x7fa95415c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:10.073 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.072+0000 7fa95a800640 1 --2- 172.21.15.145:0/889400506 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa9540ce330 0x7fa9541572a0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:10.073 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.072+0000 7fa95a800640 1 -- 172.21.15.145:0/889400506 >> 172.21.15.145:0/889400506 conn(0x7fa95406c8d0 msgr2=0x7fa9540b42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:10.073 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.073+0000 7fa95a800640 1 -- 172.21.15.145:0/889400506 shutdown_connections 2024-05-09T04:35:10.073 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.073+0000 7fa95a800640 1 -- 172.21.15.145:0/889400506 wait complete. 2024-05-09T04:35:10.092 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-09T04:35:10.100 INFO:tasks.ceph.mds.c.smithi145.stderr: -9827> 2024-05-09T04:34:47.483+0000 7fd003c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-09T04:35:10.186 INFO:tasks.ceph.mds.c.smithi145.stdout:starting mds.c at 2024-05-09T04:35:10.280 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.280+0000 7fb8d2e00640 1 Processor -- start 2024-05-09T04:35:10.280 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.280+0000 7fb8d2e00640 1 -- start start 2024-05-09T04:35:10.281 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.281+0000 7fb8d2e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb8cc06dbc0 0x7fb8cc144420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:10.281 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.282+0000 7fb8d2e00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8cc160080 con 0x7fb8cc162e20 2024-05-09T04:35:10.281 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.282+0000 7fb8d2e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8cc160360 con 0x7fb8cc0b4820 2024-05-09T04:35:10.281 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.282+0000 7fb8d2e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb8cc0b5490 con 0x7fb8cc06dbc0 2024-05-09T04:35:10.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.282+0000 7fb8d0e00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7fb8cc162e20 0x7fb8cc144960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:45290/0 (socket says 172.21.15.145:45290) 2024-05-09T04:35:10.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.282+0000 7fb8d0e00640 1 -- 172.21.15.145:0/611197418 learned_addr learned my addr 172.21.15.145:0/611197418 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:35:10.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.282+0000 7fb8cb400640 1 --2- 172.21.15.145:0/611197418 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb8cc06dbc0 0x7fb8cc144420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:10.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.282+0000 7fb8caa00640 1 -- 172.21.15.145:0/611197418 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3766968451 0 0) 0x7fb8cc160080 con 0x7fb8cc162e20 2024-05-09T04:35:10.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.282+0000 7fb8caa00640 1 -- 172.21.15.145:0/611197418 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb8b0003520 con 0x7fb8cc162e20 2024-05-09T04:35:10.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.283+0000 7fb8caa00640 1 -- 172.21.15.145:0/611197418 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 489036191 0 0) 0x7fb8b0003520 con 0x7fb8cc162e20 2024-05-09T04:35:10.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.283+0000 7fb8caa00640 1 -- 172.21.15.145:0/611197418 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb8cc06dbc0 msgr2=0x7fb8cc144420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:10.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.283+0000 7fb8caa00640 1 --2- 172.21.15.145:0/611197418 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb8cc06dbc0 0x7fb8cc144420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:10.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.283+0000 7fb8caa00640 1 -- 172.21.15.145:0/611197418 >> v1:172.21.15.192:6789/0 conn(0x7fb8cc0b4820 legacy=0x7fb8cc15f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:10.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.283+0000 7fb8caa00640 1 -- 172.21.15.145:0/611197418 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8cc160640 con 0x7fb8cc162e20 2024-05-09T04:35:10.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.283+0000 7fb8caa00640 1 -- 172.21.15.145:0/611197418 <== mon.0 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3474413261 0 0) 0x7fb8bc001a70 con 0x7fb8cc162e20 2024-05-09T04:35:10.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.283+0000 7fb8caa00640 1 -- 172.21.15.145:0/611197418 >> v1:172.21.15.145:6789/0 conn(0x7fb8cc162e20 legacy=0x7fb8cc144960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:10.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.283+0000 7fb8caa00640 1 --2- 172.21.15.145:0/611197418 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8b0003a80 0x7fb8b0003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:10.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.283+0000 7fb8caa00640 1 --2- 172.21.15.145:0/611197418 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb8cc06dbc0 0x7fb8b0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:10.285 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.283+0000 7fb8caa00640 1 --2- 172.21.15.145:0/611197418 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb8b0004e60 0x7fb8b000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:10.285 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.283+0000 7fb8caa00640 1 -- 172.21.15.145:0/611197418 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc0b5490 con 0x7fb8b0004e60 2024-05-09T04:35:10.285 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.283+0000 7fb8caa00640 1 -- 172.21.15.145:0/611197418 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b00054f0 con 0x7fb8b0003a80 2024-05-09T04:35:10.285 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.283+0000 7fb8caa00640 1 -- 172.21.15.145:0/611197418 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb8b000bf00 con 0x7fb8cc06dbc0 2024-05-09T04:35:10.285 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.283+0000 7fb8d0e00640 1 --2- 172.21.15.145:0/611197418 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb8b0004e60 0x7fb8b000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:10.285 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.283+0000 7fb8cbe00640 1 --2- 172.21.15.145:0/611197418 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8b0003a80 0x7fb8b0003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:10.285 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.283+0000 7fb8cb400640 1 --2- 172.21.15.145:0/611197418 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb8cc06dbc0 0x7fb8b0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:10.285 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.284+0000 7fb8d0e00640 1 -- 172.21.15.145:0/611197418 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb8cc06dbc0 msgr2=0x7fb8b0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:10.285 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.284+0000 7fb8d0e00640 1 --2- 172.21.15.145:0/611197418 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb8cc06dbc0 0x7fb8b0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:10.285 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.284+0000 7fb8d0e00640 1 -- 172.21.15.145:0/611197418 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8b0003a80 msgr2=0x7fb8b0003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:10.285 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.284+0000 7fb8d0e00640 1 --2- 172.21.15.145:0/611197418 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8b0003a80 0x7fb8b0003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:10.285 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.284+0000 7fb8d0e00640 1 -- 172.21.15.145:0/611197418 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8b0003520 con 0x7fb8b0004e60 2024-05-09T04:35:10.285 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.284+0000 7fb8cbe00640 1 --2- 172.21.15.145:0/611197418 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8b0003a80 0x7fb8b0003ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:35:10.287 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.285+0000 7fb8d0e00640 1 --2- 172.21.15.145:0/611197418 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb8b0004e60 0x7fb8b000b9c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc00a800 tx=0x7fb8b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27dad729015cd908 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:10.287 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.285+0000 7fb8caa00640 1 -- 172.21.15.145:0/611197418 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8bc00a450 con 0x7fb8b0004e60 2024-05-09T04:35:10.287 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.285+0000 7fb8caa00640 1 -- 172.21.15.145:0/611197418 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8bc02f070 con 0x7fb8b0004e60 2024-05-09T04:35:10.287 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.285+0000 7fb8caa00640 1 -- 172.21.15.145:0/611197418 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8bc02bba0 con 0x7fb8b0004e60 2024-05-09T04:35:10.287 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.286+0000 7fb8d2e00640 1 -- 172.21.15.145:0/611197418 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb8b0004e60 msgr2=0x7fb8b000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:10.287 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.286+0000 7fb8d2e00640 1 --2- 172.21.15.145:0/611197418 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb8b0004e60 0x7fb8b000b9c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc00a800 tx=0x7fb8b0005830 comp rx=0 tx=0).stop 2024-05-09T04:35:10.287 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.286+0000 7fb8d0e00640 1 -- 172.21.15.145:0/611197418 reap_dead start 2024-05-09T04:35:10.287 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.286+0000 7fb8d2e00640 1 -- 172.21.15.145:0/611197418 shutdown_connections 2024-05-09T04:35:10.287 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.286+0000 7fb8d2e00640 1 -- 172.21.15.145:0/611197418 >> 172.21.15.145:0/611197418 conn(0x7fb8cc0d15c0 msgr2=0x7fb8cc0d19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:10.287 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.286+0000 7fb8d2e00640 1 -- 172.21.15.145:0/611197418 shutdown_connections 2024-05-09T04:35:10.287 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.286+0000 7fb8d2e00640 1 -- 172.21.15.145:0/611197418 wait complete. 2024-05-09T04:35:10.288 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.287+0000 7fb8d2e00640 1 Processor -- start 2024-05-09T04:35:10.288 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.288+0000 7fb8d2e00640 1 -- start start 2024-05-09T04:35:10.288 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.288+0000 7fb8d2e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8b0004e60 0x7fb8cc0d0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:10.289 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.288+0000 7fb8d2e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb8cc0cdfd0 0x7fb8cc15cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:10.289 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.289+0000 7fb8d2e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb8cc06f960 0x7fb8cc06fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:10.289 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.289+0000 7fb8d2e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc0b4d80 con 0x7fb8cc0cdfd0 2024-05-09T04:35:10.289 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.289+0000 7fb8d2e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc0b4c00 con 0x7fb8b0004e60 2024-05-09T04:35:10.289 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.289+0000 7fb8d2e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb8cc0b4f00 con 0x7fb8cc06f960 2024-05-09T04:35:10.289 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.289+0000 7fb8cb400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb8cc0cdfd0 0x7fb8cc15cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:10.289 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.289+0000 7fb8cb400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb8cc0cdfd0 0x7fb8cc15cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:53382/0 (socket says 172.21.15.145:53382) 2024-05-09T04:35:10.289 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.289+0000 7fb8cb400640 1 -- 172.21.15.145:0/2798190785 learned_addr learned my addr 172.21.15.145:0/2798190785 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:10.290 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.289+0000 7fb8cb400640 1 -- 172.21.15.145:0/2798190785 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb8cc06f960 msgr2=0x7fb8cc06fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:10.290 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.289+0000 7fb8cb400640 1 --2- 172.21.15.145:0/2798190785 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb8cc06f960 0x7fb8cc06fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:10.290 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.289+0000 7fb8cb400640 1 -- 172.21.15.145:0/2798190785 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8b0004e60 msgr2=0x7fb8cc0d0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:10.290 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.289+0000 7fb8cbe00640 1 --2- 172.21.15.145:0/2798190785 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8b0004e60 0x7fb8cc0d0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:10.290 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.289+0000 7fb8cb400640 1 --2- 172.21.15.145:0/2798190785 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8b0004e60 0x7fb8cc0d0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:10.291 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.289+0000 7fb8cb400640 1 -- 172.21.15.145:0/2798190785 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8cc15d220 con 0x7fb8cc0cdfd0 2024-05-09T04:35:10.291 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.290+0000 7fb8cb400640 1 --2- 172.21.15.145:0/2798190785 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb8cc0cdfd0 0x7fb8cc15cce0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fb8b400f560 tx=0x7fb8b400fa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:10.291 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.290+0000 7fb8c3e00640 1 -- 172.21.15.145:0/2798190785 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b4019020 con 0x7fb8cc0cdfd0 2024-05-09T04:35:10.291 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.290+0000 7fb8c3e00640 1 -- 172.21.15.145:0/2798190785 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8b4014870 con 0x7fb8cc0cdfd0 2024-05-09T04:35:10.291 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.290+0000 7fb8d2e00640 1 -- 172.21.15.145:0/2798190785 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8cc0ce740 con 0x7fb8cc0cdfd0 2024-05-09T04:35:10.291 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.290+0000 7fb8c3e00640 1 -- 172.21.15.145:0/2798190785 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b4014c60 con 0x7fb8cc0cdfd0 2024-05-09T04:35:10.291 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.290+0000 7fb8d2e00640 1 -- 172.21.15.145:0/2798190785 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8cc0ced00 con 0x7fb8cc0cdfd0 2024-05-09T04:35:10.291 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.291+0000 7fb8d2e00640 1 -- 172.21.15.145:0/2798190785 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8cc06a310 con 0x7fb8cc0cdfd0 2024-05-09T04:35:10.292 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.292+0000 7fb8c3e00640 1 -- 172.21.15.145:0/2798190785 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb8b401c020 con 0x7fb8cc0cdfd0 2024-05-09T04:35:10.293 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.293+0000 7fb8c3e00640 1 --2- 172.21.15.145:0/2798190785 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fb8a80762d0 0x7fb8a8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:10.293 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.293+0000 7fb8c3e00640 1 -- 172.21.15.145:0/2798190785 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6190+0+0 (secure 0 0 0) 0x7fb8b4097ff0 con 0x7fb8cc0cdfd0 2024-05-09T04:35:10.293 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.294+0000 7fb8cbe00640 1 --2- 172.21.15.145:0/2798190785 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fb8a80762d0 0x7fb8a8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:10.294 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.294+0000 7fb8cbe00640 1 --2- 172.21.15.145:0/2798190785 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fb8a80762d0 0x7fb8a8078790 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4001630 tx=0x7fb8c4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:10.297 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.298+0000 7fb8c3e00640 1 -- 172.21.15.145:0/2798190785 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb8b4023020 con 0x7fb8cc0cdfd0 2024-05-09T04:35:10.550 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:10.550+0000 7fb8d2e00640 1 -- 172.21.15.145:0/2798190785 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fb8cc163d30 con 0x7fb8cc0cdfd0 2024-05-09T04:35:11.073 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.073+0000 7fb8c3e00640 1 -- 172.21.15.145:0/2798190785 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v20) ==== 104+0+0 (secure 0 0 0) 0x7fb8b4061680 con 0x7fb8cc0cdfd0 2024-05-09T04:35:11.077 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.077+0000 7fb8d2e00640 1 -- 172.21.15.145:0/2798190785 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fb8a80762d0 msgr2=0x7fb8a8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:11.077 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.077+0000 7fb8d2e00640 1 --2- 172.21.15.145:0/2798190785 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fb8a80762d0 0x7fb8a8078790 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4001630 tx=0x7fb8c4009040 comp rx=0 tx=0).stop 2024-05-09T04:35:11.077 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.077+0000 7fb8d2e00640 1 -- 172.21.15.145:0/2798190785 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb8cc0cdfd0 msgr2=0x7fb8cc15cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:11.077 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.077+0000 7fb8d2e00640 1 --2- 172.21.15.145:0/2798190785 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb8cc0cdfd0 0x7fb8cc15cce0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fb8b400f560 tx=0x7fb8b400fa20 comp rx=0 tx=0).stop 2024-05-09T04:35:11.078 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.078+0000 7fb8d2e00640 1 -- 172.21.15.145:0/2798190785 shutdown_connections 2024-05-09T04:35:11.078 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.078+0000 7fb8d2e00640 1 --2- 172.21.15.145:0/2798190785 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fb8a80762d0 0x7fb8a8078790 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:11.078 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.078+0000 7fb8d2e00640 1 --2- 172.21.15.145:0/2798190785 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb8cc06f960 0x7fb8cc06fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:11.078 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.078+0000 7fb8d2e00640 1 --2- 172.21.15.145:0/2798190785 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb8cc0cdfd0 0x7fb8cc15cce0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:11.078 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.078+0000 7fb8d2e00640 1 --2- 172.21.15.145:0/2798190785 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8b0004e60 0x7fb8cc0d0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:11.078 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.078+0000 7fb8d2e00640 1 -- 172.21.15.145:0/2798190785 >> 172.21.15.145:0/2798190785 conn(0x7fb8cc0d15c0 msgr2=0x7fb8cc15e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:11.078 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.078+0000 7fb8d2e00640 1 -- 172.21.15.145:0/2798190785 shutdown_connections 2024-05-09T04:35:11.079 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.078+0000 7fb8d2e00640 1 -- 172.21.15.145:0/2798190785 wait complete. 2024-05-09T04:35:11.100 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-09T04:35:11.347 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.347+0000 7fd0a4c00640 1 Processor -- start 2024-05-09T04:35:11.347 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.347+0000 7fd0a4c00640 1 -- start start 2024-05-09T04:35:11.348 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.348+0000 7fd0a4c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd0a00d33f0 0x7fd0a00d5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:11.348 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.348+0000 7fd0a4c00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd0a00b2600 0x7fd0a00b2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:11.348 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.348+0000 7fd0a4c00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0a00d0fd0 con 0x7fd0a00b40a0 2024-05-09T04:35:11.348 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.348+0000 7fd0a4c00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fd0a00b2170 con 0x7fd0a00b2600 2024-05-09T04:35:11.349 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.348+0000 7fd0a4c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd0a00d0a10 con 0x7fd0a00d33f0 2024-05-09T04:35:11.349 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.349+0000 7fd09f400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd0a00d33f0 0x7fd0a00d5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:11.349 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.349+0000 7fd09fe00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd0a00b2600 0x7fd0a00b2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:11.349 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.349+0000 7fd097e00640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7fd0a00b40a0 0x7fd0a00d37d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.145:34910/0 (socket says 172.21.15.145:34910) 2024-05-09T04:35:11.349 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.349+0000 7fd097e00640 1 -- 172.21.15.145:0/1242286245 learned_addr learned my addr 172.21.15.145:0/1242286245 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:35:11.349 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.349+0000 7fd09f400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd0a00d33f0 0x7fd0a00d5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.145:54332/0 (socket says 172.21.15.145:54332) 2024-05-09T04:35:11.350 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.350+0000 7fd09fe00640 1 -- 172.21.15.145:0/1242286245 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd0a00d33f0 msgr2=0x7fd0a00d5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:11.350 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.350+0000 7fd09fe00640 1 --2- 172.21.15.145:0/1242286245 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd0a00d33f0 0x7fd0a00d5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:11.350 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.350+0000 7fd09fe00640 1 -- 172.21.15.145:0/1242286245 >> v1:172.21.15.192:6789/0 conn(0x7fd0a00b40a0 legacy=0x7fd0a00d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:11.350 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.350+0000 7fd09fe00640 1 -- 172.21.15.145:0/1242286245 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a00d0cf0 con 0x7fd0a00b2600 2024-05-09T04:35:11.351 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.350+0000 7fd09ea00640 1 -- 172.21.15.145:0/1242286245 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3650827273 0 0) 0x7fd0a00d0fd0 con 0x7fd0a00b40a0 2024-05-09T04:35:11.351 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.352+0000 7fd09fe00640 1 --2- 172.21.15.145:0/1242286245 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0a00b2600 0x7fd0a00b2a50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd098009d90 tx=0x7fd09802a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29082d7b0ba3c3f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:11.352 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.352+0000 7fd09ea00640 1 -- 172.21.15.145:0/1242286245 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd098031600 con 0x7fd0a00b2600 2024-05-09T04:35:11.352 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.352+0000 7fd09ea00640 1 -- 172.21.15.145:0/1242286245 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0980317a0 con 0x7fd0a00b2600 2024-05-09T04:35:11.352 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.352+0000 7fd09ea00640 1 -- 172.21.15.145:0/1242286245 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd098031b70 con 0x7fd0a00b2600 2024-05-09T04:35:11.353 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.353+0000 7fd0a4c00640 1 -- 172.21.15.145:0/1242286245 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0a00b2600 msgr2=0x7fd0a00b2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:11.353 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.353+0000 7fd0a4c00640 1 --2- 172.21.15.145:0/1242286245 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0a00b2600 0x7fd0a00b2a50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd098009d90 tx=0x7fd09802a5d0 comp rx=0 tx=0).stop 2024-05-09T04:35:11.353 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.353+0000 7fd0a4c00640 1 -- 172.21.15.145:0/1242286245 shutdown_connections 2024-05-09T04:35:11.353 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.353+0000 7fd0a4c00640 1 --2- 172.21.15.145:0/1242286245 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0a00b2600 0x7fd0a00b2a50 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:11.353 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.353+0000 7fd0a4c00640 1 --2- 172.21.15.145:0/1242286245 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd0a00d33f0 0x7fd0a00d5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:11.353 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.353+0000 7fd0a4c00640 1 -- 172.21.15.145:0/1242286245 >> 172.21.15.145:0/1242286245 conn(0x7fd0a009e400 msgr2=0x7fd0a00d50c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:11.353 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.353+0000 7fd0a4c00640 1 -- 172.21.15.145:0/1242286245 shutdown_connections 2024-05-09T04:35:11.354 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.353+0000 7fd0a4c00640 1 -- 172.21.15.145:0/1242286245 wait complete. 2024-05-09T04:35:11.354 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.355+0000 7fd0a4c00640 1 Processor -- start 2024-05-09T04:35:11.355 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.355+0000 7fd0a4c00640 1 -- start start 2024-05-09T04:35:11.357 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.356+0000 7fd0a4c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd0a00b2600 0x7fd0a00cd230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:11.357 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.356+0000 7fd0a4c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd0a00b40a0 0x7fd0a0154400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:11.357 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.356+0000 7fd0a4c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0a00d33f0 0x7fd0a00b2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:11.358 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.356+0000 7fd0a4c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a0162c10 con 0x7fd0a00b40a0 2024-05-09T04:35:11.358 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.356+0000 7fd0a4c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a00d06e0 con 0x7fd0a00d33f0 2024-05-09T04:35:11.358 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.356+0000 7fd0a4c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd0a0162d90 con 0x7fd0a00b2600 2024-05-09T04:35:11.358 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.356+0000 7fd097e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd0a00b40a0 0x7fd0a0154400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:11.358 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.356+0000 7fd097e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd0a00b40a0 0x7fd0a0154400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:43214/0 (socket says 172.21.15.145:43214) 2024-05-09T04:35:11.358 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.356+0000 7fd097e00640 1 -- 172.21.15.145:0/1139177490 learned_addr learned my addr 172.21.15.145:0/1139177490 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:11.358 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.357+0000 7fd09fe00640 1 --2- 172.21.15.145:0/1139177490 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0a00d33f0 0x7fd0a00b2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:11.358 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.357+0000 7fd09f400640 1 --2- 172.21.15.145:0/1139177490 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd0a00b2600 0x7fd0a00cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:11.358 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.357+0000 7fd097e00640 1 -- 172.21.15.145:0/1139177490 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd0a00b2600 msgr2=0x7fd0a00cd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:11.358 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.357+0000 7fd097e00640 1 --2- 172.21.15.145:0/1139177490 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd0a00b2600 0x7fd0a00cd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:11.358 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.357+0000 7fd097e00640 1 -- 172.21.15.145:0/1139177490 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0a00d33f0 msgr2=0x7fd0a00b2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:11.358 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.357+0000 7fd097e00640 1 --2- 172.21.15.145:0/1139177490 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0a00d33f0 0x7fd0a00b2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:11.359 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.357+0000 7fd097e00640 1 -- 172.21.15.145:0/1139177490 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a00cc190 con 0x7fd0a00b40a0 2024-05-09T04:35:11.359 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.357+0000 7fd09fe00640 1 --2- 172.21.15.145:0/1139177490 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0a00d33f0 0x7fd0a00b2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:35:11.359 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.357+0000 7fd097e00640 1 --2- 172.21.15.145:0/1139177490 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd0a00b40a0 0x7fd0a0154400 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fd08800c3e0 tx=0x7fd08800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:11.359 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.357+0000 7fd097400640 1 -- 172.21.15.145:0/1139177490 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0880074e0 con 0x7fd0a00b40a0 2024-05-09T04:35:11.359 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.357+0000 7fd097400640 1 -- 172.21.15.145:0/1139177490 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0880047e0 con 0x7fd0a00b40a0 2024-05-09T04:35:11.359 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.357+0000 7fd0a4c00640 1 -- 172.21.15.145:0/1139177490 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0a00cc470 con 0x7fd0a00b40a0 2024-05-09T04:35:11.359 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.357+0000 7fd097400640 1 -- 172.21.15.145:0/1139177490 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd088004bd0 con 0x7fd0a00b40a0 2024-05-09T04:35:11.359 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.358+0000 7fd0a4c00640 1 -- 172.21.15.145:0/1139177490 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0a00cca30 con 0x7fd0a00b40a0 2024-05-09T04:35:11.360 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.359+0000 7fd0a4c00640 1 -- 172.21.15.145:0/1139177490 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0a0076930 con 0x7fd0a00b40a0 2024-05-09T04:35:11.360 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.360+0000 7fd097400640 1 -- 172.21.15.145:0/1139177490 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd088011070 con 0x7fd0a00b40a0 2024-05-09T04:35:11.361 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.361+0000 7fd097400640 1 --2- 172.21.15.145:0/1139177490 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fd07c0762d0 0x7fd07c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:11.361 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.361+0000 7fd097400640 1 -- 172.21.15.145:0/1139177490 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6190+0+0 (secure 0 0 0) 0x7fd0880974d0 con 0x7fd0a00b40a0 2024-05-09T04:35:11.361 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.362+0000 7fd09f400640 1 --2- 172.21.15.145:0/1139177490 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fd07c0762d0 0x7fd07c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:11.362 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.363+0000 7fd09f400640 1 --2- 172.21.15.145:0/1139177490 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fd07c0762d0 0x7fd07c078790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd090009a10 tx=0x7fd090008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:11.365 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.366+0000 7fd097400640 1 -- 172.21.15.145:0/1139177490 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd088060bb0 con 0x7fd0a00b40a0 2024-05-09T04:35:11.571 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:11.571+0000 7fd0a4c00640 1 -- 172.21.15.145:0/1139177490 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd0a00d5fd0 con 0x7fd0a00b40a0 2024-05-09T04:35:12.077 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.077+0000 7fd097400640 1 -- 172.21.15.145:0/1139177490 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v37) ==== 181+0+0 (secure 0 0 0) 0x7fd0880604b0 con 0x7fd0a00b40a0 2024-05-09T04:35:12.077 INFO:teuthology.orchestra.run.smithi145.stderr:pool 'cephfs_metadata' removed 2024-05-09T04:35:12.081 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.081+0000 7fd0a4c00640 1 -- 172.21.15.145:0/1139177490 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fd07c0762d0 msgr2=0x7fd07c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:12.082 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.081+0000 7fd0a4c00640 1 --2- 172.21.15.145:0/1139177490 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fd07c0762d0 0x7fd07c078790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd090009a10 tx=0x7fd090008040 comp rx=0 tx=0).stop 2024-05-09T04:35:12.082 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.081+0000 7fd0a4c00640 1 -- 172.21.15.145:0/1139177490 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd0a00b40a0 msgr2=0x7fd0a0154400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:12.082 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.081+0000 7fd0a4c00640 1 --2- 172.21.15.145:0/1139177490 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd0a00b40a0 0x7fd0a0154400 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fd08800c3e0 tx=0x7fd08800c8a0 comp rx=0 tx=0).stop 2024-05-09T04:35:12.082 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.082+0000 7fd0a4c00640 1 -- 172.21.15.145:0/1139177490 shutdown_connections 2024-05-09T04:35:12.082 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.082+0000 7fd0a4c00640 1 --2- 172.21.15.145:0/1139177490 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fd07c0762d0 0x7fd07c078790 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:12.082 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.082+0000 7fd0a4c00640 1 --2- 172.21.15.145:0/1139177490 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0a00d33f0 0x7fd0a00b2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:12.082 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.082+0000 7fd0a4c00640 1 --2- 172.21.15.145:0/1139177490 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd0a00b40a0 0x7fd0a0154400 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:12.082 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.082+0000 7fd0a4c00640 1 --2- 172.21.15.145:0/1139177490 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd0a00b2600 0x7fd0a00cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:12.082 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.082+0000 7fd0a4c00640 1 -- 172.21.15.145:0/1139177490 >> 172.21.15.145:0/1139177490 conn(0x7fd0a009e400 msgr2=0x7fd0a00d50c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:12.083 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.083+0000 7fd0a4c00640 1 -- 172.21.15.145:0/1139177490 shutdown_connections 2024-05-09T04:35:12.083 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.083+0000 7fd0a4c00640 1 -- 172.21.15.145:0/1139177490 wait complete. 2024-05-09T04:35:12.106 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-09T04:35:12.296 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.296+0000 7f8657000640 1 Processor -- start 2024-05-09T04:35:12.296 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.296+0000 7f8657000640 1 -- start start 2024-05-09T04:35:12.298 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.298+0000 7f8657000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f8650146970 0x7f86500cea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:12.298 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.298+0000 7f8657000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f865006e060 0x7f865015d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:12.298 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.298+0000 7f8657000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f865015dc10 con 0x7f86500cd550 2024-05-09T04:35:12.298 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.298+0000 7f8657000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f8650072ee0 con 0x7f8650146970 2024-05-09T04:35:12.298 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.298+0000 7f8657000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f865015def0 con 0x7f865006e060 2024-05-09T04:35:12.298 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.298+0000 7f8655c00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f8650146970 0x7f86500cea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:12.298 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.298+0000 7f8656600640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f86500cd550 0x7f8650073370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:35484/0 (socket says 172.21.15.145:35484) 2024-05-09T04:35:12.298 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.298+0000 7f8656600640 1 -- 172.21.15.145:0/916152807 learned_addr learned my addr 172.21.15.145:0/916152807 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:35:12.299 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.299+0000 7f8655200640 1 --2- 172.21.15.145:0/916152807 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f865006e060 0x7f865015d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:12.299 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.299+0000 7f8655c00640 1 -- 172.21.15.145:0/916152807 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f865006e060 msgr2=0x7f865015d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:12.299 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.299+0000 7f8655c00640 1 --2- 172.21.15.145:0/916152807 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f865006e060 0x7f865015d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:12.299 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.299+0000 7f8655c00640 1 -- 172.21.15.145:0/916152807 >> v1:172.21.15.145:6789/0 conn(0x7f86500cd550 legacy=0x7f8650073370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:12.299 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.299+0000 7f8655c00640 1 -- 172.21.15.145:0/916152807 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f865015e1d0 con 0x7f8650146970 2024-05-09T04:35:12.300 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.299+0000 7f864fe00640 1 -- 172.21.15.145:0/916152807 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1096463084 0 0) 0x7f865015dc10 con 0x7f86500cd550 2024-05-09T04:35:12.300 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.299+0000 7f8655200640 1 --2- 172.21.15.145:0/916152807 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f865006e060 0x7f865015d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:35:12.300 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.300+0000 7f8655c00640 1 --2- 172.21.15.145:0/916152807 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8650146970 0x7f86500cea40 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f86440097a0 tx=0x7f864402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16616e4a539cea3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:12.301 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.300+0000 7f864fe00640 1 -- 172.21.15.145:0/916152807 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8644037070 con 0x7f8650146970 2024-05-09T04:35:12.301 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.300+0000 7f864fe00640 1 -- 172.21.15.145:0/916152807 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8644032440 con 0x7f8650146970 2024-05-09T04:35:12.301 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.300+0000 7f864fe00640 1 -- 172.21.15.145:0/916152807 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8644032810 con 0x7f8650146970 2024-05-09T04:35:12.301 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.301+0000 7f8657000640 1 -- 172.21.15.145:0/916152807 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8650146970 msgr2=0x7f86500cea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:12.301 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.301+0000 7f8657000640 1 --2- 172.21.15.145:0/916152807 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8650146970 0x7f86500cea40 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f86440097a0 tx=0x7f864402a180 comp rx=0 tx=0).stop 2024-05-09T04:35:12.301 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.301+0000 7f8657000640 1 -- 172.21.15.145:0/916152807 shutdown_connections 2024-05-09T04:35:12.301 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.301+0000 7f8657000640 1 --2- 172.21.15.145:0/916152807 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f865006e060 0x7f865015d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:12.301 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.301+0000 7f8657000640 1 --2- 172.21.15.145:0/916152807 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8650146970 0x7f86500cea40 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:12.301 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.301+0000 7f8657000640 1 -- 172.21.15.145:0/916152807 >> 172.21.15.145:0/916152807 conn(0x7f86500b2850 msgr2=0x7f86500b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:12.301 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.301+0000 7f8657000640 1 -- 172.21.15.145:0/916152807 shutdown_connections 2024-05-09T04:35:12.302 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.301+0000 7f8657000640 1 -- 172.21.15.145:0/916152807 wait complete. 2024-05-09T04:35:12.302 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.303+0000 7f8657000640 1 Processor -- start 2024-05-09T04:35:12.302 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.303+0000 7f8657000640 1 -- start start 2024-05-09T04:35:12.304 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.304+0000 7f8657000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f865006e060 0x7f8650144ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:12.304 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.304+0000 7f8657000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f86500cd550 0x7f86501f2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:12.304 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.304+0000 7f8657000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8650146970 0x7f865013ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:12.304 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.304+0000 7f8657000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f86500ce7c0 con 0x7f8650146970 2024-05-09T04:35:12.304 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.304+0000 7f8657000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f86500ce640 con 0x7f865006e060 2024-05-09T04:35:12.304 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.304+0000 7f8657000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f86500cd9e0 con 0x7f86500cd550 2024-05-09T04:35:12.304 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.304+0000 7f8656600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8650146970 0x7f865013ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:12.304 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.304+0000 7f8656600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8650146970 0x7f865013ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:43238/0 (socket says 172.21.15.145:43238) 2024-05-09T04:35:12.304 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.304+0000 7f8656600640 1 -- 172.21.15.145:0/241959549 learned_addr learned my addr 172.21.15.145:0/241959549 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:12.306 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.304+0000 7f8656600640 1 -- 172.21.15.145:0/241959549 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f86500cd550 msgr2=0x7f86501f2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:12.306 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.304+0000 7f8656600640 1 --2- 172.21.15.145:0/241959549 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f86500cd550 0x7f86501f2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:12.306 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.304+0000 7f8656600640 1 -- 172.21.15.145:0/241959549 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f865006e060 msgr2=0x7f8650144ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:12.306 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.304+0000 7f8655c00640 1 --2- 172.21.15.145:0/241959549 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f865006e060 0x7f8650144ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:12.306 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.304+0000 7f8656600640 1 --2- 172.21.15.145:0/241959549 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f865006e060 0x7f8650144ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:12.306 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.304+0000 7f8656600640 1 -- 172.21.15.145:0/241959549 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f865013dfe0 con 0x7f8650146970 2024-05-09T04:35:12.306 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.304+0000 7f8655c00640 1 --2- 172.21.15.145:0/241959549 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f865006e060 0x7f8650144ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:35:12.306 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.305+0000 7f8656600640 1 --2- 172.21.15.145:0/241959549 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8650146970 0x7f865013ec10 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f864800c7a0 tx=0x7f864800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:12.307 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.305+0000 7f864d600640 1 -- 172.21.15.145:0/241959549 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8648018070 con 0x7f8650146970 2024-05-09T04:35:12.307 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.305+0000 7f864d600640 1 -- 172.21.15.145:0/241959549 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f864800a5e0 con 0x7f8650146970 2024-05-09T04:35:12.307 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.305+0000 7f8657000640 1 -- 172.21.15.145:0/241959549 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86501426f0 con 0x7f8650146970 2024-05-09T04:35:12.307 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.305+0000 7f864d600640 1 -- 172.21.15.145:0/241959549 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864800aa50 con 0x7f8650146970 2024-05-09T04:35:12.307 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.305+0000 7f8657000640 1 -- 172.21.15.145:0/241959549 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8650142cb0 con 0x7f8650146970 2024-05-09T04:35:12.307 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.306+0000 7f8657000640 1 -- 172.21.15.145:0/241959549 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f865015c770 con 0x7f8650146970 2024-05-09T04:35:12.307 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.307+0000 7f864d600640 1 -- 172.21.15.145:0/241959549 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f864801b070 con 0x7f8650146970 2024-05-09T04:35:12.308 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.308+0000 7f864d600640 1 --2- 172.21.15.145:0/241959549 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f86300762d0 0x7f8630078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:12.308 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.308+0000 7f864d600640 1 -- 172.21.15.145:0/241959549 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5752+0+0 (secure 0 0 0) 0x7f8648096c50 con 0x7f8650146970 2024-05-09T04:35:12.308 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.309+0000 7f8655c00640 1 --2- 172.21.15.145:0/241959549 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f86300762d0 0x7f8630078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:12.309 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.309+0000 7f8655c00640 1 --2- 172.21.15.145:0/241959549 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f86300762d0 0x7f8630078790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8644034000 tx=0x7f864402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:12.313 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.313+0000 7f864d600640 1 -- 172.21.15.145:0/241959549 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f86480604e0 con 0x7f8650146970 2024-05-09T04:35:12.520 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:12.520+0000 7f8657000640 1 -- 172.21.15.145:0/241959549 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f865015c1b0 con 0x7f8650146970 2024-05-09T04:35:13.081 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.081+0000 7f864d600640 1 -- 172.21.15.145:0/241959549 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v38) ==== 169+0+0 (secure 0 0 0) 0x7f864805fde0 con 0x7f8650146970 2024-05-09T04:35:13.081 INFO:teuthology.orchestra.run.smithi145.stderr:pool 'cephfs_data' removed 2024-05-09T04:35:13.085 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.085+0000 7f8657000640 1 -- 172.21.15.145:0/241959549 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f86300762d0 msgr2=0x7f8630078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:13.085 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.085+0000 7f8657000640 1 --2- 172.21.15.145:0/241959549 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f86300762d0 0x7f8630078790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8644034000 tx=0x7f864402e000 comp rx=0 tx=0).stop 2024-05-09T04:35:13.085 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.085+0000 7f8657000640 1 -- 172.21.15.145:0/241959549 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8650146970 msgr2=0x7f865013ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:13.085 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.085+0000 7f8657000640 1 --2- 172.21.15.145:0/241959549 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8650146970 0x7f865013ec10 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f864800c7a0 tx=0x7f864800cc60 comp rx=0 tx=0).stop 2024-05-09T04:35:13.085 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.085+0000 7f8657000640 1 -- 172.21.15.145:0/241959549 shutdown_connections 2024-05-09T04:35:13.086 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.085+0000 7f8657000640 1 --2- 172.21.15.145:0/241959549 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f86300762d0 0x7f8630078790 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:13.086 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.085+0000 7f8657000640 1 --2- 172.21.15.145:0/241959549 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8650146970 0x7f865013ec10 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:13.086 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.085+0000 7f8657000640 1 --2- 172.21.15.145:0/241959549 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f86500cd550 0x7f86501f2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:13.086 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.085+0000 7f8657000640 1 --2- 172.21.15.145:0/241959549 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f865006e060 0x7f8650144ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:13.086 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.085+0000 7f8657000640 1 -- 172.21.15.145:0/241959549 >> 172.21.15.145:0/241959549 conn(0x7f86500b2850 msgr2=0x7f865015b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:13.086 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.086+0000 7f8657000640 1 -- 172.21.15.145:0/241959549 shutdown_connections 2024-05-09T04:35:13.086 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.086+0000 7f8657000640 1 -- 172.21.15.145:0/241959549 wait complete. 2024-05-09T04:35:13.113 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm data_new data_new --yes-i-really-really-mean-it 2024-05-09T04:35:13.307 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.307+0000 7f8744c00640 1 Processor -- start 2024-05-09T04:35:13.308 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.308+0000 7f8744c00640 1 -- start start 2024-05-09T04:35:13.309 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.309+0000 7f8744c00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f874015f340 0x7f87400bab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:13.309 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.309+0000 7f8744c00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f87400d1bf0 0x7f87400d2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:13.309 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.309+0000 7f8744c00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87400b9920 con 0x7f87400d3360 2024-05-09T04:35:13.309 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.309+0000 7f8744c00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f87400b9c00 con 0x7f87400d1bf0 2024-05-09T04:35:13.309 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.309+0000 7f8744c00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f87400b9ee0 con 0x7f874015f340 2024-05-09T04:35:13.309 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.309+0000 7f873d400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f87400d1bf0 0x7f87400d2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:13.310 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.309+0000 7f873e800640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f87400d3360 0x7f87400d3790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:35500/0 (socket says 172.21.15.145:35500) 2024-05-09T04:35:13.310 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.309+0000 7f873e800640 1 -- 172.21.15.145:0/3805426818 learned_addr learned my addr 172.21.15.145:0/3805426818 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:35:13.310 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.310+0000 7f873de00640 1 --2- 172.21.15.145:0/3805426818 >> v2:172.21.15.192:3300/0 conn(0x7f874015f340 0x7f87400bab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:13.310 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.310+0000 7f873d400640 1 -- 172.21.15.145:0/3805426818 >> v2:172.21.15.192:3300/0 conn(0x7f874015f340 msgr2=0x7f87400bab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:13.311 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.310+0000 7f873d400640 1 --2- 172.21.15.145:0/3805426818 >> v2:172.21.15.192:3300/0 conn(0x7f874015f340 0x7f87400bab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:13.311 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.310+0000 7f873d400640 1 -- 172.21.15.145:0/3805426818 >> v1:172.21.15.145:6789/0 conn(0x7f87400d3360 legacy=0x7f87400d3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:13.311 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.310+0000 7f873d400640 1 -- 172.21.15.145:0/3805426818 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87400ba1c0 con 0x7f87400d1bf0 2024-05-09T04:35:13.311 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.310+0000 7f873ca00640 1 -- 172.21.15.145:0/3805426818 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2045638283 0 0) 0x7f87400b9920 con 0x7f87400d3360 2024-05-09T04:35:13.311 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.311+0000 7f873d400640 1 --2- 172.21.15.145:0/3805426818 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f87400d1bf0 0x7f87400d2080 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f87400d1bc0 tx=0x7f872c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=942804534dcae18 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:13.312 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.311+0000 7f873ca00640 1 -- 172.21.15.145:0/3805426818 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f872c037070 con 0x7f87400d1bf0 2024-05-09T04:35:13.312 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.311+0000 7f873ca00640 1 -- 172.21.15.145:0/3805426818 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f872c032440 con 0x7f87400d1bf0 2024-05-09T04:35:13.312 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.311+0000 7f873ca00640 1 -- 172.21.15.145:0/3805426818 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f872c032810 con 0x7f87400d1bf0 2024-05-09T04:35:13.312 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.312+0000 7f8744c00640 1 -- 172.21.15.145:0/3805426818 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f87400d1bf0 msgr2=0x7f87400d2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:13.312 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.312+0000 7f8744c00640 1 --2- 172.21.15.145:0/3805426818 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f87400d1bf0 0x7f87400d2080 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f87400d1bc0 tx=0x7f872c02a1a0 comp rx=0 tx=0).stop 2024-05-09T04:35:13.312 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.312+0000 7f8744c00640 1 -- 172.21.15.145:0/3805426818 shutdown_connections 2024-05-09T04:35:13.313 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.312+0000 7f8744c00640 1 --2- 172.21.15.145:0/3805426818 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f87400d1bf0 0x7f87400d2080 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:13.313 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.312+0000 7f8744c00640 1 --2- 172.21.15.145:0/3805426818 >> v2:172.21.15.192:3300/0 conn(0x7f874015f340 0x7f87400bab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:13.313 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.312+0000 7f8744c00640 1 -- 172.21.15.145:0/3805426818 >> 172.21.15.145:0/3805426818 conn(0x7f87400685e0 msgr2=0x7f87400689f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:13.313 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.312+0000 7f8744c00640 1 -- 172.21.15.145:0/3805426818 shutdown_connections 2024-05-09T04:35:13.313 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.312+0000 7f8744c00640 1 -- 172.21.15.145:0/3805426818 wait complete. 2024-05-09T04:35:13.313 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.314+0000 7f8744c00640 1 Processor -- start 2024-05-09T04:35:13.313 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.314+0000 7f8744c00640 1 -- start start 2024-05-09T04:35:13.316 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.315+0000 7f8744c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87400d1bf0 0x7f87400ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:13.316 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.315+0000 7f8744c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f87400d3360 0x7f874015c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:13.316 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.315+0000 7f8744c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f874015f340 0x7f874015cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:13.316 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.315+0000 7f8744c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f874015f720 con 0x7f874015f340 2024-05-09T04:35:13.316 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.315+0000 7f8744c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f87400bb040 con 0x7f87400d1bf0 2024-05-09T04:35:13.316 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.315+0000 7f8744c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f874015f8a0 con 0x7f87400d3360 2024-05-09T04:35:13.316 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.315+0000 7f873e800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f874015f340 0x7f874015cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:13.316 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.315+0000 7f873e800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f874015f340 0x7f874015cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:43246/0 (socket says 172.21.15.145:43246) 2024-05-09T04:35:13.316 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.315+0000 7f873e800640 1 -- 172.21.15.145:0/4041754886 learned_addr learned my addr 172.21.15.145:0/4041754886 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:13.316 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.315+0000 7f873de00640 1 --2- 172.21.15.145:0/4041754886 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87400d1bf0 0x7f87400ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:13.317 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.315+0000 7f873d400640 1 --2- 172.21.15.145:0/4041754886 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f87400d3360 0x7f874015c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:13.317 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.315+0000 7f873e800640 1 -- 172.21.15.145:0/4041754886 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f87400d3360 msgr2=0x7f874015c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:13.317 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.315+0000 7f873e800640 1 --2- 172.21.15.145:0/4041754886 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f87400d3360 0x7f874015c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:13.317 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.315+0000 7f873e800640 1 -- 172.21.15.145:0/4041754886 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87400d1bf0 msgr2=0x7f87400ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:13.317 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.315+0000 7f873e800640 1 --2- 172.21.15.145:0/4041754886 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87400d1bf0 0x7f87400ceb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:13.317 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.315+0000 7f873e800640 1 -- 172.21.15.145:0/4041754886 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87400cb730 con 0x7f874015f340 2024-05-09T04:35:13.317 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.316+0000 7f873de00640 1 --2- 172.21.15.145:0/4041754886 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87400d1bf0 0x7f87400ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:35:13.317 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.316+0000 7f873e800640 1 --2- 172.21.15.145:0/4041754886 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f874015f340 0x7f874015cad0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f873800c7a0 tx=0x7f873800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:13.317 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.316+0000 7f8736000640 1 -- 172.21.15.145:0/4041754886 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8738018070 con 0x7f874015f340 2024-05-09T04:35:13.317 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.316+0000 7f8736000640 1 -- 172.21.15.145:0/4041754886 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f873800a5e0 con 0x7f874015f340 2024-05-09T04:35:13.318 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.316+0000 7f8736000640 1 -- 172.21.15.145:0/4041754886 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f873800aa10 con 0x7f874015f340 2024-05-09T04:35:13.318 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.316+0000 7f8744c00640 1 -- 172.21.15.145:0/4041754886 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87400cba10 con 0x7f874015f340 2024-05-09T04:35:13.318 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.316+0000 7f8744c00640 1 -- 172.21.15.145:0/4041754886 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87400ce310 con 0x7f874015f340 2024-05-09T04:35:13.318 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.318+0000 7f8744c00640 1 -- 172.21.15.145:0/4041754886 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87400694e0 con 0x7f874015f340 2024-05-09T04:35:13.318 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.318+0000 7f8736000640 1 -- 172.21.15.145:0/4041754886 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f873801b070 con 0x7f874015f340 2024-05-09T04:35:13.319 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.319+0000 7f8736000640 1 --2- 172.21.15.145:0/4041754886 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f87200762d0 0x7f8720078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:13.319 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.320+0000 7f8736000640 1 -- 172.21.15.145:0/4041754886 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5338+0+0 (secure 0 0 0) 0x7f8738096b00 con 0x7f874015f340 2024-05-09T04:35:13.319 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.320+0000 7f873de00640 1 --2- 172.21.15.145:0/4041754886 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f87200762d0 0x7f8720078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:13.320 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.321+0000 7f873de00640 1 --2- 172.21.15.145:0/4041754886 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f87200762d0 0x7f8720078790 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f873000b820 tx=0x7f8730008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:13.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.324+0000 7f8736000640 1 -- 172.21.15.145:0/4041754886 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8738060530 con 0x7f874015f340 2024-05-09T04:35:13.533 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:13.533+0000 7f8744c00640 1 -- 172.21.15.145:0/4041754886 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "data_new", "pool2": "data_new", "yes_i_really_really_mean_it": true} v 0) -- 0x7f87400c88c0 con 0x7f874015f340 2024-05-09T04:35:14.109 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.109+0000 7f8736000640 1 -- 172.21.15.145:0/4041754886 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "data_new", "pool2": "data_new", "yes_i_really_really_mean_it": true}]=0 pool 'data_new' removed v39) ==== 160+0+0 (secure 0 0 0) 0x7f873805fe30 con 0x7f874015f340 2024-05-09T04:35:14.110 INFO:teuthology.orchestra.run.smithi145.stderr:pool 'data_new' removed 2024-05-09T04:35:14.114 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.114+0000 7f8744c00640 1 -- 172.21.15.145:0/4041754886 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f87200762d0 msgr2=0x7f8720078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:14.114 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.114+0000 7f8744c00640 1 --2- 172.21.15.145:0/4041754886 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f87200762d0 0x7f8720078790 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f873000b820 tx=0x7f8730008040 comp rx=0 tx=0).stop 2024-05-09T04:35:14.114 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.114+0000 7f8744c00640 1 -- 172.21.15.145:0/4041754886 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f874015f340 msgr2=0x7f874015cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:14.114 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.114+0000 7f8744c00640 1 --2- 172.21.15.145:0/4041754886 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f874015f340 0x7f874015cad0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f873800c7a0 tx=0x7f873800cc60 comp rx=0 tx=0).stop 2024-05-09T04:35:14.114 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.114+0000 7f8744c00640 1 -- 172.21.15.145:0/4041754886 shutdown_connections 2024-05-09T04:35:14.114 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.114+0000 7f8744c00640 1 --2- 172.21.15.145:0/4041754886 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f87200762d0 0x7f8720078790 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:14.114 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.114+0000 7f8744c00640 1 --2- 172.21.15.145:0/4041754886 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f874015f340 0x7f874015cad0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:14.114 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.114+0000 7f8744c00640 1 --2- 172.21.15.145:0/4041754886 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f87400d3360 0x7f874015c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:14.114 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.114+0000 7f8744c00640 1 --2- 172.21.15.145:0/4041754886 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87400d1bf0 0x7f87400ceb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:14.115 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.114+0000 7f8744c00640 1 -- 172.21.15.145:0/4041754886 >> 172.21.15.145:0/4041754886 conn(0x7f87400685e0 msgr2=0x7f874015d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:14.115 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.114+0000 7f8744c00640 1 -- 172.21.15.145:0/4041754886 shutdown_connections 2024-05-09T04:35:14.115 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.114+0000 7f8744c00640 1 -- 172.21.15.145:0/4041754886 wait complete. 2024-05-09T04:35:14.138 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_symlink_recovery 2024-05-09T04:35:14.328 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.328+0000 7f5a1ac00640 1 Processor -- start 2024-05-09T04:35:14.328 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.328+0000 7f5a1ac00640 1 -- start start 2024-05-09T04:35:14.330 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.330+0000 7f5a1ac00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f5a1415f4e0 0x7f5a1416a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:14.330 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.330+0000 7f5a1ac00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a140cfda0 con 0x7f5a140d3cd0 2024-05-09T04:35:14.330 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.330+0000 7f5a1ac00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a140d0080 con 0x7f5a1415f100 2024-05-09T04:35:14.330 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.330+0000 7f5a1ac00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f5a140b8e70 con 0x7f5a1415f4e0 2024-05-09T04:35:14.330 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.330+0000 7f5a18e00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f5a140d3cd0 0x7f5a140d1950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:35512/0 (socket says 172.21.15.145:35512) 2024-05-09T04:35:14.330 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.330+0000 7f5a18e00640 1 -- 172.21.15.145:0/1520906233 learned_addr learned my addr 172.21.15.145:0/1520906233 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:35:14.332 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.331+0000 7f5a1a200640 1 --2- 172.21.15.145:0/1520906233 >> v2:172.21.15.192:3300/0 conn(0x7f5a1415f4e0 0x7f5a1416a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:14.332 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.331+0000 7f5a13e00640 1 -- 172.21.15.145:0/1520906233 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1092538211 0 0) 0x7f5a140cfda0 con 0x7f5a140d3cd0 2024-05-09T04:35:14.332 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.331+0000 7f5a13e00640 1 -- 172.21.15.145:0/1520906233 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f59fc003520 con 0x7f5a140d3cd0 2024-05-09T04:35:14.332 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.331+0000 7f5a13e00640 1 -- 172.21.15.145:0/1520906233 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3132847626 0 0) 0x7f59fc003520 con 0x7f5a140d3cd0 2024-05-09T04:35:14.332 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.331+0000 7f5a13e00640 1 -- 172.21.15.145:0/1520906233 >> v2:172.21.15.192:3300/0 conn(0x7f5a1415f4e0 msgr2=0x7f5a1416a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:14.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.331+0000 7f5a13e00640 1 --2- 172.21.15.145:0/1520906233 >> v2:172.21.15.192:3300/0 conn(0x7f5a1415f4e0 0x7f5a1416a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:14.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.331+0000 7f5a13e00640 1 -- 172.21.15.145:0/1520906233 >> v1:172.21.15.192:6789/0 conn(0x7f5a1415f100 legacy=0x7f5a140d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:14.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.331+0000 7f5a13e00640 1 -- 172.21.15.145:0/1520906233 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a140d0360 con 0x7f5a140d3cd0 2024-05-09T04:35:14.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.331+0000 7f5a13e00640 1 -- 172.21.15.145:0/1520906233 <== mon.0 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3474413261 0 0) 0x7f5a000016f0 con 0x7f5a140d3cd0 2024-05-09T04:35:14.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.331+0000 7f5a13e00640 1 -- 172.21.15.145:0/1520906233 >> v1:172.21.15.145:6789/0 conn(0x7f5a140d3cd0 legacy=0x7f5a140d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:14.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.331+0000 7f5a13e00640 1 --2- 172.21.15.145:0/1520906233 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f59fc003ad0 0x7f59fc003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:14.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.331+0000 7f5a13e00640 1 --2- 172.21.15.145:0/1520906233 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f59fc004800 0x7f59fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:14.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.331+0000 7f5a13e00640 1 --2- 172.21.15.145:0/1520906233 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f59fc005230 0x7f59fc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:14.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.331+0000 7f5a13e00640 1 -- 172.21.15.145:0/1520906233 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5a140b8e70 con 0x7f59fc005230 2024-05-09T04:35:14.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.331+0000 7f5a13e00640 1 -- 172.21.15.145:0/1520906233 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f59fc00c190 con 0x7f59fc003ad0 2024-05-09T04:35:14.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.331+0000 7f5a13e00640 1 -- 172.21.15.145:0/1520906233 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f59fc00c390 con 0x7f59fc004800 2024-05-09T04:35:14.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.332+0000 7f5a1a200640 1 --2- 172.21.15.145:0/1520906233 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f59fc005230 0x7f59fc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:14.334 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.332+0000 7f5a19800640 1 --2- 172.21.15.145:0/1520906233 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f59fc003ad0 0x7f59fc003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:14.334 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.332+0000 7f5a18e00640 1 --2- 172.21.15.145:0/1520906233 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f59fc004800 0x7f59fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:14.334 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.332+0000 7f5a1a200640 1 -- 172.21.15.145:0/1520906233 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f59fc004800 msgr2=0x7f59fc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:14.334 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.332+0000 7f5a1a200640 1 --2- 172.21.15.145:0/1520906233 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f59fc004800 0x7f59fc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:14.334 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.332+0000 7f5a1a200640 1 -- 172.21.15.145:0/1520906233 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f59fc003ad0 msgr2=0x7f59fc003ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:14.334 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.332+0000 7f5a1a200640 1 --2- 172.21.15.145:0/1520906233 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f59fc003ad0 0x7f59fc003ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:14.334 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.332+0000 7f5a1a200640 1 -- 172.21.15.145:0/1520906233 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59fc003520 con 0x7f59fc005230 2024-05-09T04:35:14.334 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.332+0000 7f5a19800640 1 --2- 172.21.15.145:0/1520906233 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f59fc003ad0 0x7f59fc003ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:35:14.336 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.333+0000 7f5a1a200640 1 -- 172.21.15.145:0/1520906233 reap_dead start 2024-05-09T04:35:14.336 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.333+0000 7f5a1a200640 1 --2- 172.21.15.145:0/1520906233 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f59fc005230 0x7f59fc00b9c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c009f90 tx=0x7f59fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba0dacd3c7673d51 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:14.336 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.334+0000 7f5a13e00640 1 -- 172.21.15.145:0/1520906233 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c035020 con 0x7f59fc005230 2024-05-09T04:35:14.336 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.334+0000 7f5a13e00640 1 -- 172.21.15.145:0/1520906233 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a0c003b90 con 0x7f59fc005230 2024-05-09T04:35:14.336 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.334+0000 7f5a13e00640 1 -- 172.21.15.145:0/1520906233 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c009c10 con 0x7f59fc005230 2024-05-09T04:35:14.336 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.334+0000 7f5a1ac00640 1 -- 172.21.15.145:0/1520906233 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f59fc005230 msgr2=0x7f59fc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:14.336 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.334+0000 7f5a1ac00640 1 --2- 172.21.15.145:0/1520906233 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f59fc005230 0x7f59fc00b9c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c009f90 tx=0x7f59fc005830 comp rx=0 tx=0).stop 2024-05-09T04:35:14.336 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.334+0000 7f5a1ac00640 1 -- 172.21.15.145:0/1520906233 shutdown_connections 2024-05-09T04:35:14.336 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.334+0000 7f5a1ac00640 1 --2- 172.21.15.145:0/1520906233 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f59fc005230 0x7f59fc00b9c0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:14.336 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.334+0000 7f5a1ac00640 1 -- 172.21.15.145:0/1520906233 >> 172.21.15.145:0/1520906233 conn(0x7f5a1409e2e0 msgr2=0x7f5a140d5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:14.336 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.334+0000 7f5a1ac00640 1 -- 172.21.15.145:0/1520906233 shutdown_connections 2024-05-09T04:35:14.336 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.334+0000 7f5a1ac00640 1 -- 172.21.15.145:0/1520906233 wait complete. 2024-05-09T04:35:14.336 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.336+0000 7f5a1ac00640 1 Processor -- start 2024-05-09T04:35:14.337 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.336+0000 7f5a1ac00640 1 -- start start 2024-05-09T04:35:14.338 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.337+0000 7f5a1ac00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f59fc005230 0x7f5a14153c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:14.339 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.337+0000 7f5a1ac00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a1415fa10 0x7f5a14151200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:14.339 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.337+0000 7f5a1ac00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a14162220 0x7f5a141f3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:14.339 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.337+0000 7f5a1ac00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5a140d0e40 con 0x7f5a1415fa10 2024-05-09T04:35:14.339 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.337+0000 7f5a1ac00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5a140d0920 con 0x7f5a14162220 2024-05-09T04:35:14.339 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.337+0000 7f5a1ac00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f5a14160180 con 0x7f59fc005230 2024-05-09T04:35:14.339 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.338+0000 7f5a18e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a1415fa10 0x7f5a14151200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:14.339 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.338+0000 7f5a18e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a1415fa10 0x7f5a14151200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:43266/0 (socket says 172.21.15.145:43266) 2024-05-09T04:35:14.339 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.338+0000 7f5a18e00640 1 -- 172.21.15.145:0/3513121690 learned_addr learned my addr 172.21.15.145:0/3513121690 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:14.339 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.338+0000 7f5a19800640 1 --2- 172.21.15.145:0/3513121690 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f59fc005230 0x7f5a14153c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:14.339 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.338+0000 7f5a1a200640 1 --2- 172.21.15.145:0/3513121690 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a14162220 0x7f5a141f3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:14.339 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.338+0000 7f5a18e00640 1 -- 172.21.15.145:0/3513121690 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f59fc005230 msgr2=0x7f5a14153c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:14.339 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.338+0000 7f5a18e00640 1 --2- 172.21.15.145:0/3513121690 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f59fc005230 0x7f5a14153c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:14.339 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.338+0000 7f5a18e00640 1 -- 172.21.15.145:0/3513121690 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a14162220 msgr2=0x7f5a141f3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:14.340 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.338+0000 7f5a18e00640 1 --2- 172.21.15.145:0/3513121690 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a14162220 0x7f5a141f3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:14.340 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.338+0000 7f5a18e00640 1 -- 172.21.15.145:0/3513121690 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a14160460 con 0x7f5a1415fa10 2024-05-09T04:35:14.340 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.338+0000 7f5a18e00640 1 --2- 172.21.15.145:0/3513121690 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a1415fa10 0x7f5a14151200 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5a0000e700 tx=0x7f5a0000ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:14.340 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.339+0000 7f5a11600640 1 -- 172.21.15.145:0/3513121690 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0000c6c0 con 0x7f5a1415fa10 2024-05-09T04:35:14.340 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.339+0000 7f5a11600640 1 -- 172.21.15.145:0/3513121690 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a00018070 con 0x7f5a1415fa10 2024-05-09T04:35:14.340 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.339+0000 7f5a11600640 1 -- 172.21.15.145:0/3513121690 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0000cac0 con 0x7f5a1415fa10 2024-05-09T04:35:14.340 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.339+0000 7f5a1ac00640 1 -- 172.21.15.145:0/3513121690 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a14160740 con 0x7f5a1415fa10 2024-05-09T04:35:14.340 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.339+0000 7f5a1ac00640 1 -- 172.21.15.145:0/3513121690 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a14160d00 con 0x7f5a1415fa10 2024-05-09T04:35:14.340 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.340+0000 7f5a1ac00640 1 -- 172.21.15.145:0/3513121690 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a1406a310 con 0x7f5a1415fa10 2024-05-09T04:35:14.341 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.341+0000 7f5a11600640 1 -- 172.21.15.145:0/3513121690 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5a0000cc60 con 0x7f5a1415fa10 2024-05-09T04:35:14.342 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.342+0000 7f5a11600640 1 --2- 172.21.15.145:0/3513121690 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f59f40762d0 0x7f59f4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:14.342 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.342+0000 7f5a11600640 1 -- 172.21.15.145:0/3513121690 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4943+0+0 (secure 0 0 0) 0x7f5a000970f0 con 0x7f5a1415fa10 2024-05-09T04:35:14.342 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.342+0000 7f5a19800640 1 --2- 172.21.15.145:0/3513121690 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f59f40762d0 0x7f59f4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:14.343 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.343+0000 7f5a19800640 1 --2- 172.21.15.145:0/3513121690 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f59f40762d0 0x7f59f4078790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5a08000c00 tx=0x7f5a08009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:14.346 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.347+0000 7f5a11600640 1 -- 172.21.15.145:0/3513121690 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5a00060d40 con 0x7f5a1415fa10 2024-05-09T04:35:14.553 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.553+0000 7f5a1ac00640 1 -- 172.21.15.145:0/3513121690 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_symlink_recovery"} v 0) -- 0x7f5a140d0fc0 con 0x7f5a1415fa10 2024-05-09T04:35:14.559 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.559+0000 7f5a11600640 1 -- 172.21.15.145:0/3513121690 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_symlink_recovery"}]=0 v13) ==== 103+0+0 (secure 0 0 0) 0x7f5a00060640 con 0x7f5a1415fa10 2024-05-09T04:35:14.563 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.562+0000 7f5a1ac00640 1 -- 172.21.15.145:0/3513121690 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f59f40762d0 msgr2=0x7f59f4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:14.563 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.562+0000 7f5a1ac00640 1 --2- 172.21.15.145:0/3513121690 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f59f40762d0 0x7f59f4078790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5a08000c00 tx=0x7f5a08009040 comp rx=0 tx=0).stop 2024-05-09T04:35:14.563 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.562+0000 7f5a1ac00640 1 -- 172.21.15.145:0/3513121690 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a1415fa10 msgr2=0x7f5a14151200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:14.563 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.562+0000 7f5a1ac00640 1 --2- 172.21.15.145:0/3513121690 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a1415fa10 0x7f5a14151200 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5a0000e700 tx=0x7f5a0000ebc0 comp rx=0 tx=0).stop 2024-05-09T04:35:14.563 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.563+0000 7f5a1ac00640 1 -- 172.21.15.145:0/3513121690 shutdown_connections 2024-05-09T04:35:14.563 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.563+0000 7f5a1ac00640 1 --2- 172.21.15.145:0/3513121690 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f59f40762d0 0x7f59f4078790 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:14.563 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.563+0000 7f5a1ac00640 1 --2- 172.21.15.145:0/3513121690 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a14162220 0x7f5a141f3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:14.563 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.563+0000 7f5a1ac00640 1 --2- 172.21.15.145:0/3513121690 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5a1415fa10 0x7f5a14151200 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:14.563 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.563+0000 7f5a1ac00640 1 --2- 172.21.15.145:0/3513121690 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f59fc005230 0x7f5a14153c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:14.563 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.563+0000 7f5a1ac00640 1 -- 172.21.15.145:0/3513121690 >> 172.21.15.145:0/3513121690 conn(0x7f5a1409e2e0 msgr2=0x7f5a140b8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:14.564 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.563+0000 7f5a1ac00640 1 -- 172.21.15.145:0/3513121690 shutdown_connections 2024-05-09T04:35:14.564 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.563+0000 7f5a1ac00640 1 -- 172.21.15.145:0/3513121690 wait complete. 2024-05-09T04:35:14.583 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-09T04:35:14.784 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.784+0000 7f8416e00640 1 Processor -- start 2024-05-09T04:35:14.785 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.785+0000 7f8416e00640 1 -- start start 2024-05-09T04:35:14.786 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.786+0000 7f8416e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f841015bd80 0x7f8410072140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:14.786 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.786+0000 7f8416e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f84100b4600 0x7f841006fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:14.786 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.786+0000 7f8416e00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84100b5e80 con 0x7f8410070020 2024-05-09T04:35:14.786 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.786+0000 7f8416e00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f8410070490 con 0x7f841015bd80 2024-05-09T04:35:14.786 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.786+0000 7f8416e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f84100ae690 con 0x7f84100b4600 2024-05-09T04:35:14.786 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.787+0000 7f8416400640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f8410070020 0x7f84100aeda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:35524/0 (socket says 172.21.15.145:35524) 2024-05-09T04:35:14.787 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.787+0000 7f8415a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f841015bd80 0x7f8410072140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:14.787 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.787+0000 7f8416400640 1 -- 172.21.15.145:0/523824244 learned_addr learned my addr 172.21.15.145:0/523824244 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:35:14.787 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.787+0000 7f8415a00640 1 -- 172.21.15.145:0/523824244 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f84100b4600 msgr2=0x7f841006fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:14.787 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.787+0000 7f8415000640 1 --2- 172.21.15.145:0/523824244 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f84100b4600 0x7f841006fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:14.788 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.787+0000 7f8415a00640 1 --2- 172.21.15.145:0/523824244 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f84100b4600 0x7f841006fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:14.788 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.787+0000 7f8415a00640 1 -- 172.21.15.145:0/523824244 >> v1:172.21.15.145:6789/0 conn(0x7f8410070020 legacy=0x7f84100aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:14.788 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.787+0000 7f840fe00640 1 -- 172.21.15.145:0/523824244 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 242235439 0 0) 0x7f84100b5e80 con 0x7f8410070020 2024-05-09T04:35:14.788 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.787+0000 7f8415a00640 1 -- 172.21.15.145:0/523824244 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84100b0630 con 0x7f841015bd80 2024-05-09T04:35:14.789 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.788+0000 7f8415a00640 1 --2- 172.21.15.145:0/523824244 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f841015bd80 0x7f8410072140 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f84040097a0 tx=0x7f840402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=504b16cf2261ddcb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:14.789 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.788+0000 7f840fe00640 1 -- 172.21.15.145:0/523824244 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8404037070 con 0x7f841015bd80 2024-05-09T04:35:14.789 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.789+0000 7f840fe00640 1 -- 172.21.15.145:0/523824244 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8404032440 con 0x7f841015bd80 2024-05-09T04:35:14.789 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.789+0000 7f840fe00640 1 -- 172.21.15.145:0/523824244 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8404032810 con 0x7f841015bd80 2024-05-09T04:35:14.789 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.789+0000 7f8416e00640 1 -- 172.21.15.145:0/523824244 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f841015bd80 msgr2=0x7f8410072140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:14.790 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.789+0000 7f8416e00640 1 --2- 172.21.15.145:0/523824244 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f841015bd80 0x7f8410072140 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f84040097a0 tx=0x7f840402a180 comp rx=0 tx=0).stop 2024-05-09T04:35:14.790 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.789+0000 7f8416e00640 1 -- 172.21.15.145:0/523824244 shutdown_connections 2024-05-09T04:35:14.790 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.789+0000 7f8416e00640 1 --2- 172.21.15.145:0/523824244 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f84100b4600 0x7f841006fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:14.790 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.789+0000 7f8416e00640 1 --2- 172.21.15.145:0/523824244 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f841015bd80 0x7f8410072140 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:14.790 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.789+0000 7f8416e00640 1 -- 172.21.15.145:0/523824244 >> 172.21.15.145:0/523824244 conn(0x7f84100b28f0 msgr2=0x7f84100b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:14.790 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.789+0000 7f8416e00640 1 -- 172.21.15.145:0/523824244 shutdown_connections 2024-05-09T04:35:14.790 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.789+0000 7f8416e00640 1 -- 172.21.15.145:0/523824244 wait complete. 2024-05-09T04:35:14.791 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.791+0000 7f8416e00640 1 Processor -- start 2024-05-09T04:35:14.791 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.791+0000 7f8416e00640 1 -- start start 2024-05-09T04:35:14.792 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.792+0000 7f8416e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8410070020 0x7f84100cad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:14.792 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.792+0000 7f8416e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84100b4600 0x7f84100cbe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:14.792 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.792+0000 7f8416e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f841015bd80 0x7f84100cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:14.792 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.792+0000 7f8416e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f84100b58c0 con 0x7f8410070020 2024-05-09T04:35:14.792 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.792+0000 7f8416e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f84100ae8c0 con 0x7f84100b4600 2024-05-09T04:35:14.792 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.792+0000 7f8416e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f84100b5ba0 con 0x7f841015bd80 2024-05-09T04:35:14.792 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.792+0000 7f8415a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8410070020 0x7f84100cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:14.793 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.793+0000 7f8415a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8410070020 0x7f84100cad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:43278/0 (socket says 172.21.15.145:43278) 2024-05-09T04:35:14.793 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.793+0000 7f8415a00640 1 -- 172.21.15.145:0/1800279597 learned_addr learned my addr 172.21.15.145:0/1800279597 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:14.793 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.793+0000 7f8415a00640 1 -- 172.21.15.145:0/1800279597 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f841015bd80 msgr2=0x7f84100cc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:14.794 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.793+0000 7f8415a00640 1 --2- 172.21.15.145:0/1800279597 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f841015bd80 0x7f84100cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:14.794 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.793+0000 7f8415a00640 1 -- 172.21.15.145:0/1800279597 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84100b4600 msgr2=0x7f84100cbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:14.794 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.793+0000 7f8415a00640 1 --2- 172.21.15.145:0/1800279597 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84100b4600 0x7f84100cbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:14.794 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.793+0000 7f8415a00640 1 -- 172.21.15.145:0/1800279597 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8410156300 con 0x7f8410070020 2024-05-09T04:35:14.794 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.793+0000 7f8415a00640 1 --2- 172.21.15.145:0/1800279597 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8410070020 0x7f84100cad10 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f84040097a0 tx=0x7f84040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:14.794 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.794+0000 7f840d600640 1 -- 172.21.15.145:0/1800279597 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8404037050 con 0x7f8410070020 2024-05-09T04:35:14.794 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.794+0000 7f840d600640 1 -- 172.21.15.145:0/1800279597 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8404038040 con 0x7f8410070020 2024-05-09T04:35:14.794 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.794+0000 7f840d600640 1 -- 172.21.15.145:0/1800279597 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8404004490 con 0x7f8410070020 2024-05-09T04:35:14.794 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.794+0000 7f8416e00640 1 -- 172.21.15.145:0/1800279597 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f841015c720 con 0x7f8410070020 2024-05-09T04:35:14.794 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.794+0000 7f8416e00640 1 -- 172.21.15.145:0/1800279597 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f841015cce0 con 0x7f8410070020 2024-05-09T04:35:14.795 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.795+0000 7f8416e00640 1 -- 172.21.15.145:0/1800279597 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f841006a310 con 0x7f8410070020 2024-05-09T04:35:14.796 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.796+0000 7f840d600640 1 -- 172.21.15.145:0/1800279597 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8404030980 con 0x7f8410070020 2024-05-09T04:35:14.797 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.797+0000 7f840d600640 1 --2- 172.21.15.145:0/1800279597 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f83f00762d0 0x7f83f0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:14.797 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.797+0000 7f840d600640 1 -- 172.21.15.145:0/1800279597 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4943+0+0 (secure 0 0 0) 0x7f84040b7ed0 con 0x7f8410070020 2024-05-09T04:35:14.797 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.798+0000 7f8415000640 1 --2- 172.21.15.145:0/1800279597 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f83f00762d0 0x7f83f0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:14.798 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.798+0000 7f8415000640 1 --2- 172.21.15.145:0/1800279597 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f83f00762d0 0x7f83f0078790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f83fc006fd0 tx=0x7f83fc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:14.801 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:14.802+0000 7f840d600640 1 -- 172.21.15.145:0/1800279597 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8404081b20 con 0x7f8410070020 2024-05-09T04:35:15.014 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.014+0000 7f8416e00640 1 -- 172.21.15.145:0/1800279597 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f841006e2f0 con 0x7f8410070020 2024-05-09T04:35:15.019 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.019+0000 7f840d600640 1 -- 172.21.15.145:0/1800279597 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v14) ==== 104+0+0 (secure 0 0 0) 0x7f8404081420 con 0x7f8410070020 2024-05-09T04:35:15.022 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.022+0000 7f8416e00640 1 -- 172.21.15.145:0/1800279597 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f83f00762d0 msgr2=0x7f83f0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:15.023 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.022+0000 7f8416e00640 1 --2- 172.21.15.145:0/1800279597 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f83f00762d0 0x7f83f0078790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f83fc006fd0 tx=0x7f83fc008040 comp rx=0 tx=0).stop 2024-05-09T04:35:15.023 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.022+0000 7f8416e00640 1 -- 172.21.15.145:0/1800279597 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8410070020 msgr2=0x7f84100cad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:15.023 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.022+0000 7f8416e00640 1 --2- 172.21.15.145:0/1800279597 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8410070020 0x7f84100cad10 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f84040097a0 tx=0x7f84040046c0 comp rx=0 tx=0).stop 2024-05-09T04:35:15.023 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.023+0000 7f8416e00640 1 -- 172.21.15.145:0/1800279597 shutdown_connections 2024-05-09T04:35:15.023 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.023+0000 7f8416e00640 1 --2- 172.21.15.145:0/1800279597 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f83f00762d0 0x7f83f0078790 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:15.023 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.023+0000 7f8416e00640 1 --2- 172.21.15.145:0/1800279597 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f841015bd80 0x7f84100cc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:15.023 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.023+0000 7f8416e00640 1 --2- 172.21.15.145:0/1800279597 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84100b4600 0x7f84100cbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:15.023 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.023+0000 7f8416e00640 1 --2- 172.21.15.145:0/1800279597 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8410070020 0x7f84100cad10 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:15.023 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.023+0000 7f8416e00640 1 -- 172.21.15.145:0/1800279597 >> 172.21.15.145:0/1800279597 conn(0x7f84100b28f0 msgr2=0x7f84100cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:15.023 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.023+0000 7f8416e00640 1 -- 172.21.15.145:0/1800279597 shutdown_connections 2024-05-09T04:35:15.023 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.023+0000 7f8416e00640 1 -- 172.21.15.145:0/1800279597 wait complete. 2024-05-09T04:35:15.041 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_backtrace.TestBacktrace.test_backtrace' 2024-05-09T04:35:15.218 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.218+0000 7f6907e00640 1 Processor -- start 2024-05-09T04:35:15.219 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.219+0000 7f6907e00640 1 -- start start 2024-05-09T04:35:15.220 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.220+0000 7f6907e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f69080b0640 0x7f690806d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:15.220 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.220+0000 7f6907e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f690806d9d0 0x7f690806feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:15.220 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.220+0000 7f6907e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69080b56a0 con 0x7f690815efa0 2024-05-09T04:35:15.220 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.220+0000 7f6907e00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f690806de20 con 0x7f69080b0640 2024-05-09T04:35:15.220 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.220+0000 7f6907e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f69080b4e00 con 0x7f690806d9d0 2024-05-09T04:35:15.220 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.220+0000 7f6906000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f69080b0640 0x7f690806d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:15.220 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.221+0000 7f6906000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f69080b0640 0x7f690806d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:43284/0 (socket says 172.21.15.145:43284) 2024-05-09T04:35:15.221 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.221+0000 7f6906000640 1 -- 172.21.15.145:0/1836559736 learned_addr learned my addr 172.21.15.145:0/1836559736 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:15.221 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.221+0000 7f6906000640 1 -- 172.21.15.145:0/1836559736 >> v2:172.21.15.192:3300/0 conn(0x7f690806d9d0 msgr2=0x7f690806feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:15.221 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.221+0000 7f6906000640 1 --2- 172.21.15.145:0/1836559736 >> v2:172.21.15.192:3300/0 conn(0x7f690806d9d0 0x7f690806feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:15.221 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.221+0000 7f6906000640 1 -- 172.21.15.145:0/1836559736 >> v1:172.21.15.192:6789/0 conn(0x7f690815efa0 legacy=0x7f69080b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:15.221 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.221+0000 7f6906000640 1 -- 172.21.15.145:0/1836559736 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69080b50e0 con 0x7f69080b0640 2024-05-09T04:35:15.222 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.222+0000 7f6906000640 1 --2- 172.21.15.145:0/1836559736 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f69080b0640 0x7f690806d490 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f690806f050 tx=0x7f68f002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c527d249de21c077 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:15.222 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.222+0000 7f6905600640 1 -- 172.21.15.145:0/1836559736 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f0037070 con 0x7f69080b0640 2024-05-09T04:35:15.222 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.222+0000 7f6905600640 1 -- 172.21.15.145:0/1836559736 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68f0032440 con 0x7f69080b0640 2024-05-09T04:35:15.222 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.222+0000 7f6905600640 1 -- 172.21.15.145:0/1836559736 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f0032810 con 0x7f69080b0640 2024-05-09T04:35:15.223 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.223+0000 7f6907e00640 1 -- 172.21.15.145:0/1836559736 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f69080b0640 msgr2=0x7f690806d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:15.223 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.223+0000 7f6907e00640 1 --2- 172.21.15.145:0/1836559736 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f69080b0640 0x7f690806d490 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f690806f050 tx=0x7f68f002a1e0 comp rx=0 tx=0).stop 2024-05-09T04:35:15.223 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.223+0000 7f6907e00640 1 -- 172.21.15.145:0/1836559736 shutdown_connections 2024-05-09T04:35:15.223 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.223+0000 7f6907e00640 1 --2- 172.21.15.145:0/1836559736 >> v2:172.21.15.192:3300/0 conn(0x7f690806d9d0 0x7f690806feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:15.223 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.223+0000 7f6907e00640 1 --2- 172.21.15.145:0/1836559736 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f69080b0640 0x7f690806d490 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:15.223 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.223+0000 7f6907e00640 1 -- 172.21.15.145:0/1836559736 >> 172.21.15.145:0/1836559736 conn(0x7f69080729a0 msgr2=0x7f6908072db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:15.223 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.223+0000 7f6907e00640 1 -- 172.21.15.145:0/1836559736 shutdown_connections 2024-05-09T04:35:15.223 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.223+0000 7f6907e00640 1 -- 172.21.15.145:0/1836559736 wait complete. 2024-05-09T04:35:15.224 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.225+0000 7f6907e00640 1 Processor -- start 2024-05-09T04:35:15.224 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.225+0000 7f6907e00640 1 -- start start 2024-05-09T04:35:15.226 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.225+0000 7f6907e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f690806d9d0 0x7f6908144f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:15.226 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.226+0000 7f6907e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69080b0640 0x7f6908160aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:15.226 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.226+0000 7f6907e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f690815efa0 0x7f69081588a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:15.226 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.226+0000 7f6907e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f69080b0aa0 con 0x7f690815efa0 2024-05-09T04:35:15.226 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.226+0000 7f6907e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f690806f990 con 0x7f69080b0640 2024-05-09T04:35:15.226 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.226+0000 7f6907e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f690815f380 con 0x7f690806d9d0 2024-05-09T04:35:15.226 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.226+0000 7f6907400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f690815efa0 0x7f69081588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:15.226 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.226+0000 7f6907400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f690815efa0 0x7f69081588a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:43290/0 (socket says 172.21.15.145:43290) 2024-05-09T04:35:15.226 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.226+0000 7f6907400640 1 -- 172.21.15.145:0/2494886732 learned_addr learned my addr 172.21.15.145:0/2494886732 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:15.226 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.226+0000 7f6907400640 1 -- 172.21.15.145:0/2494886732 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f690806d9d0 msgr2=0x7f6908144f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:15.226 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.226+0000 7f6907400640 1 --2- 172.21.15.145:0/2494886732 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f690806d9d0 0x7f6908144f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:15.226 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.226+0000 7f6907400640 1 -- 172.21.15.145:0/2494886732 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69080b0640 msgr2=0x7f6908160aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:15.227 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.226+0000 7f6907400640 1 --2- 172.21.15.145:0/2494886732 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69080b0640 0x7f6908160aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:15.227 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.226+0000 7f6907400640 1 -- 172.21.15.145:0/2494886732 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f690815fa00 con 0x7f690815efa0 2024-05-09T04:35:15.228 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.226+0000 7f6907400640 1 --2- 172.21.15.145:0/2494886732 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f690815efa0 0x7f69081588a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f68f8009fd0 tx=0x7f68f800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:15.228 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.227+0000 7f68fea00640 1 -- 172.21.15.145:0/2494886732 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f8011830 con 0x7f690815efa0 2024-05-09T04:35:15.228 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.227+0000 7f68fea00640 1 -- 172.21.15.145:0/2494886732 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68f8016070 con 0x7f690815efa0 2024-05-09T04:35:15.228 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.227+0000 7f6907e00640 1 -- 172.21.15.145:0/2494886732 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f690815fce0 con 0x7f690815efa0 2024-05-09T04:35:15.228 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.227+0000 7f68fea00640 1 -- 172.21.15.145:0/2494886732 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f8011bd0 con 0x7f690815efa0 2024-05-09T04:35:15.228 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.227+0000 7f6907e00640 1 -- 172.21.15.145:0/2494886732 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69081602a0 con 0x7f690815efa0 2024-05-09T04:35:15.228 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.228+0000 7f6907e00640 1 -- 172.21.15.145:0/2494886732 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69080d1370 con 0x7f690815efa0 2024-05-09T04:35:15.228 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.229+0000 7f68fea00640 1 -- 172.21.15.145:0/2494886732 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f68f801b460 con 0x7f690815efa0 2024-05-09T04:35:15.230 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.230+0000 7f68fea00640 1 --2- 172.21.15.145:0/2494886732 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f68e40762d0 0x7f68e4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:15.230 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.230+0000 7f68fea00640 1 -- 172.21.15.145:0/2494886732 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4943+0+0 (secure 0 0 0) 0x7f68f8097000 con 0x7f690815efa0 2024-05-09T04:35:15.230 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.230+0000 7f6906a00640 1 --2- 172.21.15.145:0/2494886732 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f68e40762d0 0x7f68e4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:15.230 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.231+0000 7f6906a00640 1 --2- 172.21.15.145:0/2494886732 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f68e40762d0 0x7f68e4078790 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f690000abb0 tx=0x7f6900009250 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:15.234 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.234+0000 7f68fea00640 1 -- 172.21.15.145:0/2494886732 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f68f8060cf0 con 0x7f690815efa0 2024-05-09T04:35:15.516 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:15.516+0000 7f6907e00640 1 -- 172.21.15.145:0/2494886732 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_backtrace.TestBacktrace.test_backtrace"]} v 0) -- 0x7f69080694e0 con 0x7f690815efa0 2024-05-09T04:35:16.124 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.124+0000 7f68fea00640 1 -- 172.21.15.145:0/2494886732 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_backtrace.TestBacktrace.test_backtrace"]}]=0 v99) ==== 135+0+0 (secure 0 0 0) 0x7f68f80605f0 con 0x7f690815efa0 2024-05-09T04:35:16.128 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.128+0000 7f6907e00640 1 -- 172.21.15.145:0/2494886732 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f68e40762d0 msgr2=0x7f68e4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:16.128 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.128+0000 7f6907e00640 1 --2- 172.21.15.145:0/2494886732 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f68e40762d0 0x7f68e4078790 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f690000abb0 tx=0x7f6900009250 comp rx=0 tx=0).stop 2024-05-09T04:35:16.128 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.128+0000 7f6907e00640 1 -- 172.21.15.145:0/2494886732 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f690815efa0 msgr2=0x7f69081588a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:16.129 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.128+0000 7f6907e00640 1 --2- 172.21.15.145:0/2494886732 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f690815efa0 0x7f69081588a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f68f8009fd0 tx=0x7f68f800b8e0 comp rx=0 tx=0).stop 2024-05-09T04:35:16.129 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.128+0000 7f6907e00640 1 -- 172.21.15.145:0/2494886732 shutdown_connections 2024-05-09T04:35:16.129 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.128+0000 7f6907e00640 1 --2- 172.21.15.145:0/2494886732 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f68e40762d0 0x7f68e4078790 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:16.129 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.128+0000 7f6907e00640 1 --2- 172.21.15.145:0/2494886732 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f690815efa0 0x7f69081588a0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:16.129 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.128+0000 7f6907e00640 1 --2- 172.21.15.145:0/2494886732 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69080b0640 0x7f6908160aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:16.129 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.128+0000 7f6907e00640 1 --2- 172.21.15.145:0/2494886732 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f690806d9d0 0x7f6908144f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:16.129 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.128+0000 7f6907e00640 1 -- 172.21.15.145:0/2494886732 >> 172.21.15.145:0/2494886732 conn(0x7f69080729a0 msgr2=0x7f690815d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:16.129 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.128+0000 7f6907e00640 1 -- 172.21.15.145:0/2494886732 shutdown_connections 2024-05-09T04:35:16.129 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.128+0000 7f6907e00640 1 -- 172.21.15.145:0/2494886732 wait complete. 2024-05-09T04:35:16.151 INFO:tasks.cephfs_test_runner:That the 'parent' 'layout' and 'symlink' xattrs on the head objects of files ... ok 2024-05-09T04:35:16.151 INFO:tasks.cephfs_test_runner:Starting test: test_backtrace_flush_on_deleted_data_pool (tasks.cephfs.test_backtrace.TestBacktrace) that the MDS does not go read-only when handling backtrace update errors 2024-05-09T04:35:16.151 INFO:tasks.cephfs_test_runner:test_backtrace_flush_on_deleted_data_pool (tasks.cephfs.test_backtrace.TestBacktrace) 2024-05-09T04:35:16.152 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T04:35:16.343 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.344+0000 7fc167200640 1 Processor -- start 2024-05-09T04:35:16.344 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.344+0000 7fc167200640 1 -- start start 2024-05-09T04:35:16.345 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.345+0000 7fc167200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc1681572c0 0x7fc16815deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:16.345 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.345+0000 7fc167200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1680aecd0 con 0x7fc1680ae710 2024-05-09T04:35:16.345 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.345+0000 7fc167200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1680aefb0 con 0x7fc1680cff30 2024-05-09T04:35:16.345 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.345+0000 7fc167200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc1680b6290 con 0x7fc1681572c0 2024-05-09T04:35:16.345 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.346+0000 7fc15fe00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7fc1680ae710 0x7fc168161220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:35538/0 (socket says 172.21.15.145:35538) 2024-05-09T04:35:16.345 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.346+0000 7fc15fe00640 1 -- 172.21.15.145:0/4257356815 learned_addr learned my addr 172.21.15.145:0/4257356815 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:35:16.347 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.346+0000 7fc165e00640 1 --2- 172.21.15.145:0/4257356815 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc1681572c0 0x7fc16815deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:16.347 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.346+0000 7fc165400640 1 -- 172.21.15.145:0/4257356815 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2070049197 0 0) 0x7fc1680aecd0 con 0x7fc1680ae710 2024-05-09T04:35:16.348 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.346+0000 7fc165400640 1 -- 172.21.15.145:0/4257356815 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc148003520 con 0x7fc1680ae710 2024-05-09T04:35:16.348 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.346+0000 7fc165400640 1 -- 172.21.15.145:0/4257356815 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2692304633 0 0) 0x7fc148003520 con 0x7fc1680ae710 2024-05-09T04:35:16.348 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.346+0000 7fc165400640 1 -- 172.21.15.145:0/4257356815 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc1681572c0 msgr2=0x7fc16815deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:16.348 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.346+0000 7fc165400640 1 --2- 172.21.15.145:0/4257356815 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc1681572c0 0x7fc16815deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:16.348 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.346+0000 7fc165400640 1 -- 172.21.15.145:0/4257356815 >> v1:172.21.15.192:6789/0 conn(0x7fc1680cff30 legacy=0x7fc16806e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:16.348 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.346+0000 7fc165400640 1 -- 172.21.15.145:0/4257356815 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1680af290 con 0x7fc1680ae710 2024-05-09T04:35:16.348 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.346+0000 7fc165400640 1 -- 172.21.15.145:0/4257356815 <== mon.0 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3474413261 0 0) 0x7fc1500016f0 con 0x7fc1680ae710 2024-05-09T04:35:16.348 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.346+0000 7fc165400640 1 -- 172.21.15.145:0/4257356815 >> v1:172.21.15.145:6789/0 conn(0x7fc1680ae710 legacy=0x7fc168161220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:16.348 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.347+0000 7fc165400640 1 --2- 172.21.15.145:0/4257356815 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc1681572c0 0x7fc148003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:16.348 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.347+0000 7fc165400640 1 --2- 172.21.15.145:0/4257356815 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc148004000 0x7fc148004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:16.348 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.347+0000 7fc165400640 1 --2- 172.21.15.145:0/4257356815 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc148004d40 0x7fc1480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:16.348 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.347+0000 7fc165400640 1 -- 172.21.15.145:0/4257356815 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc1680b6290 con 0x7fc148004d40 2024-05-09T04:35:16.348 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.347+0000 7fc165400640 1 -- 172.21.15.145:0/4257356815 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc14800bb50 con 0x7fc148004000 2024-05-09T04:35:16.349 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.347+0000 7fc165400640 1 -- 172.21.15.145:0/4257356815 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc14800bd50 con 0x7fc1681572c0 2024-05-09T04:35:16.349 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.347+0000 7fc166800640 1 --2- 172.21.15.145:0/4257356815 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc148004d40 0x7fc1480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:16.349 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.347+0000 7fc165e00640 1 --2- 172.21.15.145:0/4257356815 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc1681572c0 0x7fc148003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:16.349 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.347+0000 7fc15fe00640 1 --2- 172.21.15.145:0/4257356815 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc148004000 0x7fc148004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:16.349 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.347+0000 7fc166800640 1 -- 172.21.15.145:0/4257356815 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc1681572c0 msgr2=0x7fc148003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:16.349 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.347+0000 7fc166800640 1 --2- 172.21.15.145:0/4257356815 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc1681572c0 0x7fc148003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:16.349 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.347+0000 7fc166800640 1 -- 172.21.15.145:0/4257356815 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc148004000 msgr2=0x7fc148004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:16.349 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.347+0000 7fc166800640 1 --2- 172.21.15.145:0/4257356815 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc148004000 0x7fc148004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:16.349 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.347+0000 7fc166800640 1 -- 172.21.15.145:0/4257356815 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc148003520 con 0x7fc148004d40 2024-05-09T04:35:16.349 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.347+0000 7fc15fe00640 1 --2- 172.21.15.145:0/4257356815 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc148004000 0x7fc148004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:35:16.351 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.348+0000 7fc166800640 1 --2- 172.21.15.145:0/4257356815 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc148004d40 0x7fc1480051c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc160001e90 tx=0x7fc148005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb5c32137649d539 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:16.351 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.349+0000 7fc165400640 1 -- 172.21.15.145:0/4257356815 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc160001ab0 con 0x7fc148004d40 2024-05-09T04:35:16.351 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.349+0000 7fc165400640 1 -- 172.21.15.145:0/4257356815 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc16002ae50 con 0x7fc148004d40 2024-05-09T04:35:16.351 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.349+0000 7fc165400640 1 -- 172.21.15.145:0/4257356815 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc160003b70 con 0x7fc148004d40 2024-05-09T04:35:16.351 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.349+0000 7fc167200640 1 -- 172.21.15.145:0/4257356815 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc148004d40 msgr2=0x7fc1480051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:16.351 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.349+0000 7fc167200640 1 --2- 172.21.15.145:0/4257356815 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc148004d40 0x7fc1480051c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc160001e90 tx=0x7fc148005830 comp rx=0 tx=0).stop 2024-05-09T04:35:16.351 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.349+0000 7fc166800640 1 -- 172.21.15.145:0/4257356815 reap_dead start 2024-05-09T04:35:16.351 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.349+0000 7fc167200640 1 -- 172.21.15.145:0/4257356815 shutdown_connections 2024-05-09T04:35:16.352 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.349+0000 7fc167200640 1 -- 172.21.15.145:0/4257356815 >> 172.21.15.145:0/4257356815 conn(0x7fc16806c8a0 msgr2=0x7fc16806ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:16.352 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.349+0000 7fc167200640 1 -- 172.21.15.145:0/4257356815 shutdown_connections 2024-05-09T04:35:16.352 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.349+0000 7fc167200640 1 -- 172.21.15.145:0/4257356815 wait complete. 2024-05-09T04:35:16.352 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.351+0000 7fc167200640 1 Processor -- start 2024-05-09T04:35:16.352 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.351+0000 7fc167200640 1 -- start start 2024-05-09T04:35:16.354 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.352+0000 7fc167200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc168156490 0x7fc1680c5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:16.354 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.352+0000 7fc167200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc168156870 0x7fc16815be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:16.354 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.352+0000 7fc167200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc16815c340 0x7fc1680721b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:16.354 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.352+0000 7fc167200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc168160d00 con 0x7fc168156870 2024-05-09T04:35:16.354 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.352+0000 7fc167200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc168161930 con 0x7fc168156490 2024-05-09T04:35:16.354 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.352+0000 7fc167200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc1681576a0 con 0x7fc16815c340 2024-05-09T04:35:16.354 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.352+0000 7fc15fe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc168156870 0x7fc16815be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:16.354 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.352+0000 7fc15fe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc168156870 0x7fc16815be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:43306/0 (socket says 172.21.15.145:43306) 2024-05-09T04:35:16.354 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.352+0000 7fc15fe00640 1 -- 172.21.15.145:0/4048579239 learned_addr learned my addr 172.21.15.145:0/4048579239 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:16.354 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.353+0000 7fc15fe00640 1 -- 172.21.15.145:0/4048579239 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc16815c340 msgr2=0x7fc1680721b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:16.354 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.353+0000 7fc15fe00640 1 --2- 172.21.15.145:0/4048579239 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc16815c340 0x7fc1680721b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:16.354 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.353+0000 7fc15fe00640 1 -- 172.21.15.145:0/4048579239 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc168156490 msgr2=0x7fc1680c5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:16.355 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.353+0000 7fc15fe00640 1 --2- 172.21.15.145:0/4048579239 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc168156490 0x7fc1680c5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:16.355 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.353+0000 7fc15fe00640 1 -- 172.21.15.145:0/4048579239 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1681f9e90 con 0x7fc168156870 2024-05-09T04:35:16.355 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.353+0000 7fc15fe00640 1 --2- 172.21.15.145:0/4048579239 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc168156870 0x7fc16815be00 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc15000d8b0 tx=0x7fc15000dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:16.355 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.353+0000 7fc15e000640 1 -- 172.21.15.145:0/4048579239 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1500074e0 con 0x7fc168156870 2024-05-09T04:35:16.355 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.353+0000 7fc15e000640 1 -- 172.21.15.145:0/4048579239 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc150007680 con 0x7fc168156870 2024-05-09T04:35:16.355 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.353+0000 7fc167200640 1 -- 172.21.15.145:0/4048579239 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1680c6940 con 0x7fc168156870 2024-05-09T04:35:16.355 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.353+0000 7fc15e000640 1 -- 172.21.15.145:0/4048579239 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1500049e0 con 0x7fc168156870 2024-05-09T04:35:16.355 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.353+0000 7fc167200640 1 -- 172.21.15.145:0/4048579239 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc16809d0a0 con 0x7fc168156870 2024-05-09T04:35:16.355 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.354+0000 7fc167200640 1 -- 172.21.15.145:0/4048579239 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc16815c760 con 0x7fc168156870 2024-05-09T04:35:16.356 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.355+0000 7fc15e000640 1 -- 172.21.15.145:0/4048579239 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc15001b020 con 0x7fc168156870 2024-05-09T04:35:16.356 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.356+0000 7fc15e000640 1 --2- 172.21.15.145:0/4048579239 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fc144076300 0x7fc1440787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:16.356 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.356+0000 7fc15e000640 1 -- 172.21.15.145:0/4048579239 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4943+0+0 (secure 0 0 0) 0x7fc150097110 con 0x7fc168156870 2024-05-09T04:35:16.356 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.357+0000 7fc165e00640 1 --2- 172.21.15.145:0/4048579239 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fc144076300 0x7fc1440787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:16.357 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.357+0000 7fc165e00640 1 --2- 172.21.15.145:0/4048579239 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fc144076300 0x7fc1440787c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc158009910 tx=0x7fc158008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:16.360 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.361+0000 7fc15e000640 1 -- 172.21.15.145:0/4048579239 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc150060d60 con 0x7fc168156870 2024-05-09T04:35:16.570 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.570+0000 7fc167200640 1 -- 172.21.15.145:0/4048579239 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc16806a310 con 0x7fc168156870 2024-05-09T04:35:16.571 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.571+0000 7fc15e000640 1 -- 172.21.15.145:0/4048579239 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+13070 (secure 0 0 0) 0x7fc150060660 con 0x7fc168156870 2024-05-09T04:35:16.571 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:35:16.572 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":39,"fsid":"957ee6c1-073b-44cf-9b93-9491b3fdfc1d","created":"2024-05-09T04:33:26.544367+0000","modified":"2024-05-09T04:35:14.098861+0000","last_up_change":"2024-05-09T04:33:39.039318+0000","last_in_change":"2024-05-09T04:33:29.902361+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T04:33:39.795344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22e550ad-847b-4952-80e6-63d859ed1a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6801","nonce":3182572912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6803","nonce":3182572912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6807","nonce":3182572912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6805","nonce":3182572912}]},"public_addr":"172.21.15.145:6801/3182572912","cluster_addr":"172.21.15.145:6803/3182572912","heartbeat_back_addr":"172.21.15.145:6807/3182572912","heartbeat_front_addr":"172.21.15.145:6805/3182572912","state":["exists","up"]},{"osd":1,"uuid":"0f6b9fca-0c41-4145-8d74-598ddb42f2a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6825","nonce":1845225555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6827","nonce":1845225555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6831","nonce":1845225555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6829","nonce":1845225555}]},"public_addr":"172.21.15.145:6825/1845225555","cluster_addr":"172.21.15.145:6827/1845225555","heartbeat_back_addr":"172.21.15.145:6831/1845225555","heartbeat_front_addr":"172.21.15.145:6829/1845225555","state":["exists","up"]},{"osd":2,"uuid":"b3b1f67e-5790-483f-8610-e05a5bce4c30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6817","nonce":2168958993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6819","nonce":2168958993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6823","nonce":2168958993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6821","nonce":2168958993}]},"public_addr":"172.21.15.145:6817/2168958993","cluster_addr":"172.21.15.145:6819/2168958993","heartbeat_back_addr":"172.21.15.145:6823/2168958993","heartbeat_front_addr":"172.21.15.145:6821/2168958993","state":["exists","up"]},{"osd":3,"uuid":"884787a5-e037-4d31-9aae-743030eba4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6809","nonce":337009826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6811","nonce":337009826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6815","nonce":337009826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6813","nonce":337009826}]},"public_addr":"172.21.15.145:6809/337009826","cluster_addr":"172.21.15.145:6811/337009826","heartbeat_back_addr":"172.21.15.145:6815/337009826","heartbeat_front_addr":"172.21.15.145:6813/337009826","state":["exists","up"]},{"osd":4,"uuid":"0759fc1e-c5fc-4a2f-86bb-e8eccd83f833","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6818","nonce":3117700995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6821","nonce":3117700995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6827","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6829","nonce":3117700995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6823","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6825","nonce":3117700995}]},"public_addr":"172.21.15.192:6818/3117700995","cluster_addr":"172.21.15.192:6821/3117700995","heartbeat_back_addr":"172.21.15.192:6829/3117700995","heartbeat_front_addr":"172.21.15.192:6825/3117700995","state":["exists","up"]},{"osd":5,"uuid":"9443cc22-9379-46e3-9173-98b592bf2c16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6801","nonce":1328373268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6803","nonce":1328373268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6807","nonce":1328373268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6805","nonce":1328373268}]},"public_addr":"172.21.15.192:6801/1328373268","cluster_addr":"172.21.15.192:6803/1328373268","heartbeat_back_addr":"172.21.15.192:6807/1328373268","heartbeat_front_addr":"172.21.15.192:6805/1328373268","state":["exists","up"]},{"osd":6,"uuid":"d05a3311-505a-45b5-85b9-aa3113e96987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6817","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6820","nonce":1868011057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6824","nonce":1868011057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6831","nonce":1868011057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6828","nonce":1868011057}]},"public_addr":"172.21.15.192:6820/1868011057","cluster_addr":"172.21.15.192:6824/1868011057","heartbeat_back_addr":"172.21.15.192:6831/1868011057","heartbeat_front_addr":"172.21.15.192:6828/1868011057","state":["exists","up"]},{"osd":7,"uuid":"c7c23dbc-2029-43ec-9f53-25f3d58019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6809","nonce":3368512157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6811","nonce":3368512157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6815","nonce":3368512157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6813","nonce":3368512157}]},"public_addr":"172.21.15.192:6809/3368512157","cluster_addr":"172.21.15.192:6811/3368512157","heartbeat_back_addr":"172.21.15.192:6815/3368512157","heartbeat_front_addr":"172.21.15.192:6813/3368512157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.091188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.032567+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:34.797468+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.228839+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.622400+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.473103+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.566092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.349395+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.145:6835/4260798843":"2024-05-10T04:35:10.066277+0000","172.21.15.145:6834/4260798843":"2024-05-10T04:35:10.066277+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T04:35:16.574 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.574+0000 7fc167200640 1 -- 172.21.15.145:0/4048579239 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fc144076300 msgr2=0x7fc1440787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:16.574 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.574+0000 7fc167200640 1 --2- 172.21.15.145:0/4048579239 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fc144076300 0x7fc1440787c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc158009910 tx=0x7fc158008040 comp rx=0 tx=0).stop 2024-05-09T04:35:16.574 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.574+0000 7fc167200640 1 -- 172.21.15.145:0/4048579239 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc168156870 msgr2=0x7fc16815be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:16.575 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.574+0000 7fc167200640 1 --2- 172.21.15.145:0/4048579239 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc168156870 0x7fc16815be00 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc15000d8b0 tx=0x7fc15000dd70 comp rx=0 tx=0).stop 2024-05-09T04:35:16.575 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.574+0000 7fc167200640 1 -- 172.21.15.145:0/4048579239 shutdown_connections 2024-05-09T04:35:16.575 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.574+0000 7fc167200640 1 --2- 172.21.15.145:0/4048579239 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fc144076300 0x7fc1440787c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:16.575 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.574+0000 7fc167200640 1 --2- 172.21.15.145:0/4048579239 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc16815c340 0x7fc1680721b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:16.575 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.574+0000 7fc167200640 1 --2- 172.21.15.145:0/4048579239 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc168156870 0x7fc16815be00 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:16.575 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.574+0000 7fc167200640 1 --2- 172.21.15.145:0/4048579239 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc168156490 0x7fc1680c5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:16.575 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.574+0000 7fc167200640 1 -- 172.21.15.145:0/4048579239 >> 172.21.15.145:0/4048579239 conn(0x7fc16806c8a0 msgr2=0x7fc16815ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:16.575 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.575+0000 7fc167200640 1 -- 172.21.15.145:0/4048579239 shutdown_connections 2024-05-09T04:35:16.575 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.575+0000 7fc167200640 1 -- 172.21.15.145:0/4048579239 wait complete. 2024-05-09T04:35:16.593 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T04:33:39.795344+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T04:35:16.594 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T04:35:16.796 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.791+0000 7ff894800640 1 Processor -- start 2024-05-09T04:35:16.797 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.791+0000 7ff894800640 1 -- start start 2024-05-09T04:35:16.797 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.793+0000 7ff894800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff88c1400b0 0x7ff88c0cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:16.797 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.793+0000 7ff894800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff88c06e560 con 0x7ff88c143450 2024-05-09T04:35:16.797 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.793+0000 7ff894800640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff88c140490 con 0x7ff88c142f30 2024-05-09T04:35:16.797 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.793+0000 7ff894800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff88c0d0290 con 0x7ff88c1400b0 2024-05-09T04:35:16.797 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.793+0000 7ff891400640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7ff88c143450 0x7ff88c142820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:35546/0 (socket says 172.21.15.145:35546) 2024-05-09T04:35:16.797 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.793+0000 7ff891400640 1 -- 172.21.15.145:0/1854940557 learned_addr learned my addr 172.21.15.145:0/1854940557 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:35:16.797 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.794+0000 7ff891e00640 1 --2- 172.21.15.145:0/1854940557 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff88c1400b0 0x7ff88c0cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:16.797 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.794+0000 7ff890a00640 1 -- 172.21.15.145:0/1854940557 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2523059162 0 0) 0x7ff88c06e560 con 0x7ff88c143450 2024-05-09T04:35:16.797 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.794+0000 7ff890a00640 1 -- 172.21.15.145:0/1854940557 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff874003520 con 0x7ff88c143450 2024-05-09T04:35:16.797 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.794+0000 7ff890a00640 1 -- 172.21.15.145:0/1854940557 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1345451329 0 0) 0x7ff874003520 con 0x7ff88c143450 2024-05-09T04:35:16.798 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.794+0000 7ff890a00640 1 -- 172.21.15.145:0/1854940557 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff88c1400b0 msgr2=0x7ff88c0cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:16.798 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.794+0000 7ff890a00640 1 --2- 172.21.15.145:0/1854940557 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff88c1400b0 0x7ff88c0cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:16.798 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.794+0000 7ff890a00640 1 -- 172.21.15.145:0/1854940557 >> v1:172.21.15.192:6789/0 conn(0x7ff88c142f30 legacy=0x7ff88c06ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:16.798 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.794+0000 7ff890a00640 1 -- 172.21.15.145:0/1854940557 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff88c0d0570 con 0x7ff88c143450 2024-05-09T04:35:16.798 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.794+0000 7ff890a00640 1 -- 172.21.15.145:0/1854940557 <== mon.0 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3474413261 0 0) 0x7ff87c0016f0 con 0x7ff88c143450 2024-05-09T04:35:16.798 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.794+0000 7ff890a00640 1 -- 172.21.15.145:0/1854940557 >> v1:172.21.15.145:6789/0 conn(0x7ff88c143450 legacy=0x7ff88c142820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:16.798 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.794+0000 7ff890a00640 1 --2- 172.21.15.145:0/1854940557 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff88c1400b0 0x7ff874003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:16.798 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.794+0000 7ff890a00640 1 --2- 172.21.15.145:0/1854940557 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff874004000 0x7ff874004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:16.798 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.794+0000 7ff890a00640 1 --2- 172.21.15.145:0/1854940557 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff874004d40 0x7ff8740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:16.798 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.794+0000 7ff890a00640 1 -- 172.21.15.145:0/1854940557 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff88c0d0290 con 0x7ff874004d40 2024-05-09T04:35:16.798 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.794+0000 7ff890a00640 1 -- 172.21.15.145:0/1854940557 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff87400bb50 con 0x7ff874004000 2024-05-09T04:35:16.798 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.794+0000 7ff890a00640 1 -- 172.21.15.145:0/1854940557 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff87400bd50 con 0x7ff88c1400b0 2024-05-09T04:35:16.799 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.795+0000 7ff892800640 1 --2- 172.21.15.145:0/1854940557 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff874004d40 0x7ff8740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:16.799 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.795+0000 7ff891e00640 1 --2- 172.21.15.145:0/1854940557 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff88c1400b0 0x7ff874003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:16.799 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.795+0000 7ff891400640 1 --2- 172.21.15.145:0/1854940557 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff874004000 0x7ff874004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:16.799 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.795+0000 7ff892800640 1 -- 172.21.15.145:0/1854940557 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff88c1400b0 msgr2=0x7ff874003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:16.799 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.795+0000 7ff892800640 1 --2- 172.21.15.145:0/1854940557 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff88c1400b0 0x7ff874003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:16.799 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.795+0000 7ff892800640 1 -- 172.21.15.145:0/1854940557 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff874004000 msgr2=0x7ff874004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:16.799 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.795+0000 7ff892800640 1 --2- 172.21.15.145:0/1854940557 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff874004000 0x7ff874004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:16.799 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.795+0000 7ff892800640 1 -- 172.21.15.145:0/1854940557 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff874003520 con 0x7ff874004d40 2024-05-09T04:35:16.799 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.796+0000 7ff892800640 1 --2- 172.21.15.145:0/1854940557 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff874004d40 0x7ff8740051c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff888001e90 tx=0x7ff874005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdd3f2eed4985c96 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:16.802 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.797+0000 7ff890a00640 1 -- 172.21.15.145:0/1854940557 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff888001ab0 con 0x7ff874004d40 2024-05-09T04:35:16.802 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.797+0000 7ff890a00640 1 -- 172.21.15.145:0/1854940557 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff88802ae50 con 0x7ff874004d40 2024-05-09T04:35:16.802 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.797+0000 7ff890a00640 1 -- 172.21.15.145:0/1854940557 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff888003b70 con 0x7ff874004d40 2024-05-09T04:35:16.803 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.797+0000 7ff894800640 1 -- 172.21.15.145:0/1854940557 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff874004d40 msgr2=0x7ff8740051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:16.803 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.797+0000 7ff894800640 1 --2- 172.21.15.145:0/1854940557 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff874004d40 0x7ff8740051c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff888001e90 tx=0x7ff874005830 comp rx=0 tx=0).stop 2024-05-09T04:35:16.803 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.797+0000 7ff892800640 1 -- 172.21.15.145:0/1854940557 reap_dead start 2024-05-09T04:35:16.803 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.797+0000 7ff894800640 1 -- 172.21.15.145:0/1854940557 shutdown_connections 2024-05-09T04:35:16.803 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.797+0000 7ff894800640 1 -- 172.21.15.145:0/1854940557 >> 172.21.15.145:0/1854940557 conn(0x7ff88c06cab0 msgr2=0x7ff88c0d0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:16.803 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.797+0000 7ff894800640 1 -- 172.21.15.145:0/1854940557 shutdown_connections 2024-05-09T04:35:16.803 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.797+0000 7ff894800640 1 -- 172.21.15.145:0/1854940557 wait complete. 2024-05-09T04:35:16.803 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.799+0000 7ff894800640 1 Processor -- start 2024-05-09T04:35:16.803 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.799+0000 7ff894800640 1 -- start start 2024-05-09T04:35:16.803 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.800+0000 7ff894800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff88c1501e0 0x7ff88c13d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:16.803 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.800+0000 7ff894800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff88c13e8d0 0x7ff88c15b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:16.804 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.800+0000 7ff894800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff88c13cab0 0x7ff88c15b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:16.804 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.800+0000 7ff894800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff88c0cf7d0 con 0x7ff88c13e8d0 2024-05-09T04:35:16.804 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.800+0000 7ff894800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff88c0d0b30 con 0x7ff88c1501e0 2024-05-09T04:35:16.804 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.800+0000 7ff894800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff88c0cf950 con 0x7ff88c13cab0 2024-05-09T04:35:16.804 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.801+0000 7ff891400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff88c13e8d0 0x7ff88c15b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:16.804 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.801+0000 7ff891400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff88c13e8d0 0x7ff88c15b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:43326/0 (socket says 172.21.15.145:43326) 2024-05-09T04:35:16.804 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.801+0000 7ff891400640 1 -- 172.21.15.145:0/1387585588 learned_addr learned my addr 172.21.15.145:0/1387585588 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:16.804 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.801+0000 7ff891e00640 1 --2- 172.21.15.145:0/1387585588 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff88c1501e0 0x7ff88c13d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:16.804 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.801+0000 7ff891400640 1 -- 172.21.15.145:0/1387585588 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff88c13cab0 msgr2=0x7ff88c15b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:16.804 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.801+0000 7ff892800640 1 --2- 172.21.15.145:0/1387585588 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff88c13cab0 0x7ff88c15b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:16.805 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.801+0000 7ff891400640 1 --2- 172.21.15.145:0/1387585588 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff88c13cab0 0x7ff88c15b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:16.805 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.801+0000 7ff891400640 1 -- 172.21.15.145:0/1387585588 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff88c1501e0 msgr2=0x7ff88c13d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:16.805 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.801+0000 7ff891400640 1 --2- 172.21.15.145:0/1387585588 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff88c1501e0 0x7ff88c13d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:16.805 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.801+0000 7ff891400640 1 -- 172.21.15.145:0/1387585588 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff88c142460 con 0x7ff88c13e8d0 2024-05-09T04:35:16.805 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.801+0000 7ff891e00640 1 --2- 172.21.15.145:0/1387585588 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff88c1501e0 0x7ff88c13d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:35:16.805 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.801+0000 7ff891400640 1 --2- 172.21.15.145:0/1387585588 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff88c13e8d0 0x7ff88c15b440 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff87c00e7d0 tx=0x7ff87c00ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:16.805 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.802+0000 7ff886000640 1 -- 172.21.15.145:0/1387585588 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87c019070 con 0x7ff88c13e8d0 2024-05-09T04:35:16.805 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.802+0000 7ff886000640 1 -- 172.21.15.145:0/1387585588 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff87c00c870 con 0x7ff88c13e8d0 2024-05-09T04:35:16.805 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.802+0000 7ff886000640 1 -- 172.21.15.145:0/1387585588 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87c00cc60 con 0x7ff88c13e8d0 2024-05-09T04:35:16.805 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.802+0000 7ff894800640 1 -- 172.21.15.145:0/1387585588 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff88c13d830 con 0x7ff88c13e8d0 2024-05-09T04:35:16.805 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.802+0000 7ff894800640 1 -- 172.21.15.145:0/1387585588 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff88c13ddf0 con 0x7ff88c13e8d0 2024-05-09T04:35:16.807 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.803+0000 7ff894800640 1 -- 172.21.15.145:0/1387585588 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff88c06a310 con 0x7ff88c13e8d0 2024-05-09T04:35:16.807 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.804+0000 7ff886000640 1 -- 172.21.15.145:0/1387585588 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff87c01c070 con 0x7ff88c13e8d0 2024-05-09T04:35:16.807 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.805+0000 7ff886000640 1 --2- 172.21.15.145:0/1387585588 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff8700762d0 0x7ff870078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:16.807 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.805+0000 7ff886000640 1 -- 172.21.15.145:0/1387585588 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4943+0+0 (secure 0 0 0) 0x7ff87c096aa0 con 0x7ff88c13e8d0 2024-05-09T04:35:16.807 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.806+0000 7ff891e00640 1 --2- 172.21.15.145:0/1387585588 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff8700762d0 0x7ff870078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:16.807 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.806+0000 7ff891e00640 1 --2- 172.21.15.145:0/1387585588 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff8700762d0 0x7ff870078790 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff880009ab0 tx=0x7ff880008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:16.810 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:16.811+0000 7ff886000640 1 -- 172.21.15.145:0/1387585588 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff87c0606f0 con 0x7ff88c13e8d0 2024-05-09T04:35:17.013 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.013+0000 7ff894800640 1 -- 172.21.15.145:0/1387585588 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff88c15b010 con 0x7ff88c13e8d0 2024-05-09T04:35:17.013 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.014+0000 7ff886000640 1 -- 172.21.15.145:0/1387585588 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v39) ==== 93+0+10 (secure 0 0 0) 0x7ff87c05fff0 con 0x7ff88c13e8d0 2024-05-09T04:35:17.013 INFO:teuthology.orchestra.run.smithi145.stdout:pg_num: 1 2024-05-09T04:35:17.017 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.016+0000 7ff894800640 1 -- 172.21.15.145:0/1387585588 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff8700762d0 msgr2=0x7ff870078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:17.017 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.016+0000 7ff894800640 1 --2- 172.21.15.145:0/1387585588 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff8700762d0 0x7ff870078790 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff880009ab0 tx=0x7ff880008040 comp rx=0 tx=0).stop 2024-05-09T04:35:17.017 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.016+0000 7ff894800640 1 -- 172.21.15.145:0/1387585588 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff88c13e8d0 msgr2=0x7ff88c15b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:17.017 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.016+0000 7ff894800640 1 --2- 172.21.15.145:0/1387585588 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff88c13e8d0 0x7ff88c15b440 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff87c00e7d0 tx=0x7ff87c00ec90 comp rx=0 tx=0).stop 2024-05-09T04:35:17.017 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.017+0000 7ff894800640 1 -- 172.21.15.145:0/1387585588 shutdown_connections 2024-05-09T04:35:17.017 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.017+0000 7ff894800640 1 --2- 172.21.15.145:0/1387585588 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff8700762d0 0x7ff870078790 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:17.017 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.017+0000 7ff894800640 1 --2- 172.21.15.145:0/1387585588 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff88c13cab0 0x7ff88c15b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:17.017 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.017+0000 7ff894800640 1 --2- 172.21.15.145:0/1387585588 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff88c13e8d0 0x7ff88c15b440 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:17.017 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.017+0000 7ff894800640 1 --2- 172.21.15.145:0/1387585588 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff88c1501e0 0x7ff88c13d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:17.018 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.017+0000 7ff894800640 1 -- 172.21.15.145:0/1387585588 >> 172.21.15.145:0/1387585588 conn(0x7ff88c06cab0 msgr2=0x7ff88c15f5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:17.018 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.017+0000 7ff894800640 1 -- 172.21.15.145:0/1387585588 shutdown_connections 2024-05-09T04:35:17.018 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.017+0000 7ff894800640 1 -- 172.21.15.145:0/1387585588 wait complete. 2024-05-09T04:35:17.036 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_backtrace.TestBacktrace.test_backtrace_flush_on_deleted_data_pool' 2024-05-09T04:35:17.233 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.233+0000 7f6cc2800640 1 Processor -- start 2024-05-09T04:35:17.233 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.234+0000 7f6cc2800640 1 -- start start 2024-05-09T04:35:17.235 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.235+0000 7f6cc2800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f6cbc15bec0 0x7f6cbc15f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:17.235 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.235+0000 7f6cc2800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f6cbc15fde0 0x7f6cbc0cfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:17.235 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.235+0000 7f6cc2800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cbc160280 con 0x7f6cbc15bae0 2024-05-09T04:35:17.235 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.235+0000 7f6cc2800640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f6cbc160840 con 0x7f6cbc15bec0 2024-05-09T04:35:17.235 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.235+0000 7f6cc2800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f6cbc0b4190 con 0x7f6cbc15fde0 2024-05-09T04:35:17.235 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.235+0000 7f6cc0a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f6cbc15bec0 0x7f6cbc15f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:17.235 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.235+0000 7f6cc1400640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f6cbc15bae0 0x7f6cbc0721b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:35562/0 (socket says 172.21.15.145:35562) 2024-05-09T04:35:17.236 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.235+0000 7f6cc1400640 1 -- 172.21.15.145:0/3487971538 learned_addr learned my addr 172.21.15.145:0/3487971538 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:35:17.236 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.236+0000 7f6cc0a00640 1 -- 172.21.15.145:0/3487971538 >> v2:172.21.15.192:3300/0 conn(0x7f6cbc15fde0 msgr2=0x7f6cbc0cfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:17.236 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.236+0000 7f6cc0a00640 1 --2- 172.21.15.145:0/3487971538 >> v2:172.21.15.192:3300/0 conn(0x7f6cbc15fde0 0x7f6cbc0cfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:17.236 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.236+0000 7f6cc0a00640 1 -- 172.21.15.145:0/3487971538 >> v1:172.21.15.145:6789/0 conn(0x7f6cbc15bae0 legacy=0x7f6cbc0721b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:17.236 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.236+0000 7f6cc0a00640 1 -- 172.21.15.145:0/3487971538 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cbc160560 con 0x7f6cbc15bec0 2024-05-09T04:35:17.236 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.236+0000 7f6cbbe00640 1 -- 172.21.15.145:0/3487971538 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1641778119 0 0) 0x7f6cbc160280 con 0x7f6cbc15bae0 2024-05-09T04:35:17.237 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.237+0000 7f6cc0a00640 1 --2- 172.21.15.145:0/3487971538 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6cbc15bec0 0x7f6cbc15f8a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc0af2c0 tx=0x7f6ca802a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3a9e3f3dec58c9b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:17.237 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.237+0000 7f6cbbe00640 1 -- 172.21.15.145:0/3487971538 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca8037070 con 0x7f6cbc15bec0 2024-05-09T04:35:17.237 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.237+0000 7f6cbbe00640 1 -- 172.21.15.145:0/3487971538 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ca8032440 con 0x7f6cbc15bec0 2024-05-09T04:35:17.237 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.237+0000 7f6cbbe00640 1 -- 172.21.15.145:0/3487971538 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca8032810 con 0x7f6cbc15bec0 2024-05-09T04:35:17.238 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.238+0000 7f6cc2800640 1 -- 172.21.15.145:0/3487971538 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6cbc15bec0 msgr2=0x7f6cbc15f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:17.238 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.238+0000 7f6cc2800640 1 --2- 172.21.15.145:0/3487971538 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6cbc15bec0 0x7f6cbc15f8a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc0af2c0 tx=0x7f6ca802a130 comp rx=0 tx=0).stop 2024-05-09T04:35:17.238 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.238+0000 7f6cc2800640 1 -- 172.21.15.145:0/3487971538 shutdown_connections 2024-05-09T04:35:17.238 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.238+0000 7f6cc2800640 1 --2- 172.21.15.145:0/3487971538 >> v2:172.21.15.192:3300/0 conn(0x7f6cbc15fde0 0x7f6cbc0cfe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:17.238 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.238+0000 7f6cc2800640 1 --2- 172.21.15.145:0/3487971538 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6cbc15bec0 0x7f6cbc15f8a0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:17.238 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.238+0000 7f6cc2800640 1 -- 172.21.15.145:0/3487971538 >> 172.21.15.145:0/3487971538 conn(0x7f6cbc06c8a0 msgr2=0x7f6cbc06ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:17.238 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.238+0000 7f6cc2800640 1 -- 172.21.15.145:0/3487971538 shutdown_connections 2024-05-09T04:35:17.239 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.238+0000 7f6cc2800640 1 -- 172.21.15.145:0/3487971538 wait complete. 2024-05-09T04:35:17.239 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.240+0000 7f6cc2800640 1 Processor -- start 2024-05-09T04:35:17.239 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.240+0000 7f6cc2800640 1 -- start start 2024-05-09T04:35:17.241 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.241+0000 7f6cc2800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cbc15bae0 0x7f6cbc1f5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:17.241 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.241+0000 7f6cc2800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6cbc15bec0 0x7f6cbc1598d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:17.241 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.241+0000 7f6cc2800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6cbc15fde0 0x7f6cbc159e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:17.241 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.241+0000 7f6cc2800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f6cbc0ae050 con 0x7f6cbc15fde0 2024-05-09T04:35:17.241 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.241+0000 7f6cc2800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f6cbc0aded0 con 0x7f6cbc15bae0 2024-05-09T04:35:17.241 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.241+0000 7f6cc2800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f6cbc0d0370 con 0x7f6cbc15bec0 2024-05-09T04:35:17.241 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.241+0000 7f6cc1e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6cbc15fde0 0x7f6cbc159e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:17.241 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.241+0000 7f6cc1e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6cbc15fde0 0x7f6cbc159e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:43344/0 (socket says 172.21.15.145:43344) 2024-05-09T04:35:17.241 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.241+0000 7f6cc1e00640 1 -- 172.21.15.145:0/3624760045 learned_addr learned my addr 172.21.15.145:0/3624760045 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:17.244 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.242+0000 7f6cc1e00640 1 -- 172.21.15.145:0/3624760045 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6cbc15bec0 msgr2=0x7f6cbc1598d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:17.244 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.242+0000 7f6cc1e00640 1 --2- 172.21.15.145:0/3624760045 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6cbc15bec0 0x7f6cbc1598d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:17.244 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.242+0000 7f6cc1e00640 1 -- 172.21.15.145:0/3624760045 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cbc15bae0 msgr2=0x7f6cbc1f5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:17.244 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.242+0000 7f6cc1e00640 1 --2- 172.21.15.145:0/3624760045 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cbc15bae0 0x7f6cbc1f5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:17.244 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.242+0000 7f6cc1e00640 1 -- 172.21.15.145:0/3624760045 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cbc0c6a00 con 0x7f6cbc15fde0 2024-05-09T04:35:17.244 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.242+0000 7f6cc1e00640 1 --2- 172.21.15.145:0/3624760045 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6cbc15fde0 0x7f6cbc159e10 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f6cb400b9e0 tx=0x7f6cb400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:17.244 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.242+0000 7f6cb9600640 1 -- 172.21.15.145:0/3624760045 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb4019070 con 0x7f6cbc15fde0 2024-05-09T04:35:17.244 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.242+0000 7f6cb9600640 1 -- 172.21.15.145:0/3624760045 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cb400cb20 con 0x7f6cbc15fde0 2024-05-09T04:35:17.244 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.242+0000 7f6cb9600640 1 -- 172.21.15.145:0/3624760045 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb4014440 con 0x7f6cbc15fde0 2024-05-09T04:35:17.244 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.242+0000 7f6cc2800640 1 -- 172.21.15.145:0/3624760045 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cbc0c6ce0 con 0x7f6cbc15fde0 2024-05-09T04:35:17.245 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.242+0000 7f6cc2800640 1 -- 172.21.15.145:0/3624760045 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cbc0c72a0 con 0x7f6cbc15fde0 2024-05-09T04:35:17.245 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.243+0000 7f6cc2800640 1 -- 172.21.15.145:0/3624760045 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cbc06a310 con 0x7f6cbc15fde0 2024-05-09T04:35:17.245 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.245+0000 7f6cb9600640 1 -- 172.21.15.145:0/3624760045 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6cb401c070 con 0x7f6cbc15fde0 2024-05-09T04:35:17.246 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.246+0000 7f6cb9600640 1 --2- 172.21.15.145:0/3624760045 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f6c9c0762d0 0x7f6c9c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:17.246 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.247+0000 7f6cb9600640 1 -- 172.21.15.145:0/3624760045 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4943+0+0 (secure 0 0 0) 0x7f6cb40970f0 con 0x7f6cbc15fde0 2024-05-09T04:35:17.246 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.247+0000 7f6cc1400640 1 --2- 172.21.15.145:0/3624760045 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f6c9c0762d0 0x7f6c9c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:17.247 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.247+0000 7f6cc1400640 1 --2- 172.21.15.145:0/3624760045 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f6c9c0762d0 0x7f6c9c078790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6cb00015d0 tx=0x7f6cb00078d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:17.251 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.251+0000 7f6cb9600640 1 -- 172.21.15.145:0/3624760045 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6cb4060c20 con 0x7f6cbc15fde0 2024-05-09T04:35:17.535 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:17.536+0000 7f6cc2800640 1 -- 172.21.15.145:0/3624760045 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_backtrace.TestBacktrace.test_backtrace_flush_on_deleted_data_pool"]} v 0) -- 0x7f6cbc0b5440 con 0x7f6cbc15fde0 2024-05-09T04:35:18.139 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.139+0000 7f6cb9600640 1 -- 172.21.15.145:0/3624760045 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_backtrace.TestBacktrace.test_backtrace_flush_on_deleted_data_pool"]}]=0 v101) ==== 165+0+0 (secure 0 0 0) 0x7f6cb4060520 con 0x7f6cbc15fde0 2024-05-09T04:35:18.144 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.144+0000 7f6cc2800640 1 -- 172.21.15.145:0/3624760045 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f6c9c0762d0 msgr2=0x7f6c9c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:18.144 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.144+0000 7f6cc2800640 1 --2- 172.21.15.145:0/3624760045 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f6c9c0762d0 0x7f6c9c078790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6cb00015d0 tx=0x7f6cb00078d0 comp rx=0 tx=0).stop 2024-05-09T04:35:18.144 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.144+0000 7f6cc2800640 1 -- 172.21.15.145:0/3624760045 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6cbc15fde0 msgr2=0x7f6cbc159e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:18.144 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.144+0000 7f6cc2800640 1 --2- 172.21.15.145:0/3624760045 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6cbc15fde0 0x7f6cbc159e10 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f6cb400b9e0 tx=0x7f6cb400bea0 comp rx=0 tx=0).stop 2024-05-09T04:35:18.144 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.144+0000 7f6cc2800640 1 -- 172.21.15.145:0/3624760045 shutdown_connections 2024-05-09T04:35:18.144 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.144+0000 7f6cc2800640 1 --2- 172.21.15.145:0/3624760045 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f6c9c0762d0 0x7f6c9c078790 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:18.145 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.144+0000 7f6cc2800640 1 --2- 172.21.15.145:0/3624760045 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6cbc15fde0 0x7f6cbc159e10 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:18.145 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.144+0000 7f6cc2800640 1 --2- 172.21.15.145:0/3624760045 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6cbc15bec0 0x7f6cbc1598d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:18.145 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.144+0000 7f6cc2800640 1 --2- 172.21.15.145:0/3624760045 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cbc15bae0 0x7f6cbc1f5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:18.145 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.144+0000 7f6cc2800640 1 -- 172.21.15.145:0/3624760045 >> 172.21.15.145:0/3624760045 conn(0x7f6cbc06c8a0 msgr2=0x7f6cbc0b4620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:18.145 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.144+0000 7f6cc2800640 1 -- 172.21.15.145:0/3624760045 shutdown_connections 2024-05-09T04:35:18.145 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.145+0000 7f6cc2800640 1 -- 172.21.15.145:0/3624760045 wait complete. 2024-05-09T04:35:18.166 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-05-09T04:35:18.374 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.374+0000 7fc107200640 1 Processor -- start 2024-05-09T04:35:18.374 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.374+0000 7fc107200640 1 -- start start 2024-05-09T04:35:18.375 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.375+0000 7fc107200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc1080737c0 0x7fc108072140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:18.375 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.375+0000 7fc107200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc10806fae0 0x7fc10806ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:18.375 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.375+0000 7fc107200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc108072720 con 0x7fc1080b56b0 2024-05-09T04:35:18.375 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.375+0000 7fc107200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fc108072a00 con 0x7fc1080737c0 2024-05-09T04:35:18.375 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.375+0000 7fc107200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc108072ce0 con 0x7fc10806fae0 2024-05-09T04:35:18.375 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.376+0000 7fc105e00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7fc1080b56b0 0x7fc1080af6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:35572/0 (socket says 172.21.15.145:35572) 2024-05-09T04:35:18.376 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.376+0000 7fc105e00640 1 -- 172.21.15.145:0/457869996 learned_addr learned my addr 172.21.15.145:0/457869996 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:35:18.377 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.376+0000 7fc105400640 1 --2- 172.21.15.145:0/457869996 >> v2:172.21.15.192:3300/0 conn(0x7fc1080737c0 0x7fc108072140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:18.377 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.376+0000 7fc106800640 1 --2- 172.21.15.145:0/457869996 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc10806fae0 0x7fc10806ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:18.378 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.376+0000 7fc104a00640 1 -- 172.21.15.145:0/457869996 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2577028166 0 0) 0x7fc108072720 con 0x7fc1080b56b0 2024-05-09T04:35:18.378 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.376+0000 7fc104a00640 1 -- 172.21.15.145:0/457869996 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc0ec003520 con 0x7fc1080b56b0 2024-05-09T04:35:18.378 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.377+0000 7fc104a00640 1 -- 172.21.15.145:0/457869996 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1446454761 0 0) 0x7fc0ec003520 con 0x7fc1080b56b0 2024-05-09T04:35:18.378 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.377+0000 7fc104a00640 1 -- 172.21.15.145:0/457869996 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc10806fae0 msgr2=0x7fc10806ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:18.378 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.377+0000 7fc104a00640 1 --2- 172.21.15.145:0/457869996 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc10806fae0 0x7fc10806ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:18.378 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.377+0000 7fc104a00640 1 -- 172.21.15.145:0/457869996 >> v2:172.21.15.192:3300/0 conn(0x7fc1080737c0 msgr2=0x7fc108072140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:18.378 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.377+0000 7fc104a00640 1 --2- 172.21.15.145:0/457869996 >> v2:172.21.15.192:3300/0 conn(0x7fc1080737c0 0x7fc108072140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:18.378 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.377+0000 7fc104a00640 1 -- 172.21.15.145:0/457869996 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc108072fc0 con 0x7fc1080b56b0 2024-05-09T04:35:18.378 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.377+0000 7fc104a00640 1 -- 172.21.15.145:0/457869996 <== mon.0 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3474413261 0 0) 0x7fc0f80016f0 con 0x7fc1080b56b0 2024-05-09T04:35:18.378 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.377+0000 7fc104a00640 1 -- 172.21.15.145:0/457869996 >> v1:172.21.15.145:6789/0 conn(0x7fc1080b56b0 legacy=0x7fc1080af6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:18.378 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.377+0000 7fc104a00640 1 --2- 172.21.15.145:0/457869996 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0ec003a00 0x7fc0ec003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:18.378 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.377+0000 7fc104a00640 1 --2- 172.21.15.145:0/457869996 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc10806fae0 0x7fc0ec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:18.378 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.377+0000 7fc104a00640 1 --2- 172.21.15.145:0/457869996 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0ec004d40 0x7fc0ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:18.378 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.377+0000 7fc104a00640 1 -- 172.21.15.145:0/457869996 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc108072a00 con 0x7fc0ec003a00 2024-05-09T04:35:18.379 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.377+0000 7fc104a00640 1 -- 172.21.15.145:0/457869996 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc108072ce0 con 0x7fc0ec004d40 2024-05-09T04:35:18.379 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.377+0000 7fc104a00640 1 -- 172.21.15.145:0/457869996 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc0ec00ba40 con 0x7fc10806fae0 2024-05-09T04:35:18.379 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.377+0000 7fc105e00640 1 --2- 172.21.15.145:0/457869996 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0ec003a00 0x7fc0ec003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:18.379 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.377+0000 7fc105e00640 1 -- 172.21.15.145:0/457869996 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc10806fae0 msgr2=0x7fc0ec004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:18.379 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.377+0000 7fc105e00640 1 --2- 172.21.15.145:0/457869996 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc10806fae0 0x7fc0ec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:18.379 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.377+0000 7fc105e00640 1 -- 172.21.15.145:0/457869996 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0ec004d40 msgr2=0x7fc0ec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:18.379 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.377+0000 7fc105e00640 1 --2- 172.21.15.145:0/457869996 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0ec004d40 0x7fc0ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:18.379 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.377+0000 7fc105e00640 1 -- 172.21.15.145:0/457869996 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0ec003520 con 0x7fc0ec003a00 2024-05-09T04:35:18.380 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.378+0000 7fc105e00640 1 --2- 172.21.15.145:0/457869996 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0ec003a00 0x7fc0ec003e20 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc0f800c290 tx=0x7fc0ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fa1c4d7cc7fcd50 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:18.380 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.379+0000 7fc104a00640 1 -- 172.21.15.145:0/457869996 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f8007d50 con 0x7fc0ec003a00 2024-05-09T04:35:18.380 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.379+0000 7fc104a00640 1 -- 172.21.15.145:0/457869996 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0f8005ad0 con 0x7fc0ec003a00 2024-05-09T04:35:18.381 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.379+0000 7fc104a00640 1 -- 172.21.15.145:0/457869996 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f8007070 con 0x7fc0ec003a00 2024-05-09T04:35:18.381 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.379+0000 7fc107200640 1 -- 172.21.15.145:0/457869996 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0ec003a00 msgr2=0x7fc0ec003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:18.381 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.379+0000 7fc107200640 1 --2- 172.21.15.145:0/457869996 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0ec003a00 0x7fc0ec003e20 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc0f800c290 tx=0x7fc0ec005830 comp rx=0 tx=0).stop 2024-05-09T04:35:18.381 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.379+0000 7fc106800640 1 -- 172.21.15.145:0/457869996 reap_dead start 2024-05-09T04:35:18.381 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.379+0000 7fc107200640 1 -- 172.21.15.145:0/457869996 shutdown_connections 2024-05-09T04:35:18.381 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.379+0000 7fc107200640 1 -- 172.21.15.145:0/457869996 >> 172.21.15.145:0/457869996 conn(0x7fc1080b28f0 msgr2=0x7fc1080b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:18.381 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.379+0000 7fc107200640 1 -- 172.21.15.145:0/457869996 shutdown_connections 2024-05-09T04:35:18.381 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.379+0000 7fc107200640 1 -- 172.21.15.145:0/457869996 wait complete. 2024-05-09T04:35:18.383 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.381+0000 7fc107200640 1 Processor -- start 2024-05-09T04:35:18.383 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.381+0000 7fc107200640 1 -- start start 2024-05-09T04:35:18.383 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.382+0000 7fc107200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0ec003a00 0x7fc10815da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:18.383 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.382+0000 7fc107200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc10815dfa0 0x7fc10809bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:18.383 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.382+0000 7fc107200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc10809c1c0 0x7fc10813d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:18.383 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.382+0000 7fc107200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc108070110 con 0x7fc10809c1c0 2024-05-09T04:35:18.383 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.382+0000 7fc107200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc10806ff90 con 0x7fc10815dfa0 2024-05-09T04:35:18.383 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.382+0000 7fc107200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc108070290 con 0x7fc0ec003a00 2024-05-09T04:35:18.383 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.383+0000 7fc106800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc10809c1c0 0x7fc10813d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:18.383 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.383+0000 7fc106800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc10809c1c0 0x7fc10813d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:43354/0 (socket says 172.21.15.145:43354) 2024-05-09T04:35:18.383 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.383+0000 7fc106800640 1 -- 172.21.15.145:0/2352382054 learned_addr learned my addr 172.21.15.145:0/2352382054 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:18.384 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.383+0000 7fc105e00640 1 --2- 172.21.15.145:0/2352382054 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0ec003a00 0x7fc10815da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:18.384 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.383+0000 7fc105400640 1 --2- 172.21.15.145:0/2352382054 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc10815dfa0 0x7fc10809bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:18.384 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.383+0000 7fc106800640 1 -- 172.21.15.145:0/2352382054 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0ec003a00 msgr2=0x7fc10815da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:18.384 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.383+0000 7fc106800640 1 --2- 172.21.15.145:0/2352382054 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0ec003a00 0x7fc10815da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:18.384 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.383+0000 7fc106800640 1 -- 172.21.15.145:0/2352382054 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc10815dfa0 msgr2=0x7fc10809bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:18.384 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.383+0000 7fc106800640 1 --2- 172.21.15.145:0/2352382054 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc10815dfa0 0x7fc10809bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:18.384 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.383+0000 7fc106800640 1 -- 172.21.15.145:0/2352382054 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc108157900 con 0x7fc10809c1c0 2024-05-09T04:35:18.384 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.383+0000 7fc105e00640 1 --2- 172.21.15.145:0/2352382054 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0ec003a00 0x7fc10815da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:35:18.384 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.383+0000 7fc106800640 1 --2- 172.21.15.145:0/2352382054 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc10809c1c0 0x7fc10813d110 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc1080d1c60 tx=0x7fc10000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:18.385 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.384+0000 7fc0fe000640 1 -- 172.21.15.145:0/2352382054 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc100018020 con 0x7fc10809c1c0 2024-05-09T04:35:18.385 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.384+0000 7fc0fe000640 1 -- 172.21.15.145:0/2352382054 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc10000bae0 con 0x7fc10809c1c0 2024-05-09T04:35:18.385 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.384+0000 7fc107200640 1 -- 172.21.15.145:0/2352382054 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc10813b120 con 0x7fc10809c1c0 2024-05-09T04:35:18.385 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.384+0000 7fc0fe000640 1 -- 172.21.15.145:0/2352382054 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc100014400 con 0x7fc10809c1c0 2024-05-09T04:35:18.385 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.384+0000 7fc107200640 1 -- 172.21.15.145:0/2352382054 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1081529a0 con 0x7fc10809c1c0 2024-05-09T04:35:18.385 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.385+0000 7fc107200640 1 -- 172.21.15.145:0/2352382054 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc10806a310 con 0x7fc10809c1c0 2024-05-09T04:35:18.386 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.386+0000 7fc0fe000640 1 -- 172.21.15.145:0/2352382054 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc10001b020 con 0x7fc10809c1c0 2024-05-09T04:35:18.387 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.387+0000 7fc0fe000640 1 --2- 172.21.15.145:0/2352382054 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fc0e80762d0 0x7fc0e8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:18.387 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.387+0000 7fc0fe000640 1 -- 172.21.15.145:0/2352382054 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4943+0+0 (secure 0 0 0) 0x7fc100097720 con 0x7fc10809c1c0 2024-05-09T04:35:18.387 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.387+0000 7fc105e00640 1 --2- 172.21.15.145:0/2352382054 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fc0e80762d0 0x7fc0e8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:18.387 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.388+0000 7fc105e00640 1 --2- 172.21.15.145:0/2352382054 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fc0e80762d0 0x7fc0e8078790 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8007fd0 tx=0x7fc0f802d000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:18.391 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.391+0000 7fc0fe000640 1 -- 172.21.15.145:0/2352382054 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc100022030 con 0x7fc10809c1c0 2024-05-09T04:35:18.596 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.596+0000 7fc107200640 1 -- 172.21.15.145:0/2352382054 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fc10815e380 con 0x7fc10809c1c0 2024-05-09T04:35:18.601 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.601+0000 7fc0fe000640 1 -- 172.21.15.145:0/2352382054 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v15)=0 v15) ==== 122+0+0 (secure 0 0 0) 0x7fc1000670c0 con 0x7fc10809c1c0 2024-05-09T04:35:18.604 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.604+0000 7fc107200640 1 -- 172.21.15.145:0/2352382054 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fc0e80762d0 msgr2=0x7fc0e8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:18.604 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.604+0000 7fc107200640 1 --2- 172.21.15.145:0/2352382054 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fc0e80762d0 0x7fc0e8078790 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8007fd0 tx=0x7fc0f802d000 comp rx=0 tx=0).stop 2024-05-09T04:35:18.604 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.604+0000 7fc107200640 1 -- 172.21.15.145:0/2352382054 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc10809c1c0 msgr2=0x7fc10813d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:18.605 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.604+0000 7fc107200640 1 --2- 172.21.15.145:0/2352382054 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc10809c1c0 0x7fc10813d110 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc1080d1c60 tx=0x7fc10000b480 comp rx=0 tx=0).stop 2024-05-09T04:35:18.605 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.604+0000 7fc107200640 1 -- 172.21.15.145:0/2352382054 shutdown_connections 2024-05-09T04:35:18.605 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.604+0000 7fc107200640 1 --2- 172.21.15.145:0/2352382054 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fc0e80762d0 0x7fc0e8078790 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:18.605 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.604+0000 7fc107200640 1 --2- 172.21.15.145:0/2352382054 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc10809c1c0 0x7fc10813d110 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:18.605 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.604+0000 7fc107200640 1 --2- 172.21.15.145:0/2352382054 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc10815dfa0 0x7fc10809bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:18.605 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.604+0000 7fc107200640 1 --2- 172.21.15.145:0/2352382054 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0ec003a00 0x7fc10815da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:18.605 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.604+0000 7fc107200640 1 -- 172.21.15.145:0/2352382054 >> 172.21.15.145:0/2352382054 conn(0x7fc1080b28f0 msgr2=0x7fc1080b3730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:18.605 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.604+0000 7fc107200640 1 -- 172.21.15.145:0/2352382054 shutdown_connections 2024-05-09T04:35:18.605 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:18.605+0000 7fc107200640 1 -- 172.21.15.145:0/2352382054 wait complete. 2024-05-09T04:35:18.625 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-09T04:35:18.625 DEBUG:teuthology.orchestra.run.smithi145:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-09T04:35:18.722 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-09T04:35:18.844 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-09T04:35:18.845 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-05-09T04:35:18.845 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T04:35:18.903 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-09T04:35:18.903 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-09T04:35:18.903 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T04:35:18.963 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-09T04:35:18.963 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-05-09T04:35:18.963 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T04:35:19.019 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-09T04:35:19.020 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-09T04:35:19.020 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T04:35:19.074 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-09T04:35:19.075 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-05-09T04:35:19.075 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T04:35:19.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.284+0000 7f4676200640 1 Processor -- start 2024-05-09T04:35:19.285 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.285+0000 7f4676200640 1 -- start start 2024-05-09T04:35:19.286 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.286+0000 7f4676200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f467006cb90 0x7f46700b66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:19.286 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.286+0000 7f4676200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4670071930 con 0x7f467015f330 2024-05-09T04:35:19.286 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.286+0000 7f4676200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4670071c10 con 0x7f467006fd30 2024-05-09T04:35:19.286 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.286+0000 7f4676200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f467015f710 con 0x7f467006cb90 2024-05-09T04:35:19.286 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.287+0000 7f466f400640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f467015f330 0x7f46700b5c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:35580/0 (socket says 172.21.15.145:35580) 2024-05-09T04:35:19.286 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.287+0000 7f466f400640 1 -- 172.21.15.145:0/2367332521 learned_addr learned my addr 172.21.15.145:0/2367332521 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:35:19.288 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.287+0000 7f466fe00640 1 --2- 172.21.15.145:0/2367332521 >> v2:172.21.15.192:3300/0 conn(0x7f467006cb90 0x7f46700b66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:19.288 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.287+0000 7f466e000640 1 -- 172.21.15.145:0/2367332521 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 291870937 0 0) 0x7f4670071930 con 0x7f467015f330 2024-05-09T04:35:19.289 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.287+0000 7f466e000640 1 -- 172.21.15.145:0/2367332521 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4654003520 con 0x7f467015f330 2024-05-09T04:35:19.289 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.288+0000 7f466e000640 1 -- 172.21.15.145:0/2367332521 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4110516556 0 0) 0x7f4654003520 con 0x7f467015f330 2024-05-09T04:35:19.289 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.288+0000 7f466e000640 1 -- 172.21.15.145:0/2367332521 >> v2:172.21.15.192:3300/0 conn(0x7f467006cb90 msgr2=0x7f46700b66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:19.289 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.288+0000 7f466e000640 1 --2- 172.21.15.145:0/2367332521 >> v2:172.21.15.192:3300/0 conn(0x7f467006cb90 0x7f46700b66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:19.289 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.288+0000 7f466e000640 1 -- 172.21.15.145:0/2367332521 >> v1:172.21.15.192:6789/0 conn(0x7f467006fd30 legacy=0x7f467006c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:19.289 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.288+0000 7f466e000640 1 -- 172.21.15.145:0/2367332521 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46700b5490 con 0x7f467015f330 2024-05-09T04:35:19.289 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.288+0000 7f466e000640 1 -- 172.21.15.145:0/2367332521 <== mon.0 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3474413261 0 0) 0x7f46600016f0 con 0x7f467015f330 2024-05-09T04:35:19.289 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.288+0000 7f466e000640 1 -- 172.21.15.145:0/2367332521 >> v1:172.21.15.145:6789/0 conn(0x7f467015f330 legacy=0x7f46700b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:19.289 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.288+0000 7f466e000640 1 --2- 172.21.15.145:0/2367332521 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4654003aa0 0x7f4654003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:19.289 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.288+0000 7f466e000640 1 --2- 172.21.15.145:0/2367332521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4654004800 0x7f4654004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:19.289 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.288+0000 7f466e000640 1 --2- 172.21.15.145:0/2367332521 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4654005210 0x7f465400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:19.289 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.288+0000 7f466e000640 1 -- 172.21.15.145:0/2367332521 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f467015f710 con 0x7f4654005210 2024-05-09T04:35:19.289 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.288+0000 7f466e000640 1 -- 172.21.15.145:0/2367332521 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f465400c160 con 0x7f4654004800 2024-05-09T04:35:19.290 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.288+0000 7f466e000640 1 -- 172.21.15.145:0/2367332521 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f465400c360 con 0x7f4654003aa0 2024-05-09T04:35:19.290 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.288+0000 7f466fe00640 1 --2- 172.21.15.145:0/2367332521 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4654005210 0x7f465400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:19.290 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.288+0000 7f466f400640 1 --2- 172.21.15.145:0/2367332521 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4654003aa0 0x7f4654003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:19.290 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.288+0000 7f466ea00640 1 --2- 172.21.15.145:0/2367332521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4654004800 0x7f4654004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:19.290 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.288+0000 7f466fe00640 1 -- 172.21.15.145:0/2367332521 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4654003aa0 msgr2=0x7f4654003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:19.290 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.288+0000 7f466fe00640 1 --2- 172.21.15.145:0/2367332521 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4654003aa0 0x7f4654003ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:19.290 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.288+0000 7f466fe00640 1 -- 172.21.15.145:0/2367332521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4654004800 msgr2=0x7f4654004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:19.290 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.288+0000 7f466fe00640 1 --2- 172.21.15.145:0/2367332521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4654004800 0x7f4654004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:19.290 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.288+0000 7f466fe00640 1 -- 172.21.15.145:0/2367332521 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4654003520 con 0x7f4654005210 2024-05-09T04:35:19.291 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.289+0000 7f466fe00640 1 -- 172.21.15.145:0/2367332521 reap_dead start 2024-05-09T04:35:19.291 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.290+0000 7f466fe00640 1 --2- 172.21.15.145:0/2367332521 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4654005210 0x7f465400b9c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f46700712d0 tx=0x7f4654005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20c4784202552dfd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:19.291 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.290+0000 7f466e000640 1 -- 172.21.15.145:0/2367332521 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4668035020 con 0x7f4654005210 2024-05-09T04:35:19.292 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.290+0000 7f466e000640 1 -- 172.21.15.145:0/2367332521 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4668003b90 con 0x7f4654005210 2024-05-09T04:35:19.292 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.290+0000 7f466e000640 1 -- 172.21.15.145:0/2367332521 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4668009c10 con 0x7f4654005210 2024-05-09T04:35:19.292 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.290+0000 7f4676200640 1 -- 172.21.15.145:0/2367332521 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4654005210 msgr2=0x7f465400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:19.292 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.290+0000 7f4676200640 1 --2- 172.21.15.145:0/2367332521 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4654005210 0x7f465400b9c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f46700712d0 tx=0x7f4654005830 comp rx=0 tx=0).stop 2024-05-09T04:35:19.292 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.290+0000 7f4676200640 1 -- 172.21.15.145:0/2367332521 shutdown_connections 2024-05-09T04:35:19.292 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.290+0000 7f4676200640 1 --2- 172.21.15.145:0/2367332521 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4654005210 0x7f465400b9c0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:19.292 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.290+0000 7f4676200640 1 -- 172.21.15.145:0/2367332521 >> 172.21.15.145:0/2367332521 conn(0x7f46700b3ae0 msgr2=0x7f46700b3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:19.292 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.290+0000 7f4676200640 1 -- 172.21.15.145:0/2367332521 shutdown_connections 2024-05-09T04:35:19.292 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.291+0000 7f4676200640 1 -- 172.21.15.145:0/2367332521 wait complete. 2024-05-09T04:35:19.294 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.292+0000 7f4676200640 1 Processor -- start 2024-05-09T04:35:19.294 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.292+0000 7f4676200640 1 -- start start 2024-05-09T04:35:19.294 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.293+0000 7f4676200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4654005210 0x7f467014b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:19.294 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.293+0000 7f4676200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f46700c6e30 0x7f46701f8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:19.294 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.293+0000 7f4676200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46701f8e80 0x7f4670162440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:19.294 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.293+0000 7f4676200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4670071570 con 0x7f46701f8e80 2024-05-09T04:35:19.294 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.293+0000 7f4676200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f46700716f0 con 0x7f4654005210 2024-05-09T04:35:19.294 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.293+0000 7f4676200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f46700b66a0 con 0x7f46700c6e30 2024-05-09T04:35:19.295 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.294+0000 7f466fe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46701f8e80 0x7f4670162440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:19.295 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.294+0000 7f466fe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46701f8e80 0x7f4670162440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:43372/0 (socket says 172.21.15.145:43372) 2024-05-09T04:35:19.295 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.294+0000 7f466fe00640 1 -- 172.21.15.145:0/3783382192 learned_addr learned my addr 172.21.15.145:0/3783382192 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:19.295 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.294+0000 7f466ea00640 1 --2- 172.21.15.145:0/3783382192 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f46700c6e30 0x7f46701f8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:19.295 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.294+0000 7f466f400640 1 --2- 172.21.15.145:0/3783382192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4654005210 0x7f467014b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:19.295 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.294+0000 7f466fe00640 1 -- 172.21.15.145:0/3783382192 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f46700c6e30 msgr2=0x7f46701f8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:19.295 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.294+0000 7f466fe00640 1 --2- 172.21.15.145:0/3783382192 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f46700c6e30 0x7f46701f8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:19.295 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.294+0000 7f466fe00640 1 -- 172.21.15.145:0/3783382192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4654005210 msgr2=0x7f467014b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:19.295 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.294+0000 7f466fe00640 1 --2- 172.21.15.145:0/3783382192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4654005210 0x7f467014b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:19.295 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.294+0000 7f466fe00640 1 -- 172.21.15.145:0/3783382192 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4670161440 con 0x7f46701f8e80 2024-05-09T04:35:19.295 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.295+0000 7f466fe00640 1 --2- 172.21.15.145:0/3783382192 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46701f8e80 0x7f4670162440 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f466802aeb0 tx=0x7f4668009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:19.296 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.295+0000 7f4667400640 1 -- 172.21.15.145:0/3783382192 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4668035070 con 0x7f46701f8e80 2024-05-09T04:35:19.296 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.295+0000 7f4667400640 1 -- 172.21.15.145:0/3783382192 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4668041070 con 0x7f46701f8e80 2024-05-09T04:35:19.296 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.295+0000 7f4667400640 1 -- 172.21.15.145:0/3783382192 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466803c6b0 con 0x7f46701f8e80 2024-05-09T04:35:19.296 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.295+0000 7f4676200640 1 -- 172.21.15.145:0/3783382192 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4670161160 con 0x7f46701f8e80 2024-05-09T04:35:19.296 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.295+0000 7f4676200640 1 -- 172.21.15.145:0/3783382192 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4670161c40 con 0x7f46701f8e80 2024-05-09T04:35:19.296 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.296+0000 7f4676200640 1 -- 172.21.15.145:0/3783382192 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46700694e0 con 0x7f46701f8e80 2024-05-09T04:35:19.297 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.297+0000 7f4667400640 1 -- 172.21.15.145:0/3783382192 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4668005d10 con 0x7f46701f8e80 2024-05-09T04:35:19.298 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.298+0000 7f4667400640 1 --2- 172.21.15.145:0/3783382192 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f464c0762d0 0x7f464c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:19.298 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.298+0000 7f4667400640 1 -- 172.21.15.145:0/3783382192 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4943+0+0 (secure 0 0 0) 0x7f46680b7fe0 con 0x7f46701f8e80 2024-05-09T04:35:19.298 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.299+0000 7f466f400640 1 --2- 172.21.15.145:0/3783382192 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f464c0762d0 0x7f464c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:19.299 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.299+0000 7f466f400640 1 --2- 172.21.15.145:0/3783382192 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f464c0762d0 0x7f464c078790 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4660005990 tx=0x7f4660009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:19.302 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.303+0000 7f4667400640 1 -- 172.21.15.145:0/3783382192 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f466802e050 con 0x7f46701f8e80 2024-05-09T04:35:19.549 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.550+0000 7f4676200640 1 -- 172.21.15.145:0/3783382192 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46700b70d0 con 0x7f46701f8e80 2024-05-09T04:35:19.551 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.551+0000 7f4667400640 1 -- 172.21.15.145:0/3783382192 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+3875 (secure 0 0 0) 0x7f4668081c30 con 0x7f46701f8e80 2024-05-09T04:35:19.551 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:35:19.551 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":20,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4499,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6834/3227728059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":3227728059},{"type":"v1","addr":"172.21.15.192:6834","nonce":3227728059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4502,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1603013787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6833","nonce":1603013787},{"type":"v1","addr":"172.21.15.192:6835","nonce":1603013787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4672,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/68778418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6836","nonce":68778418},{"type":"v1","addr":"172.21.15.145:6837","nonce":68778418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5220,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6835/2244147475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":2244147475},{"type":"v1","addr":"172.21.15.145:6835","nonce":2244147475}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[]} 2024-05-09T04:35:19.551 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 20 2024-05-09T04:35:19.554 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.554+0000 7f4676200640 1 -- 172.21.15.145:0/3783382192 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f464c0762d0 msgr2=0x7f464c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:19.554 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.554+0000 7f4676200640 1 --2- 172.21.15.145:0/3783382192 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f464c0762d0 0x7f464c078790 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4660005990 tx=0x7f4660009040 comp rx=0 tx=0).stop 2024-05-09T04:35:19.554 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.554+0000 7f4676200640 1 -- 172.21.15.145:0/3783382192 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46701f8e80 msgr2=0x7f4670162440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:19.554 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.554+0000 7f4676200640 1 --2- 172.21.15.145:0/3783382192 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46701f8e80 0x7f4670162440 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f466802aeb0 tx=0x7f4668009750 comp rx=0 tx=0).stop 2024-05-09T04:35:19.554 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.554+0000 7f4676200640 1 -- 172.21.15.145:0/3783382192 shutdown_connections 2024-05-09T04:35:19.554 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.554+0000 7f4676200640 1 --2- 172.21.15.145:0/3783382192 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f464c0762d0 0x7f464c078790 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:19.554 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.554+0000 7f4676200640 1 --2- 172.21.15.145:0/3783382192 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46701f8e80 0x7f4670162440 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:19.554 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.554+0000 7f4676200640 1 --2- 172.21.15.145:0/3783382192 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f46700c6e30 0x7f46701f8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:19.554 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.554+0000 7f4676200640 1 --2- 172.21.15.145:0/3783382192 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4654005210 0x7f467014b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:19.554 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.554+0000 7f4676200640 1 -- 172.21.15.145:0/3783382192 >> 172.21.15.145:0/3783382192 conn(0x7f46700b3ae0 msgr2=0x7f46700b31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:19.555 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.554+0000 7f4676200640 1 -- 172.21.15.145:0/3783382192 shutdown_connections 2024-05-09T04:35:19.555 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.554+0000 7f4676200640 1 -- 172.21.15.145:0/3783382192 wait complete. 2024-05-09T04:35:19.574 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-09T04:35:19.574 INFO:tasks.ceph.mds.a:Stopping old one... 2024-05-09T04:35:19.574 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-09T04:35:19.575 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T04:35:19.575 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-09T04:35:19.575 INFO:tasks.ceph.mds.c:Stopping old one... 2024-05-09T04:35:19.575 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-09T04:35:19.575 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T04:35:19.575 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-09T04:35:19.575 INFO:tasks.ceph.mds.b:Stopping old one... 2024-05-09T04:35:19.576 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-09T04:35:19.576 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T04:35:19.576 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-09T04:35:19.576 INFO:tasks.ceph.mds.d:Stopping old one... 2024-05-09T04:35:19.576 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-09T04:35:19.576 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T04:35:19.592 INFO:tasks.ceph.mds.a:Stopped 2024-05-09T04:35:19.592 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-09T04:35:19.593 INFO:tasks.ceph.mds.c:Stopped 2024-05-09T04:35:19.593 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-09T04:35:19.598 INFO:tasks.ceph.mds.d:Stopped 2024-05-09T04:35:19.598 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-09T04:35:19.604 INFO:tasks.ceph.mds.b:Stopped 2024-05-09T04:35:19.604 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-09T04:35:19.636 INFO:tasks.ceph.mds.a:Started 2024-05-09T04:35:19.637 DEBUG:teuthology.parallel:result is None 2024-05-09T04:35:19.639 INFO:tasks.ceph.mds.c:Started 2024-05-09T04:35:19.639 DEBUG:teuthology.parallel:result is None 2024-05-09T04:35:19.650 INFO:tasks.ceph.mds.d:Started 2024-05-09T04:35:19.651 DEBUG:teuthology.parallel:result is None 2024-05-09T04:35:19.657 INFO:tasks.ceph.mds.b:Started 2024-05-09T04:35:19.657 DEBUG:teuthology.parallel:result is None 2024-05-09T04:35:19.658 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-05-09T04:35:19.784 INFO:tasks.ceph.mds.c.smithi145.stdout:starting mds.c at 2024-05-09T04:35:19.785 INFO:tasks.ceph.mds.a.smithi145.stdout:starting mds.a at 2024-05-09T04:35:19.811 INFO:tasks.ceph.mds.d.smithi192.stdout:starting mds.d at 2024-05-09T04:35:19.812 INFO:tasks.ceph.mds.b.smithi192.stdout:starting mds.b at 2024-05-09T04:35:19.837 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.837+0000 7ff16a600640 1 Processor -- start 2024-05-09T04:35:19.837 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.837+0000 7ff16a600640 1 -- start start 2024-05-09T04:35:19.838 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.838+0000 7ff16a600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff1640d22e0 0x7ff1640cfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:19.838 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.838+0000 7ff16a600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff1640d2db0 0x7ff1640d0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:19.838 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.838+0000 7ff16a600640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1640b4b70 con 0x7ff16415f5f0 2024-05-09T04:35:19.838 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.838+0000 7ff16a600640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7ff16415f9d0 con 0x7ff1640d2db0 2024-05-09T04:35:19.838 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.838+0000 7ff16a600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff1640b4e50 con 0x7ff1640d22e0 2024-05-09T04:35:19.838 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.838+0000 7ff163e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff1640d2db0 0x7ff1640d0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:19.838 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.839+0000 7ff163e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff1640d2db0 0x7ff1640d0230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:43410/0 (socket says 172.21.15.145:43410) 2024-05-09T04:35:19.839 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.839+0000 7ff163e00640 1 -- 172.21.15.145:0/3616320124 learned_addr learned my addr 172.21.15.145:0/3616320124 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:19.839 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.839+0000 7ff162a00640 1 --2- 172.21.15.145:0/3616320124 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff1640d22e0 0x7ff1640cfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:19.839 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.839+0000 7ff163e00640 1 -- 172.21.15.145:0/3616320124 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff1640d22e0 msgr2=0x7ff1640cfcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:19.839 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.839+0000 7ff163e00640 1 --2- 172.21.15.145:0/3616320124 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff1640d22e0 0x7ff1640cfcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:19.839 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.839+0000 7ff163e00640 1 -- 172.21.15.145:0/3616320124 >> v1:172.21.15.192:6789/0 conn(0x7ff16415f5f0 legacy=0x7ff1640d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:19.839 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.839+0000 7ff162000640 1 -- 172.21.15.145:0/3616320124 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3966025272 0 0) 0x7ff1640b4b70 con 0x7ff16415f5f0 2024-05-09T04:35:19.840 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.839+0000 7ff163e00640 1 -- 172.21.15.145:0/3616320124 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1640b5130 con 0x7ff1640d2db0 2024-05-09T04:35:19.840 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.839+0000 7ff162a00640 1 --2- 172.21.15.145:0/3616320124 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff1640d22e0 0x7ff1640cfcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:35:19.841 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.840+0000 7ff163e00640 1 --2- 172.21.15.145:0/3616320124 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff1640d2db0 0x7ff1640d0230 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff15c009e50 tx=0x7ff15c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5767f245f5073a7d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:19.841 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.840+0000 7ff162000640 1 -- 172.21.15.145:0/3616320124 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15c036070 con 0x7ff1640d2db0 2024-05-09T04:35:19.841 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.840+0000 7ff162000640 1 -- 172.21.15.145:0/3616320124 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff15c031600 con 0x7ff1640d2db0 2024-05-09T04:35:19.841 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.840+0000 7ff162000640 1 -- 172.21.15.145:0/3616320124 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15c0319d0 con 0x7ff1640d2db0 2024-05-09T04:35:19.841 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.841+0000 7ff16a600640 1 -- 172.21.15.145:0/3616320124 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff1640d2db0 msgr2=0x7ff1640d0230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:19.841 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.841+0000 7ff16a600640 1 --2- 172.21.15.145:0/3616320124 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff1640d2db0 0x7ff1640d0230 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff15c009e50 tx=0x7ff15c02a710 comp rx=0 tx=0).stop 2024-05-09T04:35:19.841 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.841+0000 7ff16a600640 1 -- 172.21.15.145:0/3616320124 shutdown_connections 2024-05-09T04:35:19.841 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.841+0000 7ff16a600640 1 --2- 172.21.15.145:0/3616320124 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff1640d2db0 0x7ff1640d0230 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:19.841 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.841+0000 7ff16a600640 1 --2- 172.21.15.145:0/3616320124 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff1640d22e0 0x7ff1640cfcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:19.841 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.841+0000 7ff16a600640 1 -- 172.21.15.145:0/3616320124 >> 172.21.15.145:0/3616320124 conn(0x7ff16409e520 msgr2=0x7ff1640d51e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:19.841 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.841+0000 7ff16a600640 1 -- 172.21.15.145:0/3616320124 shutdown_connections 2024-05-09T04:35:19.842 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.841+0000 7ff16a600640 1 -- 172.21.15.145:0/3616320124 wait complete. 2024-05-09T04:35:19.842 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.843+0000 7ff16a600640 1 Processor -- start 2024-05-09T04:35:19.842 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.843+0000 7ff16a600640 1 -- start start 2024-05-09T04:35:19.844 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.844+0000 7ff16a600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff1640d22e0 0x7ff16415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:19.844 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.844+0000 7ff16a600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1640d2db0 0x7ff1640b99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:19.844 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.844+0000 7ff16a600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff16415f5f0 0x7ff1640b9f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:19.844 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.844+0000 7ff16a600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff1640d3c90 con 0x7ff1640d22e0 2024-05-09T04:35:19.845 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.844+0000 7ff16a600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff1640d4040 con 0x7ff1640d2db0 2024-05-09T04:35:19.845 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.844+0000 7ff16a600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff16415e110 con 0x7ff16415f5f0 2024-05-09T04:35:19.845 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.844+0000 7ff163400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff1640d22e0 0x7ff16415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:19.845 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.844+0000 7ff163400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff1640d22e0 0x7ff16415d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:43424/0 (socket says 172.21.15.145:43424) 2024-05-09T04:35:19.845 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.844+0000 7ff163400640 1 -- 172.21.15.145:0/308687827 learned_addr learned my addr 172.21.15.145:0/308687827 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:19.845 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.844+0000 7ff162a00640 1 --2- 172.21.15.145:0/308687827 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1640d2db0 0x7ff1640b99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:19.845 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.844+0000 7ff163e00640 1 --2- 172.21.15.145:0/308687827 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff16415f5f0 0x7ff1640b9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:19.845 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.845+0000 7ff163400640 1 -- 172.21.15.145:0/308687827 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff16415f5f0 msgr2=0x7ff1640b9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:19.845 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.845+0000 7ff163400640 1 --2- 172.21.15.145:0/308687827 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff16415f5f0 0x7ff1640b9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:19.845 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.845+0000 7ff163400640 1 -- 172.21.15.145:0/308687827 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1640d2db0 msgr2=0x7ff1640b99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:19.845 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.845+0000 7ff163400640 1 --2- 172.21.15.145:0/308687827 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1640d2db0 0x7ff1640b99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:19.845 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.845+0000 7ff163400640 1 -- 172.21.15.145:0/308687827 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1640b89b0 con 0x7ff1640d22e0 2024-05-09T04:35:19.845 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.845+0000 7ff163400640 1 --2- 172.21.15.145:0/308687827 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff1640d22e0 0x7ff16415d5d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff15400c500 tx=0x7ff15400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:19.846 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.845+0000 7ff15b400640 1 -- 172.21.15.145:0/308687827 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff154017070 con 0x7ff1640d22e0 2024-05-09T04:35:19.846 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.845+0000 7ff15b400640 1 -- 172.21.15.145:0/308687827 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1540035f0 con 0x7ff1640d22e0 2024-05-09T04:35:19.846 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.845+0000 7ff16a600640 1 -- 172.21.15.145:0/308687827 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1640b8c90 con 0x7ff1640d22e0 2024-05-09T04:35:19.846 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.845+0000 7ff16a600640 1 -- 172.21.15.145:0/308687827 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff16415db10 con 0x7ff1640d22e0 2024-05-09T04:35:19.846 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.845+0000 7ff15b400640 1 -- 172.21.15.145:0/308687827 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff154003a20 con 0x7ff1640d22e0 2024-05-09T04:35:19.846 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.846+0000 7ff16a600640 1 -- 172.21.15.145:0/308687827 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff16406a310 con 0x7ff1640d22e0 2024-05-09T04:35:19.847 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.847+0000 7ff15b400640 1 -- 172.21.15.145:0/308687827 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff15401a070 con 0x7ff1640d22e0 2024-05-09T04:35:19.849 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.849+0000 7ff15b400640 1 --2- 172.21.15.145:0/308687827 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff1440762d0 0x7ff144078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:19.849 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.849+0000 7ff15b400640 1 -- 172.21.15.145:0/308687827 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4943+0+0 (secure 0 0 0) 0x7ff1540967e0 con 0x7ff1640d22e0 2024-05-09T04:35:19.849 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.850+0000 7ff162a00640 1 --2- 172.21.15.145:0/308687827 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff1440762d0 0x7ff144078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:19.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.850+0000 7ff162a00640 1 --2- 172.21.15.145:0/308687827 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff1440762d0 0x7ff144078790 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff1640bae90 tx=0x7ff150008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:19.853 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:19.854+0000 7ff15b400640 1 -- 172.21.15.145:0/308687827 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff154060430 con 0x7ff1640d22e0 2024-05-09T04:35:20.069 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.069+0000 7ff16a600640 1 -- 172.21.15.145:0/308687827 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7ff1640cc320 con 0x7ff1640d22e0 2024-05-09T04:35:20.165 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.165+0000 7ff15b400640 1 -- 172.21.15.145:0/308687827 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v41) ==== 97+0+0 (secure 0 0 0) 0x7ff154068420 con 0x7ff1640d22e0 2024-05-09T04:35:20.165 INFO:teuthology.orchestra.run.smithi145.stderr: removed all blocklist entries 2024-05-09T04:35:20.168 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.168+0000 7ff16a600640 1 -- 172.21.15.145:0/308687827 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff1440762d0 msgr2=0x7ff144078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:20.168 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.168+0000 7ff16a600640 1 --2- 172.21.15.145:0/308687827 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff1440762d0 0x7ff144078790 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff1640bae90 tx=0x7ff150008040 comp rx=0 tx=0).stop 2024-05-09T04:35:20.168 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.168+0000 7ff16a600640 1 -- 172.21.15.145:0/308687827 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff1640d22e0 msgr2=0x7ff16415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:20.168 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.168+0000 7ff16a600640 1 --2- 172.21.15.145:0/308687827 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff1640d22e0 0x7ff16415d5d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff15400c500 tx=0x7ff15400c9c0 comp rx=0 tx=0).stop 2024-05-09T04:35:20.169 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.168+0000 7ff16a600640 1 -- 172.21.15.145:0/308687827 shutdown_connections 2024-05-09T04:35:20.169 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.168+0000 7ff16a600640 1 --2- 172.21.15.145:0/308687827 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff1440762d0 0x7ff144078790 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:20.169 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.168+0000 7ff16a600640 1 --2- 172.21.15.145:0/308687827 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff16415f5f0 0x7ff1640b9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:20.169 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.168+0000 7ff16a600640 1 --2- 172.21.15.145:0/308687827 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1640d2db0 0x7ff1640b99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:20.169 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.168+0000 7ff16a600640 1 --2- 172.21.15.145:0/308687827 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff1640d22e0 0x7ff16415d5d0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:20.169 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.168+0000 7ff16a600640 1 -- 172.21.15.145:0/308687827 >> 172.21.15.145:0/308687827 conn(0x7ff16409e520 msgr2=0x7ff1640b5830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:20.169 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.169+0000 7ff16a600640 1 -- 172.21.15.145:0/308687827 shutdown_connections 2024-05-09T04:35:20.169 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.169+0000 7ff16a600640 1 -- 172.21.15.145:0/308687827 wait complete. 2024-05-09T04:35:20.189 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-05-09T04:35:20.373 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.373+0000 7f333a400640 1 Processor -- start 2024-05-09T04:35:20.373 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.373+0000 7f333a400640 1 -- start start 2024-05-09T04:35:20.376 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.375+0000 7f333a400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f33341633e0 0x7f33341602a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:20.376 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.375+0000 7f333a400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f333406f1c0 0x7f333406ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:20.376 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.375+0000 7f333a400640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3334070c30 con 0x7f3334163000 2024-05-09T04:35:20.376 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.375+0000 7f333a400640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f333406d3d0 con 0x7f33341633e0 2024-05-09T04:35:20.376 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.375+0000 7f333a400640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f3334070f10 con 0x7f333406f1c0 2024-05-09T04:35:20.376 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.376+0000 7f3333e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f33341633e0 0x7f33341602a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:20.376 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.376+0000 7f3339000640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f3334163000 0x7f33340d14d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:35602/0 (socket says 172.21.15.145:35602) 2024-05-09T04:35:20.376 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.376+0000 7f3339000640 1 -- 172.21.15.145:0/3232565967 learned_addr learned my addr 172.21.15.145:0/3232565967 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:35:20.376 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.376+0000 7f3339a00640 1 --2- 172.21.15.145:0/3232565967 >> v2:172.21.15.192:3300/0 conn(0x7f333406f1c0 0x7f333406ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:20.376 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.376+0000 7f3333e00640 1 -- 172.21.15.145:0/3232565967 >> v2:172.21.15.192:3300/0 conn(0x7f333406f1c0 msgr2=0x7f333406ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:20.376 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.376+0000 7f3333e00640 1 --2- 172.21.15.145:0/3232565967 >> v2:172.21.15.192:3300/0 conn(0x7f333406f1c0 0x7f333406ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:20.376 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.376+0000 7f3333e00640 1 -- 172.21.15.145:0/3232565967 >> v1:172.21.15.145:6789/0 conn(0x7f3334163000 legacy=0x7f33340d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:20.376 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.376+0000 7f3333e00640 1 -- 172.21.15.145:0/3232565967 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f333406c690 con 0x7f33341633e0 2024-05-09T04:35:20.378 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.377+0000 7f3333e00640 1 --2- 172.21.15.145:0/3232565967 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f33341633e0 0x7f33341602a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f33200097a0 tx=0x7f332002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=effba22722ba9c92 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:20.378 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.377+0000 7f3333400640 1 -- 172.21.15.145:0/3232565967 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3320037070 con 0x7f33341633e0 2024-05-09T04:35:20.378 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.377+0000 7f3333400640 1 -- 172.21.15.145:0/3232565967 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f332002ae50 con 0x7f33341633e0 2024-05-09T04:35:20.378 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.377+0000 7f3333400640 1 -- 172.21.15.145:0/3232565967 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3320032660 con 0x7f33341633e0 2024-05-09T04:35:20.378 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.378+0000 7f333a400640 1 -- 172.21.15.145:0/3232565967 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f33341633e0 msgr2=0x7f33341602a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:20.378 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.378+0000 7f333a400640 1 --2- 172.21.15.145:0/3232565967 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f33341633e0 0x7f33341602a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f33200097a0 tx=0x7f332002a140 comp rx=0 tx=0).stop 2024-05-09T04:35:20.378 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.378+0000 7f333a400640 1 -- 172.21.15.145:0/3232565967 shutdown_connections 2024-05-09T04:35:20.378 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.378+0000 7f333a400640 1 --2- 172.21.15.145:0/3232565967 >> v2:172.21.15.192:3300/0 conn(0x7f333406f1c0 0x7f333406ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:20.378 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.378+0000 7f333a400640 1 --2- 172.21.15.145:0/3232565967 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f33341633e0 0x7f33341602a0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:20.378 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.378+0000 7f333a400640 1 -- 172.21.15.145:0/3232565967 >> 172.21.15.145:0/3232565967 conn(0x7f33340b3ae0 msgr2=0x7f33340b3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:20.378 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.378+0000 7f333a400640 1 -- 172.21.15.145:0/3232565967 shutdown_connections 2024-05-09T04:35:20.378 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.378+0000 7f333a400640 1 -- 172.21.15.145:0/3232565967 wait complete. 2024-05-09T04:35:20.379 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.380+0000 7f333a400640 1 Processor -- start 2024-05-09T04:35:20.379 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.380+0000 7f333a400640 1 -- start start 2024-05-09T04:35:20.382 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.381+0000 7f333a400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f333406f1c0 0x7f3334158cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:20.382 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.381+0000 7f333a400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3334163000 0x7f33341f59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:20.382 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.381+0000 7f333a400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33341633e0 0x7f333414b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:20.382 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.381+0000 7f333a400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f3334073380 con 0x7f3334163000 2024-05-09T04:35:20.382 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.381+0000 7f333a400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f3334073200 con 0x7f333406f1c0 2024-05-09T04:35:20.382 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.381+0000 7f333a400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f3334073500 con 0x7f33341633e0 2024-05-09T04:35:20.382 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.381+0000 7f3333e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3334163000 0x7f33341f59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:20.382 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.381+0000 7f3333e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3334163000 0x7f33341f59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:43450/0 (socket says 172.21.15.145:43450) 2024-05-09T04:35:20.382 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.381+0000 7f3333e00640 1 -- 172.21.15.145:0/3747183267 learned_addr learned my addr 172.21.15.145:0/3747183267 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:20.382 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.381+0000 7f3333e00640 1 -- 172.21.15.145:0/3747183267 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33341633e0 msgr2=0x7f333414b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:20.382 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.381+0000 7f3333e00640 1 --2- 172.21.15.145:0/3747183267 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33341633e0 0x7f333414b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:20.382 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.381+0000 7f3339000640 1 --2- 172.21.15.145:0/3747183267 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f333406f1c0 0x7f3334158cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:20.383 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.381+0000 7f3333e00640 1 -- 172.21.15.145:0/3747183267 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f333406f1c0 msgr2=0x7f3334158cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:20.383 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.381+0000 7f3333e00640 1 --2- 172.21.15.145:0/3747183267 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f333406f1c0 0x7f3334158cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:20.383 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.381+0000 7f3333e00640 1 -- 172.21.15.145:0/3747183267 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33340b5b50 con 0x7f3334163000 2024-05-09T04:35:20.383 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.382+0000 7f3333e00640 1 --2- 172.21.15.145:0/3747183267 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3334163000 0x7f33341f59d0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f33200098c0 tx=0x7f33200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:20.383 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.382+0000 7f3330c00640 1 -- 172.21.15.145:0/3747183267 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3320041030 con 0x7f3334163000 2024-05-09T04:35:20.383 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.382+0000 7f3330c00640 1 -- 172.21.15.145:0/3747183267 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f332002ac50 con 0x7f3334163000 2024-05-09T04:35:20.383 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.382+0000 7f333a400640 1 -- 172.21.15.145:0/3747183267 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33340b5e30 con 0x7f3334163000 2024-05-09T04:35:20.383 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.382+0000 7f3330c00640 1 -- 172.21.15.145:0/3747183267 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3320037040 con 0x7f3334163000 2024-05-09T04:35:20.383 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.382+0000 7f333a400640 1 -- 172.21.15.145:0/3747183267 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33340b63f0 con 0x7f3334163000 2024-05-09T04:35:20.383 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.383+0000 7f333a400640 1 -- 172.21.15.145:0/3747183267 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33340694e0 con 0x7f3334163000 2024-05-09T04:35:20.384 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.384+0000 7f3330c00640 1 -- 172.21.15.145:0/3747183267 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f3320044070 con 0x7f3334163000 2024-05-09T04:35:20.385 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.385+0000 7f3330c00640 1 --2- 172.21.15.145:0/3747183267 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f33140762d0 0x7f3314078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:20.385 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.385+0000 7f3330c00640 1 -- 172.21.15.145:0/3747183267 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4857+0+0 (secure 0 0 0) 0x7f33200b7920 con 0x7f3334163000 2024-05-09T04:35:20.385 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.386+0000 7f3339000640 1 --2- 172.21.15.145:0/3747183267 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f33140762d0 0x7f3314078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:20.386 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.386+0000 7f3339000640 1 --2- 172.21.15.145:0/3747183267 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f33140762d0 0x7f3314078790 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3328000f60 tx=0x7f3328009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:20.389 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.390+0000 7f3330c00640 1 -- 172.21.15.145:0/3747183267 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3320081530 con 0x7f3334163000 2024-05-09T04:35:20.668 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.668+0000 7f333a400640 1 -- 172.21.15.145:0/3747183267 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f33340743c0 con 0x7f3334163000 2024-05-09T04:35:20.669 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.669+0000 7f3330c00640 1 -- 172.21.15.145:0/3747183267 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) ==== 80+0+5304 (secure 0 0 0) 0x7f3320080e30 con 0x7f3334163000 2024-05-09T04:35:20.669 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:35:20.669 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-05-09T04:35:20.669 INFO:teuthology.orchestra.run.smithi145.stdout: "auth_dump": [ 2024-05-09T04:35:20.670 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-05-09T04:35:20.670 INFO:teuthology.orchestra.run.smithi145.stdout: "entity": "mds.a", 2024-05-09T04:35:20.670 INFO:teuthology.orchestra.run.smithi145.stdout: "key": "AQDlUTxmfnR3ERAAUsvH1ebffiEsH8dMX+Xe1w==", 2024-05-09T04:35:20.670 INFO:teuthology.orchestra.run.smithi145.stdout: "caps": { 2024-05-09T04:35:20.670 INFO:teuthology.orchestra.run.smithi145.stdout: "mds": "allow", 2024-05-09T04:35:20.670 INFO:teuthology.orchestra.run.smithi145.stdout: "mgr": "allow *", 2024-05-09T04:35:20.670 INFO:teuthology.orchestra.run.smithi145.stdout: "mon": "allow *", 2024-05-09T04:35:20.670 INFO:teuthology.orchestra.run.smithi145.stdout: "osd": "allow *" 2024-05-09T04:35:20.670 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-05-09T04:35:20.670 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:35:20.670 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-05-09T04:35:20.670 INFO:teuthology.orchestra.run.smithi145.stdout: "entity": "mds.b", 2024-05-09T04:35:20.670 INFO:teuthology.orchestra.run.smithi145.stdout: "key": "AQDlUTxm02GmLhAAI7JaX2VaRh08cXbDLpBHDg==", 2024-05-09T04:35:20.671 INFO:teuthology.orchestra.run.smithi145.stdout: "caps": { 2024-05-09T04:35:20.671 INFO:teuthology.orchestra.run.smithi145.stdout: "mds": "allow", 2024-05-09T04:35:20.671 INFO:teuthology.orchestra.run.smithi145.stdout: "mgr": "allow *", 2024-05-09T04:35:20.671 INFO:teuthology.orchestra.run.smithi145.stdout: "mon": "allow *", 2024-05-09T04:35:20.671 INFO:teuthology.orchestra.run.smithi145.stdout: "osd": "allow *" 2024-05-09T04:35:20.671 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-05-09T04:35:20.671 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:35:20.671 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-05-09T04:35:20.671 INFO:teuthology.orchestra.run.smithi145.stdout: "entity": "mds.c", 2024-05-09T04:35:20.671 INFO:teuthology.orchestra.run.smithi145.stdout: "key": "AQDlUTxm1//EHxAADR9WdmuEIMrbMzQDHi9ltQ==", 2024-05-09T04:35:20.671 INFO:teuthology.orchestra.run.smithi145.stdout: "caps": { 2024-05-09T04:35:20.672 INFO:teuthology.orchestra.run.smithi145.stdout: "mds": "allow", 2024-05-09T04:35:20.672 INFO:teuthology.orchestra.run.smithi145.stdout: "mgr": "allow *", 2024-05-09T04:35:20.672 INFO:teuthology.orchestra.run.smithi145.stdout: "mon": "allow *", 2024-05-09T04:35:20.672 INFO:teuthology.orchestra.run.smithi145.stdout: "osd": "allow *" 2024-05-09T04:35:20.672 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-05-09T04:35:20.672 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:35:20.672 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-05-09T04:35:20.672 INFO:teuthology.orchestra.run.smithi145.stdout: "entity": "mds.d", 2024-05-09T04:35:20.672 INFO:teuthology.orchestra.run.smithi145.stdout: "key": "AQDmUTxm5qxUARAA6NW1Rj3N4APnz2hYdHOpKw==", 2024-05-09T04:35:20.672 INFO:teuthology.orchestra.run.smithi145.stdout: "caps": { 2024-05-09T04:35:20.672 INFO:teuthology.orchestra.run.smithi145.stdout: "mds": "allow", 2024-05-09T04:35:20.672 INFO:teuthology.orchestra.run.smithi145.stdout: "mgr": "allow *", 2024-05-09T04:35:20.672 INFO:teuthology.orchestra.run.smithi145.stdout: "mon": "allow *", 2024-05-09T04:35:20.672 INFO:teuthology.orchestra.run.smithi145.stdout: "osd": "allow *" 2024-05-09T04:35:20.673 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-05-09T04:35:20.673 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:35:20.673 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-05-09T04:35:20.673 INFO:teuthology.orchestra.run.smithi145.stdout: "entity": "osd.0", 2024-05-09T04:35:20.673 INFO:teuthology.orchestra.run.smithi145.stdout: "key": "AQDpUTxms65yMhAARwU3xBmDigriIR7IGY00iQ==", 2024-05-09T04:35:20.673 INFO:teuthology.orchestra.run.smithi145.stdout: "caps": { 2024-05-09T04:35:20.673 INFO:teuthology.orchestra.run.smithi145.stdout: "mgr": "allow profile osd", 2024-05-09T04:35:20.673 INFO:teuthology.orchestra.run.smithi145.stdout: "mon": "allow profile osd", 2024-05-09T04:35:20.673 INFO:teuthology.orchestra.run.smithi145.stdout: "osd": "allow *" 2024-05-09T04:35:20.673 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-05-09T04:35:20.673 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:35:20.673 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-05-09T04:35:20.673 INFO:teuthology.orchestra.run.smithi145.stdout: "entity": "osd.1", 2024-05-09T04:35:20.673 INFO:teuthology.orchestra.run.smithi145.stdout: "key": "AQDtUTxmfJohERAAEVklN8QfeSfc8C8vZTlE2A==", 2024-05-09T04:35:20.673 INFO:teuthology.orchestra.run.smithi145.stdout: "caps": { 2024-05-09T04:35:20.673 INFO:teuthology.orchestra.run.smithi145.stdout: "mgr": "allow profile osd", 2024-05-09T04:35:20.674 INFO:teuthology.orchestra.run.smithi145.stdout: "mon": "allow profile osd", 2024-05-09T04:35:20.674 INFO:teuthology.orchestra.run.smithi145.stdout: "osd": "allow *" 2024-05-09T04:35:20.674 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-05-09T04:35:20.674 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:35:20.674 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-05-09T04:35:20.674 INFO:teuthology.orchestra.run.smithi145.stdout: "entity": "osd.2", 2024-05-09T04:35:20.674 INFO:teuthology.orchestra.run.smithi145.stdout: "key": "AQDwUTxmFvcpKhAARtXZ1qSYuJp6gZCTAj0AiA==", 2024-05-09T04:35:20.674 INFO:teuthology.orchestra.run.smithi145.stdout: "caps": { 2024-05-09T04:35:20.674 INFO:teuthology.orchestra.run.smithi145.stdout: "mgr": "allow profile osd", 2024-05-09T04:35:20.674 INFO:teuthology.orchestra.run.smithi145.stdout: "mon": "allow profile osd", 2024-05-09T04:35:20.674 INFO:teuthology.orchestra.run.smithi145.stdout: "osd": "allow *" 2024-05-09T04:35:20.674 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-05-09T04:35:20.674 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:35:20.675 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-05-09T04:35:20.675 INFO:teuthology.orchestra.run.smithi145.stdout: "entity": "osd.3", 2024-05-09T04:35:20.675 INFO:teuthology.orchestra.run.smithi145.stdout: "key": "AQDzUTxmqcp8NBAAWjK3tnEyljnJEu/wAx77TQ==", 2024-05-09T04:35:20.675 INFO:teuthology.orchestra.run.smithi145.stdout: "caps": { 2024-05-09T04:35:20.675 INFO:teuthology.orchestra.run.smithi145.stdout: "mgr": "allow profile osd", 2024-05-09T04:35:20.675 INFO:teuthology.orchestra.run.smithi145.stdout: "mon": "allow profile osd", 2024-05-09T04:35:20.675 INFO:teuthology.orchestra.run.smithi145.stdout: "osd": "allow *" 2024-05-09T04:35:20.675 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-05-09T04:35:20.675 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:35:20.675 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-05-09T04:35:20.675 INFO:teuthology.orchestra.run.smithi145.stdout: "entity": "osd.4", 2024-05-09T04:35:20.675 INFO:teuthology.orchestra.run.smithi145.stdout: "key": "AQD6UTxmtvWdDxAAe9r1F/szPZid+E9/Vuio7Q==", 2024-05-09T04:35:20.675 INFO:teuthology.orchestra.run.smithi145.stdout: "caps": { 2024-05-09T04:35:20.676 INFO:teuthology.orchestra.run.smithi145.stdout: "mgr": "allow profile osd", 2024-05-09T04:35:20.676 INFO:teuthology.orchestra.run.smithi145.stdout: "mon": "allow profile osd", 2024-05-09T04:35:20.676 INFO:teuthology.orchestra.run.smithi145.stdout: "osd": "allow *" 2024-05-09T04:35:20.676 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-05-09T04:35:20.676 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:35:20.676 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-05-09T04:35:20.676 INFO:teuthology.orchestra.run.smithi145.stdout: "entity": "osd.5", 2024-05-09T04:35:20.676 INFO:teuthology.orchestra.run.smithi145.stdout: "key": "AQD9UTxm0LE1GhAAIm98EcczdlqMnivEAjIsHQ==", 2024-05-09T04:35:20.676 INFO:teuthology.orchestra.run.smithi145.stdout: "caps": { 2024-05-09T04:35:20.676 INFO:teuthology.orchestra.run.smithi145.stdout: "mgr": "allow profile osd", 2024-05-09T04:35:20.676 INFO:teuthology.orchestra.run.smithi145.stdout: "mon": "allow profile osd", 2024-05-09T04:35:20.676 INFO:teuthology.orchestra.run.smithi145.stdout: "osd": "allow *" 2024-05-09T04:35:20.676 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-05-09T04:35:20.676 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:35:20.676 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-05-09T04:35:20.677 INFO:teuthology.orchestra.run.smithi145.stdout: "entity": "osd.6", 2024-05-09T04:35:20.677 INFO:teuthology.orchestra.run.smithi145.stdout: "key": "AQAAUjxmSToZJxAAYIKW6x8/ARj4Nnz1tKgGQw==", 2024-05-09T04:35:20.677 INFO:teuthology.orchestra.run.smithi145.stdout: "caps": { 2024-05-09T04:35:20.677 INFO:teuthology.orchestra.run.smithi145.stdout: "mgr": "allow profile osd", 2024-05-09T04:35:20.677 INFO:teuthology.orchestra.run.smithi145.stdout: "mon": "allow profile osd", 2024-05-09T04:35:20.677 INFO:teuthology.orchestra.run.smithi145.stdout: "osd": "allow *" 2024-05-09T04:35:20.677 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-05-09T04:35:20.677 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:35:20.677 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-05-09T04:35:20.677 INFO:teuthology.orchestra.run.smithi145.stdout: "entity": "osd.7", 2024-05-09T04:35:20.677 INFO:teuthology.orchestra.run.smithi145.stdout: "key": "AQADUjxmkEIcMxAAn5KshgT0Ugl/LXKcIssEcQ==", 2024-05-09T04:35:20.677 INFO:teuthology.orchestra.run.smithi145.stdout: "caps": { 2024-05-09T04:35:20.677 INFO:teuthology.orchestra.run.smithi145.stdout: "mgr": "allow profile osd", 2024-05-09T04:35:20.677 INFO:teuthology.orchestra.run.smithi145.stdout: "mon": "allow profile osd", 2024-05-09T04:35:20.678 INFO:teuthology.orchestra.run.smithi145.stdout: "osd": "allow *" 2024-05-09T04:35:20.678 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-05-09T04:35:20.678 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:35:20.678 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-05-09T04:35:20.678 INFO:teuthology.orchestra.run.smithi145.stdout: "entity": "client.0", 2024-05-09T04:35:20.678 INFO:teuthology.orchestra.run.smithi145.stdout: "key": "AQDmUTxmfB7QDRAAs0TGpMOXJSerMXVrobw+hA==", 2024-05-09T04:35:20.678 INFO:teuthology.orchestra.run.smithi145.stdout: "caps": { 2024-05-09T04:35:20.678 INFO:teuthology.orchestra.run.smithi145.stdout: "mds": "allow", 2024-05-09T04:35:20.678 INFO:teuthology.orchestra.run.smithi145.stdout: "mon": "allow r", 2024-05-09T04:35:20.678 INFO:teuthology.orchestra.run.smithi145.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-05-09T04:35:20.678 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-05-09T04:35:20.678 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:35:20.678 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-05-09T04:35:20.678 INFO:teuthology.orchestra.run.smithi145.stdout: "entity": "client.1", 2024-05-09T04:35:20.678 INFO:teuthology.orchestra.run.smithi145.stdout: "key": "AQDmUTxmZoCtIBAAwylggR+NZPk4qXCIbv9i9g==", 2024-05-09T04:35:20.679 INFO:teuthology.orchestra.run.smithi145.stdout: "caps": { 2024-05-09T04:35:20.679 INFO:teuthology.orchestra.run.smithi145.stdout: "mds": "allow", 2024-05-09T04:35:20.679 INFO:teuthology.orchestra.run.smithi145.stdout: "mgr": "allow r", 2024-05-09T04:35:20.679 INFO:teuthology.orchestra.run.smithi145.stdout: "mon": "allow rw", 2024-05-09T04:35:20.679 INFO:teuthology.orchestra.run.smithi145.stdout: "osd": "allow rwx" 2024-05-09T04:35:20.679 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-05-09T04:35:20.679 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:35:20.679 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-05-09T04:35:20.679 INFO:teuthology.orchestra.run.smithi145.stdout: "entity": "client.2", 2024-05-09T04:35:20.679 INFO:teuthology.orchestra.run.smithi145.stdout: "key": "AQDmUTxmAn/iFhAAj7Wb0NYC8Z7KEMrsgrOkjA==", 2024-05-09T04:35:20.679 INFO:teuthology.orchestra.run.smithi145.stdout: "caps": { 2024-05-09T04:35:20.679 INFO:teuthology.orchestra.run.smithi145.stdout: "mds": "allow", 2024-05-09T04:35:20.679 INFO:teuthology.orchestra.run.smithi145.stdout: "mgr": "allow r", 2024-05-09T04:35:20.679 INFO:teuthology.orchestra.run.smithi145.stdout: "mon": "allow rw", 2024-05-09T04:35:20.681 INFO:teuthology.orchestra.run.smithi145.stdout: "osd": "allow rwx" 2024-05-09T04:35:20.681 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-05-09T04:35:20.682 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:35:20.682 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-05-09T04:35:20.682 INFO:teuthology.orchestra.run.smithi145.stdout: "entity": "client.3", 2024-05-09T04:35:20.682 INFO:teuthology.orchestra.run.smithi145.stdout: "key": "AQDmUTxmcUBaKhAA9XU8rgRmVEKjczR91U76hA==", 2024-05-09T04:35:20.682 INFO:teuthology.orchestra.run.smithi145.stdout: "caps": { 2024-05-09T04:35:20.682 INFO:teuthology.orchestra.run.smithi145.stdout: "mds": "allow", 2024-05-09T04:35:20.682 INFO:teuthology.orchestra.run.smithi145.stdout: "mgr": "allow r", 2024-05-09T04:35:20.682 INFO:teuthology.orchestra.run.smithi145.stdout: "mon": "allow rw", 2024-05-09T04:35:20.682 INFO:teuthology.orchestra.run.smithi145.stdout: "osd": "allow rwx" 2024-05-09T04:35:20.682 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-05-09T04:35:20.683 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:35:20.683 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-05-09T04:35:20.683 INFO:teuthology.orchestra.run.smithi145.stdout: "entity": "client.admin", 2024-05-09T04:35:20.683 INFO:teuthology.orchestra.run.smithi145.stdout: "key": "AQDkUTxmcfhNChAA3lkT8ApjnE77ulBRGugimg==", 2024-05-09T04:35:20.683 INFO:teuthology.orchestra.run.smithi145.stdout: "caps": { 2024-05-09T04:35:20.683 INFO:teuthology.orchestra.run.smithi145.stdout: "mds": "allow *", 2024-05-09T04:35:20.683 INFO:teuthology.orchestra.run.smithi145.stdout: "mgr": "allow *", 2024-05-09T04:35:20.683 INFO:teuthology.orchestra.run.smithi145.stdout: "mon": "allow *", 2024-05-09T04:35:20.683 INFO:teuthology.orchestra.run.smithi145.stdout: "osd": "allow *" 2024-05-09T04:35:20.683 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-05-09T04:35:20.683 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:35:20.683 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-05-09T04:35:20.683 INFO:teuthology.orchestra.run.smithi145.stdout: "entity": "mgr.x", 2024-05-09T04:35:20.683 INFO:teuthology.orchestra.run.smithi145.stdout: "key": "AQDlUTxmYdonBxAA9AoBwsKJCkAOsuTcY2zXHQ==", 2024-05-09T04:35:20.684 INFO:teuthology.orchestra.run.smithi145.stdout: "caps": { 2024-05-09T04:35:20.684 INFO:teuthology.orchestra.run.smithi145.stdout: "mds": "allow *", 2024-05-09T04:35:20.684 INFO:teuthology.orchestra.run.smithi145.stdout: "mon": "allow profile mgr", 2024-05-09T04:35:20.684 INFO:teuthology.orchestra.run.smithi145.stdout: "osd": "allow *" 2024-05-09T04:35:20.684 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-05-09T04:35:20.684 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:35:20.684 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-05-09T04:35:20.684 INFO:teuthology.orchestra.run.smithi145.stdout: "entity": "mgr.y", 2024-05-09T04:35:20.684 INFO:teuthology.orchestra.run.smithi145.stdout: "key": "AQDkUTxmSBenNxAAns0hdQXzIWDm3C86Jrc8gw==", 2024-05-09T04:35:20.684 INFO:teuthology.orchestra.run.smithi145.stdout: "caps": { 2024-05-09T04:35:20.684 INFO:teuthology.orchestra.run.smithi145.stdout: "mds": "allow *", 2024-05-09T04:35:20.684 INFO:teuthology.orchestra.run.smithi145.stdout: "mon": "allow profile mgr", 2024-05-09T04:35:20.684 INFO:teuthology.orchestra.run.smithi145.stdout: "osd": "allow *" 2024-05-09T04:35:20.684 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-05-09T04:35:20.684 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-05-09T04:35:20.684 INFO:teuthology.orchestra.run.smithi145.stdout: ] 2024-05-09T04:35:20.685 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-05-09T04:35:20.686 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.672+0000 7f333a400640 1 -- 172.21.15.145:0/3747183267 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f33140762d0 msgr2=0x7f3314078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:20.686 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.672+0000 7f333a400640 1 --2- 172.21.15.145:0/3747183267 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f33140762d0 0x7f3314078790 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3328000f60 tx=0x7f3328009040 comp rx=0 tx=0).stop 2024-05-09T04:35:20.686 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.672+0000 7f333a400640 1 -- 172.21.15.145:0/3747183267 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3334163000 msgr2=0x7f33341f59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:20.686 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.672+0000 7f333a400640 1 --2- 172.21.15.145:0/3747183267 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3334163000 0x7f33341f59d0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f33200098c0 tx=0x7f33200046c0 comp rx=0 tx=0).stop 2024-05-09T04:35:20.686 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.672+0000 7f333a400640 1 -- 172.21.15.145:0/3747183267 shutdown_connections 2024-05-09T04:35:20.686 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.672+0000 7f333a400640 1 --2- 172.21.15.145:0/3747183267 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f33140762d0 0x7f3314078790 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:20.686 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.672+0000 7f333a400640 1 --2- 172.21.15.145:0/3747183267 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33341633e0 0x7f333414b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:20.686 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.672+0000 7f333a400640 1 --2- 172.21.15.145:0/3747183267 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3334163000 0x7f33341f59d0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:20.686 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.672+0000 7f333a400640 1 --2- 172.21.15.145:0/3747183267 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f333406f1c0 0x7f3334158cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:20.686 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.672+0000 7f333a400640 1 -- 172.21.15.145:0/3747183267 >> 172.21.15.145:0/3747183267 conn(0x7f33340b3ae0 msgr2=0x7f333415ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:20.686 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.672+0000 7f333a400640 1 -- 172.21.15.145:0/3747183267 shutdown_connections 2024-05-09T04:35:20.686 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.673+0000 7f333a400640 1 -- 172.21.15.145:0/3747183267 wait complete. 2024-05-09T04:35:20.692 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-09T04:35:20.882 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.882+0000 7fc7c2800640 1 Processor -- start 2024-05-09T04:35:20.883 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.882+0000 7fc7c2800640 1 -- start start 2024-05-09T04:35:20.885 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.884+0000 7fc7c2800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc7bc15bae0 0x7fc7bc0721b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:20.885 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.884+0000 7fc7c2800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fc7bc0cfe30 0x7fc7bc0d02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:20.885 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.884+0000 7fc7c2800640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7bc15ffb0 con 0x7fc7bc15bec0 2024-05-09T04:35:20.885 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.884+0000 7fc7c2800640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fc7bc160840 con 0x7fc7bc0cfe30 2024-05-09T04:35:20.885 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.884+0000 7fc7c2800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fc7bc0b4190 con 0x7fc7bc15bae0 2024-05-09T04:35:20.885 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.884+0000 7fc7c1e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fc7bc0cfe30 0x7fc7bc0d02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:20.885 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.884+0000 7fc7c1e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fc7bc0cfe30 0x7fc7bc0d02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:52712/0 (socket says 172.21.15.145:52712) 2024-05-09T04:35:20.886 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.884+0000 7fc7c1e00640 1 -- 172.21.15.145:0/3607355163 learned_addr learned my addr 172.21.15.145:0/3607355163 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:20.887 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.885+0000 7fc7c1400640 1 --2- 172.21.15.145:0/3607355163 >> v2:172.21.15.192:3300/0 conn(0x7fc7bc15bae0 0x7fc7bc0721b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:20.887 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.885+0000 7fc7c1e00640 1 -- 172.21.15.145:0/3607355163 >> v2:172.21.15.192:3300/0 conn(0x7fc7bc15bae0 msgr2=0x7fc7bc0721b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:20.887 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.885+0000 7fc7c1e00640 1 --2- 172.21.15.145:0/3607355163 >> v2:172.21.15.192:3300/0 conn(0x7fc7bc15bae0 0x7fc7bc0721b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:20.888 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.885+0000 7fc7c1e00640 1 -- 172.21.15.145:0/3607355163 >> v1:172.21.15.192:6789/0 conn(0x7fc7bc15bec0 legacy=0x7fc7bc15f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:20.888 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.885+0000 7fc7c1e00640 1 -- 172.21.15.145:0/3607355163 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7bc160280 con 0x7fc7bc0cfe30 2024-05-09T04:35:20.888 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.886+0000 7fc7c1e00640 1 --2- 172.21.15.145:0/3607355163 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc7bc0cfe30 0x7fc7bc0d02c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc06eb80 tx=0x7fc7b402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c402b8408d159db1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:20.888 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.886+0000 7fc7bbe00640 1 -- 172.21.15.145:0/3607355163 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b4036070 con 0x7fc7bc0cfe30 2024-05-09T04:35:20.888 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.886+0000 7fc7bbe00640 1 -- 172.21.15.145:0/3607355163 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7b4031600 con 0x7fc7bc0cfe30 2024-05-09T04:35:20.888 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.886+0000 7fc7bbe00640 1 -- 172.21.15.145:0/3607355163 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b40319d0 con 0x7fc7bc0cfe30 2024-05-09T04:35:20.888 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.887+0000 7fc7c2800640 1 -- 172.21.15.145:0/3607355163 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc7bc0cfe30 msgr2=0x7fc7bc0d02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:20.888 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.887+0000 7fc7c2800640 1 --2- 172.21.15.145:0/3607355163 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc7bc0cfe30 0x7fc7bc0d02c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc06eb80 tx=0x7fc7b402a610 comp rx=0 tx=0).stop 2024-05-09T04:35:20.888 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.887+0000 7fc7c2800640 1 -- 172.21.15.145:0/3607355163 shutdown_connections 2024-05-09T04:35:20.888 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.887+0000 7fc7c2800640 1 --2- 172.21.15.145:0/3607355163 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc7bc0cfe30 0x7fc7bc0d02c0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:20.888 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.887+0000 7fc7c2800640 1 --2- 172.21.15.145:0/3607355163 >> v2:172.21.15.192:3300/0 conn(0x7fc7bc15bae0 0x7fc7bc0721b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:20.888 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.887+0000 7fc7c2800640 1 -- 172.21.15.145:0/3607355163 >> 172.21.15.145:0/3607355163 conn(0x7fc7bc06c8a0 msgr2=0x7fc7bc06ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:20.888 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.887+0000 7fc7c2800640 1 -- 172.21.15.145:0/3607355163 shutdown_connections 2024-05-09T04:35:20.889 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.887+0000 7fc7c2800640 1 -- 172.21.15.145:0/3607355163 wait complete. 2024-05-09T04:35:20.889 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.889+0000 7fc7c2800640 1 Processor -- start 2024-05-09T04:35:20.889 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.890+0000 7fc7c2800640 1 -- start start 2024-05-09T04:35:20.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.891+0000 7fc7c2800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc7bc0cfe30 0x7fc7bc14f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:20.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.891+0000 7fc7c2800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc7bc15bae0 0x7fc7bc157d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:20.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.891+0000 7fc7c2800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7bc15bec0 0x7fc7bc15e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:20.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.891+0000 7fc7c2800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc7bc0726f0 con 0x7fc7bc0cfe30 2024-05-09T04:35:20.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.891+0000 7fc7c2800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc7bc0b3e20 con 0x7fc7bc15bec0 2024-05-09T04:35:20.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.891+0000 7fc7c2800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc7bc0aded0 con 0x7fc7bc15bae0 2024-05-09T04:35:20.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.891+0000 7fc7c1400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc7bc0cfe30 0x7fc7bc14f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:20.893 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.891+0000 7fc7c1400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc7bc0cfe30 0x7fc7bc14f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:52724/0 (socket says 172.21.15.145:52724) 2024-05-09T04:35:20.893 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.891+0000 7fc7c1400640 1 -- 172.21.15.145:0/2503298131 learned_addr learned my addr 172.21.15.145:0/2503298131 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:20.893 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.891+0000 7fc7c1400640 1 -- 172.21.15.145:0/2503298131 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc7bc15bae0 msgr2=0x7fc7bc157d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:20.893 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.891+0000 7fc7c1400640 1 --2- 172.21.15.145:0/2503298131 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc7bc15bae0 0x7fc7bc157d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:20.893 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.891+0000 7fc7c1400640 1 -- 172.21.15.145:0/2503298131 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7bc15bec0 msgr2=0x7fc7bc15e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:20.893 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.891+0000 7fc7c1400640 1 --2- 172.21.15.145:0/2503298131 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7bc15bec0 0x7fc7bc15e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:20.893 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.891+0000 7fc7c1400640 1 -- 172.21.15.145:0/2503298131 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7bc158800 con 0x7fc7bc0cfe30 2024-05-09T04:35:20.893 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.892+0000 7fc7c1400640 1 --2- 172.21.15.145:0/2503298131 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc7bc0cfe30 0x7fc7bc14f360 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc7b000d800 tx=0x7fc7b000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:20.893 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.892+0000 7fc7b9600640 1 -- 172.21.15.145:0/2503298131 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b000b6c0 con 0x7fc7bc0cfe30 2024-05-09T04:35:20.893 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.892+0000 7fc7b9600640 1 -- 172.21.15.145:0/2503298131 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7b0019070 con 0x7fc7bc0cfe30 2024-05-09T04:35:20.893 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.892+0000 7fc7b9600640 1 -- 172.21.15.145:0/2503298131 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b000ba80 con 0x7fc7bc0cfe30 2024-05-09T04:35:20.894 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.892+0000 7fc7c2800640 1 -- 172.21.15.145:0/2503298131 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7bc158ae0 con 0x7fc7bc0cfe30 2024-05-09T04:35:20.894 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.892+0000 7fc7c2800640 1 -- 172.21.15.145:0/2503298131 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7bc1590a0 con 0x7fc7bc0cfe30 2024-05-09T04:35:20.894 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.893+0000 7fc7c2800640 1 -- 172.21.15.145:0/2503298131 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7bc06a310 con 0x7fc7bc0cfe30 2024-05-09T04:35:20.894 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.895+0000 7fc7b9600640 1 -- 172.21.15.145:0/2503298131 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc7b000bc20 con 0x7fc7bc0cfe30 2024-05-09T04:35:20.896 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.896+0000 7fc7b9600640 1 --2- 172.21.15.145:0/2503298131 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fc79c076300 0x7fc79c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:20.896 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.896+0000 7fc7b9600640 1 -- 172.21.15.145:0/2503298131 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4857+0+0 (secure 0 0 0) 0x7fc7b0097850 con 0x7fc7bc0cfe30 2024-05-09T04:35:20.896 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.896+0000 7fc7c0a00640 1 --2- 172.21.15.145:0/2503298131 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fc79c076300 0x7fc79c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:20.897 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.897+0000 7fc7c0a00640 1 --2- 172.21.15.145:0/2503298131 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fc79c076300 0x7fc79c0787c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc073420 tx=0x7fc7a80079d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:20.900 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:20.901+0000 7fc7b9600640 1 -- 172.21.15.145:0/2503298131 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc7b0061460 con 0x7fc7bc0cfe30 2024-05-09T04:35:21.112 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.112+0000 7fc7c2800640 1 -- 172.21.15.145:0/2503298131 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fc7bc070360 con 0x7fc7bc0cfe30 2024-05-09T04:35:21.112 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.113+0000 7fc7b9600640 1 -- 172.21.15.145:0/2503298131 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v43) ==== 84+0+65 (secure 0 0 0) 0x7fc7b0060d60 con 0x7fc7bc0cfe30 2024-05-09T04:35:21.113 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:35:21.113 INFO:teuthology.orchestra.run.smithi145.stdout:[ 2024-05-09T04:35:21.113 INFO:teuthology.orchestra.run.smithi145.stdout: { 2024-05-09T04:35:21.113 INFO:teuthology.orchestra.run.smithi145.stdout: "poolnum": 1, 2024-05-09T04:35:21.113 INFO:teuthology.orchestra.run.smithi145.stdout: "poolname": ".mgr" 2024-05-09T04:35:21.113 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-05-09T04:35:21.113 INFO:teuthology.orchestra.run.smithi145.stdout:] 2024-05-09T04:35:21.115 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.115+0000 7fc7c2800640 1 -- 172.21.15.145:0/2503298131 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fc79c076300 msgr2=0x7fc79c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:21.116 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.115+0000 7fc7c2800640 1 --2- 172.21.15.145:0/2503298131 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fc79c076300 0x7fc79c0787c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc073420 tx=0x7fc7a80079d0 comp rx=0 tx=0).stop 2024-05-09T04:35:21.116 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.115+0000 7fc7c2800640 1 -- 172.21.15.145:0/2503298131 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc7bc0cfe30 msgr2=0x7fc7bc14f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:21.116 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.115+0000 7fc7c2800640 1 --2- 172.21.15.145:0/2503298131 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc7bc0cfe30 0x7fc7bc14f360 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc7b000d800 tx=0x7fc7b000dcc0 comp rx=0 tx=0).stop 2024-05-09T04:35:21.116 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.115+0000 7fc7c2800640 1 -- 172.21.15.145:0/2503298131 shutdown_connections 2024-05-09T04:35:21.116 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.115+0000 7fc7c2800640 1 --2- 172.21.15.145:0/2503298131 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fc79c076300 0x7fc79c0787c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:21.116 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.115+0000 7fc7c2800640 1 --2- 172.21.15.145:0/2503298131 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7bc15bec0 0x7fc7bc15e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:21.116 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.115+0000 7fc7c2800640 1 --2- 172.21.15.145:0/2503298131 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc7bc15bae0 0x7fc7bc157d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:21.116 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.115+0000 7fc7c2800640 1 --2- 172.21.15.145:0/2503298131 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc7bc0cfe30 0x7fc7bc14f360 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:21.116 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.115+0000 7fc7c2800640 1 -- 172.21.15.145:0/2503298131 >> 172.21.15.145:0/2503298131 conn(0x7fc7bc06c8a0 msgr2=0x7fc7bc09e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:21.116 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.116+0000 7fc7c2800640 1 -- 172.21.15.145:0/2503298131 shutdown_connections 2024-05-09T04:35:21.116 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.116+0000 7fc7c2800640 1 -- 172.21.15.145:0/2503298131 wait complete. 2024-05-09T04:35:21.134 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-09T04:35:21.134 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-09T04:35:21.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.333+0000 7f6992000640 1 Processor -- start 2024-05-09T04:35:21.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.333+0000 7f6992000640 1 -- start start 2024-05-09T04:35:21.335 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.334+0000 7f6992000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f698c145860 0x7f698c15c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:21.335 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.334+0000 7f6992000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f698c15ce50 con 0x7f698c06ee60 2024-05-09T04:35:21.335 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.334+0000 7f6992000640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f698c15d130 con 0x7f698c0b3420 2024-05-09T04:35:21.335 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.334+0000 7f6992000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f698c15d410 con 0x7f698c145860 2024-05-09T04:35:21.335 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.335+0000 7f6991600640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f698c06ee60 0x7f698c0cfb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:55116/0 (socket says 172.21.15.145:55116) 2024-05-09T04:35:21.335 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.335+0000 7f6991600640 1 -- 172.21.15.145:0/237092423 learned_addr learned my addr 172.21.15.145:0/237092423 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:35:21.335 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.335+0000 7f6990c00640 1 --2- 172.21.15.145:0/237092423 >> v2:172.21.15.192:3300/0 conn(0x7f698c145860 0x7f698c15c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:21.335 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.335+0000 7f698b400640 1 -- 172.21.15.145:0/237092423 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3865502996 0 0) 0x7f698c15ce50 con 0x7f698c06ee60 2024-05-09T04:35:21.335 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.335+0000 7f698b400640 1 -- 172.21.15.145:0/237092423 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6970005730 con 0x7f698c06ee60 2024-05-09T04:35:21.337 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.336+0000 7f698b400640 1 -- 172.21.15.145:0/237092423 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3492529571 0 0) 0x7f6970005730 con 0x7f698c06ee60 2024-05-09T04:35:21.337 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.336+0000 7f698b400640 1 -- 172.21.15.145:0/237092423 >> v2:172.21.15.192:3300/0 conn(0x7f698c145860 msgr2=0x7f698c15c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:21.337 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.336+0000 7f698b400640 1 --2- 172.21.15.145:0/237092423 >> v2:172.21.15.192:3300/0 conn(0x7f698c145860 0x7f698c15c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:21.337 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.336+0000 7f698b400640 1 -- 172.21.15.145:0/237092423 >> v1:172.21.15.192:6789/0 conn(0x7f698c0b3420 legacy=0x7f698c072060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:21.337 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.336+0000 7f698b400640 1 -- 172.21.15.145:0/237092423 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f698c06e660 con 0x7f698c06ee60 2024-05-09T04:35:21.338 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.336+0000 7f698b400640 1 -- 172.21.15.145:0/237092423 <== mon.0 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3474413261 0 0) 0x7f69840016f0 con 0x7f698c06ee60 2024-05-09T04:35:21.338 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.336+0000 7f698b400640 1 -- 172.21.15.145:0/237092423 >> v1:172.21.15.145:6789/0 conn(0x7f698c06ee60 legacy=0x7f698c0cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:21.338 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.336+0000 7f698b400640 1 --2- 172.21.15.145:0/237092423 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6970005c60 0x7f69700060a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:21.338 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.336+0000 7f698b400640 1 --2- 172.21.15.145:0/237092423 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6970006a10 0x7f6970006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:21.338 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.336+0000 7f698b400640 1 --2- 172.21.15.145:0/237092423 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f69700074f0 0x7f697000dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:21.338 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.336+0000 7f698b400640 1 -- 172.21.15.145:0/237092423 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f698c15d410 con 0x7f69700074f0 2024-05-09T04:35:21.338 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.336+0000 7f698b400640 1 -- 172.21.15.145:0/237092423 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f697000e3c0 con 0x7f6970005c60 2024-05-09T04:35:21.338 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.336+0000 7f698b400640 1 -- 172.21.15.145:0/237092423 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f697000e5c0 con 0x7f6970006a10 2024-05-09T04:35:21.338 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.336+0000 7f6991600640 1 --2- 172.21.15.145:0/237092423 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f69700074f0 0x7f697000dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:21.338 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.336+0000 7f6990c00640 1 --2- 172.21.15.145:0/237092423 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6970005c60 0x7f69700060a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:21.338 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.336+0000 7f6991600640 1 -- 172.21.15.145:0/237092423 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6970006a10 msgr2=0x7f6970006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:21.338 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.336+0000 7f6991600640 1 --2- 172.21.15.145:0/237092423 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6970006a10 0x7f6970006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:21.338 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.336+0000 7f6991600640 1 -- 172.21.15.145:0/237092423 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6970005c60 msgr2=0x7f69700060a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:21.338 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.336+0000 7f6991600640 1 --2- 172.21.15.145:0/237092423 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6970005c60 0x7f69700060a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:21.339 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.336+0000 7f6991600640 1 -- 172.21.15.145:0/237092423 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6970005730 con 0x7f69700074f0 2024-05-09T04:35:21.339 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.337+0000 7f6991600640 1 -- 172.21.15.145:0/237092423 reap_dead start 2024-05-09T04:35:21.339 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.337+0000 7f6991600640 1 --2- 172.21.15.145:0/237092423 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f69700074f0 0x7f697000dbd0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f698400c240 tx=0x7f6970007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c592b2c9ade825e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:21.339 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.338+0000 7f698b400640 1 -- 172.21.15.145:0/237092423 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6984007ce0 con 0x7f69700074f0 2024-05-09T04:35:21.339 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.338+0000 7f698b400640 1 -- 172.21.15.145:0/237092423 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6984005b10 con 0x7f69700074f0 2024-05-09T04:35:21.339 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.338+0000 7f698b400640 1 -- 172.21.15.145:0/237092423 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6984007070 con 0x7f69700074f0 2024-05-09T04:35:21.340 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.338+0000 7f6992000640 1 -- 172.21.15.145:0/237092423 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f69700074f0 msgr2=0x7f697000dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:21.340 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.338+0000 7f6992000640 1 --2- 172.21.15.145:0/237092423 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f69700074f0 0x7f697000dbd0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f698400c240 tx=0x7f6970007a40 comp rx=0 tx=0).stop 2024-05-09T04:35:21.340 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.338+0000 7f6992000640 1 -- 172.21.15.145:0/237092423 shutdown_connections 2024-05-09T04:35:21.340 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.338+0000 7f6992000640 1 --2- 172.21.15.145:0/237092423 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f69700074f0 0x7f697000dbd0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:21.340 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.338+0000 7f6992000640 1 -- 172.21.15.145:0/237092423 >> 172.21.15.145:0/237092423 conn(0x7f698c0b2850 msgr2=0x7f698c0b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:21.340 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.338+0000 7f6992000640 1 -- 172.21.15.145:0/237092423 shutdown_connections 2024-05-09T04:35:21.340 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.339+0000 7f6992000640 1 -- 172.21.15.145:0/237092423 wait complete. 2024-05-09T04:35:21.340 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.340+0000 7f6992000640 1 Processor -- start 2024-05-09T04:35:21.340 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.340+0000 7f6992000640 1 -- start start 2024-05-09T04:35:21.342 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.341+0000 7f6992000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f69700074f0 0x7f698c15b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:21.342 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.341+0000 7f6992000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f698c0ac880 0x7f698c0acd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:21.342 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.341+0000 7f6992000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f698c0aa860 0x7f698c0aace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:21.342 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.341+0000 7f6992000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f698c06f620 con 0x7f698c0aa860 2024-05-09T04:35:21.342 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.341+0000 7f6992000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f698c06f4a0 con 0x7f698c0ac880 2024-05-09T04:35:21.343 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.341+0000 7f6992000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f698c06f7a0 con 0x7f69700074f0 2024-05-09T04:35:21.343 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.342+0000 7f6991600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f698c0aa860 0x7f698c0aace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:21.343 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.342+0000 7f6991600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f698c0aa860 0x7f698c0aace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:52748/0 (socket says 172.21.15.145:52748) 2024-05-09T04:35:21.343 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.342+0000 7f6991600640 1 -- 172.21.15.145:0/4113098595 learned_addr learned my addr 172.21.15.145:0/4113098595 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:21.343 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.342+0000 7f698be00640 1 --2- 172.21.15.145:0/4113098595 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f698c0ac880 0x7f698c0acd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:21.343 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.342+0000 7f6990c00640 1 --2- 172.21.15.145:0/4113098595 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f69700074f0 0x7f698c15b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:21.343 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.342+0000 7f6991600640 1 -- 172.21.15.145:0/4113098595 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f69700074f0 msgr2=0x7f698c15b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:21.343 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.342+0000 7f6991600640 1 --2- 172.21.15.145:0/4113098595 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f69700074f0 0x7f698c15b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:21.343 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.342+0000 7f6991600640 1 -- 172.21.15.145:0/4113098595 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f698c0ac880 msgr2=0x7f698c0acd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:21.343 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.342+0000 7f6991600640 1 --2- 172.21.15.145:0/4113098595 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f698c0ac880 0x7f698c0acd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:21.343 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.342+0000 7f6991600640 1 -- 172.21.15.145:0/4113098595 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f698c143540 con 0x7f698c0aa860 2024-05-09T04:35:21.343 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.342+0000 7f6991600640 1 --2- 172.21.15.145:0/4113098595 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f698c0aa860 0x7f698c0aace0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f698402ad50 tx=0x7f698c15d300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:21.343 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.343+0000 7f6988c00640 1 -- 172.21.15.145:0/4113098595 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6984001c20 con 0x7f698c0aa860 2024-05-09T04:35:21.343 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.343+0000 7f6988c00640 1 -- 172.21.15.145:0/4113098595 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6984040070 con 0x7f698c0aa860 2024-05-09T04:35:21.344 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.343+0000 7f6988c00640 1 -- 172.21.15.145:0/4113098595 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69840049e0 con 0x7f698c0aa860 2024-05-09T04:35:21.344 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.343+0000 7f6992000640 1 -- 172.21.15.145:0/4113098595 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f698c143820 con 0x7f698c0aa860 2024-05-09T04:35:21.344 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.343+0000 7f6992000640 1 -- 172.21.15.145:0/4113098595 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f698c143de0 con 0x7f698c0aa860 2024-05-09T04:35:21.344 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.344+0000 7f6992000640 1 -- 172.21.15.145:0/4113098595 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f698c0b4cd0 con 0x7f698c0aa860 2024-05-09T04:35:21.345 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.345+0000 7f6988c00640 1 -- 172.21.15.145:0/4113098595 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f698403a070 con 0x7f698c0aa860 2024-05-09T04:35:21.346 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.346+0000 7f6988c00640 1 --2- 172.21.15.145:0/4113098595 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f696c0762d0 0x7f696c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:21.346 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.346+0000 7f6988c00640 1 -- 172.21.15.145:0/4113098595 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4857+0+0 (secure 0 0 0) 0x7f69840b6a20 con 0x7f698c0aa860 2024-05-09T04:35:21.346 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.347+0000 7f6990c00640 1 --2- 172.21.15.145:0/4113098595 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f696c0762d0 0x7f696c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:21.347 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.347+0000 7f6990c00640 1 --2- 172.21.15.145:0/4113098595 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f696c0762d0 0x7f696c078790 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f697c0074a0 tx=0x7f697c00d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:21.350 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.351+0000 7f6988c00640 1 -- 172.21.15.145:0/4113098595 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f698c0b4cd0 con 0x7f698c0aa860 2024-05-09T04:35:21.553 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:21.553+0000 7f6992000640 1 -- 172.21.15.145:0/4113098595 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f698c1441c0 con 0x7f698c0aa860 2024-05-09T04:35:22.182 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.182+0000 7f6988c00640 1 -- 172.21.15.145:0/4113098595 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v44) ==== 138+0+0 (secure 0 0 0) 0x7f69840805a0 con 0x7f698c0aa860 2024-05-09T04:35:22.182 INFO:teuthology.orchestra.run.smithi145.stderr:pool 'cephfs_metadata' created 2024-05-09T04:35:22.185 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.186+0000 7f6992000640 1 -- 172.21.15.145:0/4113098595 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f696c0762d0 msgr2=0x7f696c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:22.186 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.186+0000 7f6992000640 1 --2- 172.21.15.145:0/4113098595 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f696c0762d0 0x7f696c078790 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f697c0074a0 tx=0x7f697c00d040 comp rx=0 tx=0).stop 2024-05-09T04:35:22.187 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.186+0000 7f6992000640 1 -- 172.21.15.145:0/4113098595 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f698c0aa860 msgr2=0x7f698c0aace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:22.187 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.186+0000 7f6992000640 1 --2- 172.21.15.145:0/4113098595 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f698c0aa860 0x7f698c0aace0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f698402ad50 tx=0x7f698c15d300 comp rx=0 tx=0).stop 2024-05-09T04:35:22.187 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.186+0000 7f6992000640 1 -- 172.21.15.145:0/4113098595 shutdown_connections 2024-05-09T04:35:22.187 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.186+0000 7f6992000640 1 --2- 172.21.15.145:0/4113098595 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f696c0762d0 0x7f696c078790 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:22.187 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.186+0000 7f6992000640 1 --2- 172.21.15.145:0/4113098595 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f698c0aa860 0x7f698c0aace0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:22.187 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.186+0000 7f6992000640 1 --2- 172.21.15.145:0/4113098595 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f698c0ac880 0x7f698c0acd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:22.187 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.186+0000 7f6992000640 1 --2- 172.21.15.145:0/4113098595 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f69700074f0 0x7f698c15b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:22.187 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.186+0000 7f6992000640 1 -- 172.21.15.145:0/4113098595 >> 172.21.15.145:0/4113098595 conn(0x7f698c0b2850 msgr2=0x7f698c0d00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:22.187 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.186+0000 7f6992000640 1 -- 172.21.15.145:0/4113098595 shutdown_connections 2024-05-09T04:35:22.187 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.186+0000 7f6992000640 1 -- 172.21.15.145:0/4113098595 wait complete. 2024-05-09T04:35:22.212 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-09T04:35:22.397 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.397+0000 7fb2bfa00640 1 Processor -- start 2024-05-09T04:35:22.397 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.397+0000 7fb2bfa00640 1 -- start start 2024-05-09T04:35:22.399 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.399+0000 7fb2bfa00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb2b80d0920 0x7fb2b80734b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:22.399 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.399+0000 7fb2bfa00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fb2b815f220 0x7fb2b815c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:22.399 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.399+0000 7fb2bfa00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb2b806e450 0x7fb2b806e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:22.399 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.399+0000 7fb2bfa00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fb2b806ee00 con 0x7fb2b815f220 2024-05-09T04:35:22.399 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.399+0000 7fb2bfa00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fb2b80b6090 con 0x7fb2b80d0920 2024-05-09T04:35:22.399 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.399+0000 7fb2bfa00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb2b8156130 con 0x7fb2b806e450 2024-05-09T04:35:22.400 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.400+0000 7fb2b7e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fb2b815f220 0x7fb2b815c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:22.400 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.400+0000 7fb2b7e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fb2b815f220 0x7fb2b815c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:52756/0 (socket says 172.21.15.145:52756) 2024-05-09T04:35:22.400 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.400+0000 7fb2b7e00640 1 -- 172.21.15.145:0/598613606 learned_addr learned my addr 172.21.15.145:0/598613606 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:22.400 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.400+0000 7fb2bda00640 1 --2- 172.21.15.145:0/598613606 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb2b806e450 0x7fb2b806e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:22.400 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.400+0000 7fb2bd000640 1 --2- 172.21.15.145:0/598613606 >> v2:172.21.15.192:3300/0 conn(0x7fb2b80d0920 0x7fb2b80734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:22.400 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.400+0000 7fb2b7e00640 1 -- 172.21.15.145:0/598613606 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb2b806e450 msgr2=0x7fb2b806e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:22.400 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.400+0000 7fb2b7e00640 1 --2- 172.21.15.145:0/598613606 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb2b806e450 0x7fb2b806e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:22.401 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.400+0000 7fb2b7e00640 1 -- 172.21.15.145:0/598613606 >> v2:172.21.15.192:3300/0 conn(0x7fb2b80d0920 msgr2=0x7fb2b80734b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:22.401 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.400+0000 7fb2b7e00640 1 --2- 172.21.15.145:0/598613606 >> v2:172.21.15.192:3300/0 conn(0x7fb2b80d0920 0x7fb2b80734b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:22.401 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.400+0000 7fb2b7e00640 1 -- 172.21.15.145:0/598613606 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2b815cbf0 con 0x7fb2b815f220 2024-05-09T04:35:22.401 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.401+0000 7fb2b7e00640 1 --2- 172.21.15.145:0/598613606 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb2b815f220 0x7fb2b815c6b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fb2b80703d0 tx=0x7fb2a802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=295299f1b6a7c1fe server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:22.401 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.402+0000 7fb2b7400640 1 -- 172.21.15.145:0/598613606 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a8037070 con 0x7fb2b815f220 2024-05-09T04:35:22.401 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.402+0000 7fb2b7400640 1 -- 172.21.15.145:0/598613606 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2a8032440 con 0x7fb2b815f220 2024-05-09T04:35:22.402 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.402+0000 7fb2b7400640 1 -- 172.21.15.145:0/598613606 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a8032810 con 0x7fb2b815f220 2024-05-09T04:35:22.402 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.402+0000 7fb2bfa00640 1 -- 172.21.15.145:0/598613606 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb2b815f220 msgr2=0x7fb2b815c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:22.403 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.402+0000 7fb2bfa00640 1 --2- 172.21.15.145:0/598613606 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb2b815f220 0x7fb2b815c6b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fb2b80703d0 tx=0x7fb2a802a250 comp rx=0 tx=0).stop 2024-05-09T04:35:22.403 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.402+0000 7fb2bfa00640 1 -- 172.21.15.145:0/598613606 shutdown_connections 2024-05-09T04:35:22.403 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.402+0000 7fb2bfa00640 1 --2- 172.21.15.145:0/598613606 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb2b806e450 0x7fb2b806e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:22.403 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.402+0000 7fb2bfa00640 1 --2- 172.21.15.145:0/598613606 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb2b815f220 0x7fb2b815c6b0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:22.403 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.402+0000 7fb2bfa00640 1 --2- 172.21.15.145:0/598613606 >> v2:172.21.15.192:3300/0 conn(0x7fb2b80d0920 0x7fb2b80734b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:22.403 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.402+0000 7fb2bfa00640 1 -- 172.21.15.145:0/598613606 >> 172.21.15.145:0/598613606 conn(0x7fb2b80b6600 msgr2=0x7fb2b80b6a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:22.403 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.402+0000 7fb2bfa00640 1 -- 172.21.15.145:0/598613606 shutdown_connections 2024-05-09T04:35:22.403 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.402+0000 7fb2bfa00640 1 -- 172.21.15.145:0/598613606 wait complete. 2024-05-09T04:35:22.404 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.404+0000 7fb2bfa00640 1 Processor -- start 2024-05-09T04:35:22.404 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.404+0000 7fb2bfa00640 1 -- start start 2024-05-09T04:35:22.407 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.405+0000 7fb2bfa00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb2b806e450 0x7fb2b8165c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:22.407 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.405+0000 7fb2bfa00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2b80d0920 0x7fb2b8151040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:22.407 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.405+0000 7fb2bfa00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb2b815f220 0x7fb2b814fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:22.407 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.405+0000 7fb2bfa00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b815fc60 con 0x7fb2b815f220 2024-05-09T04:35:22.407 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.405+0000 7fb2bfa00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b815f980 con 0x7fb2b80d0920 2024-05-09T04:35:22.408 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.405+0000 7fb2bfa00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb2b815ff40 con 0x7fb2b806e450 2024-05-09T04:35:22.408 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.405+0000 7fb2bda00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb2b815f220 0x7fb2b814fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:22.408 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.405+0000 7fb2bda00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb2b815f220 0x7fb2b814fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:52772/0 (socket says 172.21.15.145:52772) 2024-05-09T04:35:22.408 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.405+0000 7fb2bda00640 1 -- 172.21.15.145:0/2835327246 learned_addr learned my addr 172.21.15.145:0/2835327246 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:22.408 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.406+0000 7fb2b7e00640 1 --2- 172.21.15.145:0/2835327246 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2b80d0920 0x7fb2b8151040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:22.408 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.406+0000 7fb2bd000640 1 --2- 172.21.15.145:0/2835327246 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb2b806e450 0x7fb2b8165c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:22.408 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.406+0000 7fb2bda00640 1 -- 172.21.15.145:0/2835327246 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb2b806e450 msgr2=0x7fb2b8165c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:22.409 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.406+0000 7fb2bda00640 1 --2- 172.21.15.145:0/2835327246 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb2b806e450 0x7fb2b8165c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:22.409 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.406+0000 7fb2bda00640 1 -- 172.21.15.145:0/2835327246 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2b80d0920 msgr2=0x7fb2b8151040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:22.409 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.406+0000 7fb2bda00640 1 --2- 172.21.15.145:0/2835327246 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2b80d0920 0x7fb2b8151040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:22.409 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.406+0000 7fb2bda00640 1 -- 172.21.15.145:0/2835327246 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2b81f8690 con 0x7fb2b815f220 2024-05-09T04:35:22.409 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.406+0000 7fb2bda00640 1 --2- 172.21.15.145:0/2835327246 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb2b815f220 0x7fb2b814fd70 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0007e70 tx=0x7fb2b000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:22.409 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.406+0000 7fb2b4c00640 1 -- 172.21.15.145:0/2835327246 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b0014620 con 0x7fb2b815f220 2024-05-09T04:35:22.410 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.406+0000 7fb2b4c00640 1 -- 172.21.15.145:0/2835327246 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2b0019070 con 0x7fb2b815f220 2024-05-09T04:35:22.410 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.406+0000 7fb2bfa00640 1 -- 172.21.15.145:0/2835327246 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2b81f8970 con 0x7fb2b815f220 2024-05-09T04:35:22.410 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.406+0000 7fb2b4c00640 1 -- 172.21.15.145:0/2835327246 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b0014a20 con 0x7fb2b815f220 2024-05-09T04:35:22.410 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.407+0000 7fb2bfa00640 1 -- 172.21.15.145:0/2835327246 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2b81f8f30 con 0x7fb2b815f220 2024-05-09T04:35:22.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.408+0000 7fb2bfa00640 1 -- 172.21.15.145:0/2835327246 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2b806a310 con 0x7fb2b815f220 2024-05-09T04:35:22.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.409+0000 7fb2b4c00640 1 -- 172.21.15.145:0/2835327246 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb2b0014bc0 con 0x7fb2b815f220 2024-05-09T04:35:22.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.410+0000 7fb2b4c00640 1 --2- 172.21.15.145:0/2835327246 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fb29c0762d0 0x7fb29c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:22.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.411+0000 7fb2b4c00640 1 -- 172.21.15.145:0/2835327246 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5227+0+0 (secure 0 0 0) 0x7fb2b0097d20 con 0x7fb2b815f220 2024-05-09T04:35:22.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.411+0000 7fb2bd000640 1 --2- 172.21.15.145:0/2835327246 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fb29c0762d0 0x7fb29c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:22.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.411+0000 7fb2bd000640 1 --2- 172.21.15.145:0/2835327246 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fb29c0762d0 0x7fb29c078790 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac006fd0 tx=0x7fb2ac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:22.416 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.416+0000 7fb2b4c00640 1 -- 172.21.15.145:0/2835327246 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb2b0061860 con 0x7fb2b815f220 2024-05-09T04:35:22.622 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:22.623+0000 7fb2bfa00640 1 -- 172.21.15.145:0/2835327246 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fb2b80b55d0 con 0x7fb2b815f220 2024-05-09T04:35:23.189 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.189+0000 7fb2b4c00640 1 -- 172.21.15.145:0/2835327246 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v45) ==== 170+0+0 (secure 0 0 0) 0x7fb2b0061160 con 0x7fb2b815f220 2024-05-09T04:35:23.189 INFO:teuthology.orchestra.run.smithi145.stderr:pool 'cephfs_data' created 2024-05-09T04:35:23.194 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.194+0000 7fb2bfa00640 1 -- 172.21.15.145:0/2835327246 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fb29c0762d0 msgr2=0x7fb29c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:23.194 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.194+0000 7fb2bfa00640 1 --2- 172.21.15.145:0/2835327246 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fb29c0762d0 0x7fb29c078790 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac006fd0 tx=0x7fb2ac008040 comp rx=0 tx=0).stop 2024-05-09T04:35:23.194 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.194+0000 7fb2bfa00640 1 -- 172.21.15.145:0/2835327246 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb2b815f220 msgr2=0x7fb2b814fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:23.194 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.194+0000 7fb2bfa00640 1 --2- 172.21.15.145:0/2835327246 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb2b815f220 0x7fb2b814fd70 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0007e70 tx=0x7fb2b000e4b0 comp rx=0 tx=0).stop 2024-05-09T04:35:23.194 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.194+0000 7fb2bfa00640 1 -- 172.21.15.145:0/2835327246 shutdown_connections 2024-05-09T04:35:23.194 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.194+0000 7fb2bfa00640 1 --2- 172.21.15.145:0/2835327246 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fb29c0762d0 0x7fb29c078790 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:23.194 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.194+0000 7fb2bfa00640 1 --2- 172.21.15.145:0/2835327246 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb2b815f220 0x7fb2b814fd70 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:23.194 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.194+0000 7fb2bfa00640 1 --2- 172.21.15.145:0/2835327246 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2b80d0920 0x7fb2b8151040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:23.194 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.194+0000 7fb2bfa00640 1 --2- 172.21.15.145:0/2835327246 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb2b806e450 0x7fb2b8165c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:23.195 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.194+0000 7fb2bfa00640 1 -- 172.21.15.145:0/2835327246 >> 172.21.15.145:0/2835327246 conn(0x7fb2b80b6600 msgr2=0x7fb2b8155450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:23.195 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.195+0000 7fb2bfa00640 1 -- 172.21.15.145:0/2835327246 shutdown_connections 2024-05-09T04:35:23.195 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.195+0000 7fb2bfa00640 1 -- 172.21.15.145:0/2835327246 wait complete. 2024-05-09T04:35:23.232 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-09T04:35:23.406 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.406+0000 7faf15200640 1 Processor -- start 2024-05-09T04:35:23.406 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.407+0000 7faf15200640 1 -- start start 2024-05-09T04:35:23.408 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.408+0000 7faf15200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7faf1014d2c0 0x7faf100b4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:23.408 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.408+0000 7faf15200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf100b4fe0 con 0x7faf10072be0 2024-05-09T04:35:23.408 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.408+0000 7faf15200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf100b52c0 con 0x7faf100b4a60 2024-05-09T04:35:23.408 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.408+0000 7faf15200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7faf1006e4c0 con 0x7faf1014d2c0 2024-05-09T04:35:23.409 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.409+0000 7faf0e800640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7faf10072be0 0x7faf1006e950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:55128/0 (socket says 172.21.15.145:55128) 2024-05-09T04:35:23.409 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.409+0000 7faf0e800640 1 -- 172.21.15.145:0/1188733828 learned_addr learned my addr 172.21.15.145:0/1188733828 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:35:23.411 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.410+0000 7faf0de00640 1 --2- 172.21.15.145:0/1188733828 >> v2:172.21.15.192:3300/0 conn(0x7faf1014d2c0 0x7faf100b4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:23.411 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.410+0000 7faf0ca00640 1 -- 172.21.15.145:0/1188733828 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2851927327 0 0) 0x7faf100b4fe0 con 0x7faf10072be0 2024-05-09T04:35:23.411 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.410+0000 7faf0ca00640 1 -- 172.21.15.145:0/1188733828 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faef4003520 con 0x7faf10072be0 2024-05-09T04:35:23.411 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.410+0000 7faf0ca00640 1 -- 172.21.15.145:0/1188733828 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1560201531 0 0) 0x7faf100b52c0 con 0x7faf100b4a60 2024-05-09T04:35:23.411 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.410+0000 7faf0ca00640 1 -- 172.21.15.145:0/1188733828 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faf100b4fe0 con 0x7faf100b4a60 2024-05-09T04:35:23.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.410+0000 7faf0ca00640 1 -- 172.21.15.145:0/1188733828 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1199982357 0 0) 0x7faef4003520 con 0x7faf10072be0 2024-05-09T04:35:23.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.410+0000 7faf0ca00640 1 -- 172.21.15.145:0/1188733828 >> v2:172.21.15.192:3300/0 conn(0x7faf1014d2c0 msgr2=0x7faf100b4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:23.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.410+0000 7faf0ca00640 1 --2- 172.21.15.145:0/1188733828 >> v2:172.21.15.192:3300/0 conn(0x7faf1014d2c0 0x7faf100b4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:23.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.410+0000 7faf0ca00640 1 -- 172.21.15.145:0/1188733828 >> v1:172.21.15.192:6789/0 conn(0x7faf100b4a60 legacy=0x7faf100724d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:23.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.410+0000 7faf0ca00640 1 -- 172.21.15.145:0/1188733828 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf100b55a0 con 0x7faf10072be0 2024-05-09T04:35:23.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.410+0000 7faf0ca00640 1 -- 172.21.15.145:0/1188733828 <== mon.0 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3474413261 0 0) 0x7faf08001a70 con 0x7faf10072be0 2024-05-09T04:35:23.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.410+0000 7faf0ca00640 1 -- 172.21.15.145:0/1188733828 >> v1:172.21.15.145:6789/0 conn(0x7faf10072be0 legacy=0x7faf1006e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:23.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.410+0000 7faf0ca00640 1 --2- 172.21.15.145:0/1188733828 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faef4003c30 0x7faef4004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:23.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.410+0000 7faf0ca00640 1 --2- 172.21.15.145:0/1188733828 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faef4004800 0x7faef4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:23.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.410+0000 7faf0ca00640 1 --2- 172.21.15.145:0/1188733828 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faef4005280 0x7faef400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:23.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.410+0000 7faf0ca00640 1 -- 172.21.15.145:0/1188733828 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7faf1006e4c0 con 0x7faef4005280 2024-05-09T04:35:23.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.410+0000 7faf0ca00640 1 -- 172.21.15.145:0/1188733828 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7faef400c1c0 con 0x7faef4004800 2024-05-09T04:35:23.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.410+0000 7faf0ca00640 1 -- 172.21.15.145:0/1188733828 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7faef400c3c0 con 0x7faef4003c30 2024-05-09T04:35:23.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.411+0000 7faf0e800640 1 --2- 172.21.15.145:0/1188733828 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faef4005280 0x7faef400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:23.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.411+0000 7faf0d400640 1 --2- 172.21.15.145:0/1188733828 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faef4004800 0x7faef4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:23.413 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.411+0000 7faf0de00640 1 --2- 172.21.15.145:0/1188733828 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faef4003c30 0x7faef4004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:23.413 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.411+0000 7faf0e800640 1 -- 172.21.15.145:0/1188733828 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faef4003c30 msgr2=0x7faef4004030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:23.413 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.411+0000 7faf0e800640 1 --2- 172.21.15.145:0/1188733828 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faef4003c30 0x7faef4004030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:23.413 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.411+0000 7faf0e800640 1 -- 172.21.15.145:0/1188733828 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faef4004800 msgr2=0x7faef4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:23.413 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.411+0000 7faf0e800640 1 --2- 172.21.15.145:0/1188733828 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faef4004800 0x7faef4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:23.413 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.411+0000 7faf0e800640 1 -- 172.21.15.145:0/1188733828 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faef4003520 con 0x7faef4005280 2024-05-09T04:35:23.413 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.411+0000 7faf0d400640 1 --2- 172.21.15.145:0/1188733828 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faef4004800 0x7faef4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:35:23.414 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.412+0000 7faf0e800640 1 -- 172.21.15.145:0/1188733828 reap_dead start 2024-05-09T04:35:23.414 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.412+0000 7faf0e800640 1 --2- 172.21.15.145:0/1188733828 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faef4005280 0x7faef400b9c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7faf100721b0 tx=0x7faef4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac4bb5250b97099e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:23.414 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.412+0000 7faf0ca00640 1 -- 172.21.15.145:0/1188733828 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0800a430 con 0x7faef4005280 2024-05-09T04:35:23.414 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.413+0000 7faf0ca00640 1 -- 172.21.15.145:0/1188733828 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf0802f070 con 0x7faef4005280 2024-05-09T04:35:23.414 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.413+0000 7faf0ca00640 1 -- 172.21.15.145:0/1188733828 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0802bb40 con 0x7faef4005280 2024-05-09T04:35:23.414 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.413+0000 7faf15200640 1 -- 172.21.15.145:0/1188733828 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faef4005280 msgr2=0x7faef400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:23.414 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.413+0000 7faf15200640 1 --2- 172.21.15.145:0/1188733828 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faef4005280 0x7faef400b9c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7faf100721b0 tx=0x7faef4005830 comp rx=0 tx=0).stop 2024-05-09T04:35:23.414 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.413+0000 7faf15200640 1 -- 172.21.15.145:0/1188733828 shutdown_connections 2024-05-09T04:35:23.415 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.413+0000 7faf15200640 1 --2- 172.21.15.145:0/1188733828 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faef4005280 0x7faef400b9c0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:23.415 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.413+0000 7faf15200640 1 -- 172.21.15.145:0/1188733828 >> 172.21.15.145:0/1188733828 conn(0x7faf1006cab0 msgr2=0x7faf100d0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:23.415 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.413+0000 7faf15200640 1 -- 172.21.15.145:0/1188733828 shutdown_connections 2024-05-09T04:35:23.415 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.413+0000 7faf15200640 1 -- 172.21.15.145:0/1188733828 wait complete. 2024-05-09T04:35:23.415 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.415+0000 7faf15200640 1 Processor -- start 2024-05-09T04:35:23.415 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.415+0000 7faf15200640 1 -- start start 2024-05-09T04:35:23.416 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.416+0000 7faf15200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faef4005280 0x7faf101473a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:23.416 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.416+0000 7faf15200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf10149010 0x7faf10161610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:23.416 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.416+0000 7faf15200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faf1014a820 0x7faf1014ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:23.416 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.416+0000 7faf15200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7faf100b4520 con 0x7faef4005280 2024-05-09T04:35:23.416 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.416+0000 7faf15200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7faf10167590 con 0x7faf10149010 2024-05-09T04:35:23.416 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.416+0000 7faf15200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7faf100b46a0 con 0x7faf1014a820 2024-05-09T04:35:23.416 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.417+0000 7faf0de00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faef4005280 0x7faf101473a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:23.417 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.417+0000 7faf0de00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faef4005280 0x7faf101473a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:52800/0 (socket says 172.21.15.145:52800) 2024-05-09T04:35:23.417 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.417+0000 7faf0de00640 1 -- 172.21.15.145:0/1286051878 learned_addr learned my addr 172.21.15.145:0/1286051878 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:23.418 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.417+0000 7faf0d400640 1 --2- 172.21.15.145:0/1286051878 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf10149010 0x7faf10161610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:23.418 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.417+0000 7faf0e800640 1 --2- 172.21.15.145:0/1286051878 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faf1014a820 0x7faf1014ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:23.418 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.417+0000 7faf0de00640 1 -- 172.21.15.145:0/1286051878 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faf1014a820 msgr2=0x7faf1014ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:23.418 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.417+0000 7faf0de00640 1 --2- 172.21.15.145:0/1286051878 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faf1014a820 0x7faf1014ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:23.418 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.417+0000 7faf0de00640 1 -- 172.21.15.145:0/1286051878 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf10149010 msgr2=0x7faf10161610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:23.418 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.417+0000 7faf0de00640 1 --2- 172.21.15.145:0/1286051878 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf10149010 0x7faf10161610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:23.418 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.417+0000 7faf0de00640 1 -- 172.21.15.145:0/1286051878 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf10149780 con 0x7faef4005280 2024-05-09T04:35:23.418 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.417+0000 7faf0e800640 1 --2- 172.21.15.145:0/1286051878 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faf1014a820 0x7faf1014ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:35:23.418 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.417+0000 7faf0de00640 1 --2- 172.21.15.145:0/1286051878 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faef4005280 0x7faf101473a0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7faf00010860 tx=0x7faf00010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:23.419 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.418+0000 7faf06000640 1 -- 172.21.15.145:0/1286051878 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0000f040 con 0x7faef4005280 2024-05-09T04:35:23.419 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.418+0000 7faf06000640 1 -- 172.21.15.145:0/1286051878 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf0000eb60 con 0x7faef4005280 2024-05-09T04:35:23.419 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.418+0000 7faf06000640 1 -- 172.21.15.145:0/1286051878 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0001c400 con 0x7faef4005280 2024-05-09T04:35:23.419 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.418+0000 7faf15200640 1 -- 172.21.15.145:0/1286051878 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf10149a60 con 0x7faef4005280 2024-05-09T04:35:23.419 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.418+0000 7faf15200640 1 -- 172.21.15.145:0/1286051878 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf1014a020 con 0x7faef4005280 2024-05-09T04:35:23.419 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.420+0000 7faf15200640 1 -- 172.21.15.145:0/1286051878 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf1006a310 con 0x7faef4005280 2024-05-09T04:35:23.420 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.420+0000 7faf06000640 1 -- 172.21.15.145:0/1286051878 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7faf0001c5a0 con 0x7faef4005280 2024-05-09T04:35:23.422 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.421+0000 7faf06000640 1 --2- 172.21.15.145:0/1286051878 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7faef0076300 0x7faef00787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:23.422 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.422+0000 7faf06000640 1 -- 172.21.15.145:0/1286051878 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5609+0+0 (secure 0 0 0) 0x7faf00097ca0 con 0x7faef4005280 2024-05-09T04:35:23.422 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.422+0000 7faf0d400640 1 --2- 172.21.15.145:0/1286051878 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7faef0076300 0x7faef00787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:23.422 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.422+0000 7faf0d400640 1 --2- 172.21.15.145:0/1286051878 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7faef0076300 0x7faef00787c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7faf100cf4f0 tx=0x7faefc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:23.426 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.426+0000 7faf06000640 1 -- 172.21.15.145:0/1286051878 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7faf00018030 con 0x7faef4005280 2024-05-09T04:35:23.665 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:23.665+0000 7faf15200640 1 -- 172.21.15.145:0/1286051878 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7faf100b40d0 con 0x7faef4005280 2024-05-09T04:35:24.213 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.213+0000 7faf06000640 1 -- 172.21.15.145:0/1286051878 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v23) ==== 172+0+0 (secure 0 0 0) 0x7faf000615c0 con 0x7faef4005280 2024-05-09T04:35:24.213 INFO:teuthology.orchestra.run.smithi145.stderr:new fs with metadata pool 7 and data pool 8 2024-05-09T04:35:24.217 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.218+0000 7faf15200640 1 -- 172.21.15.145:0/1286051878 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7faef0076300 msgr2=0x7faef00787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:24.218 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.218+0000 7faf15200640 1 --2- 172.21.15.145:0/1286051878 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7faef0076300 0x7faef00787c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7faf100cf4f0 tx=0x7faefc009040 comp rx=0 tx=0).stop 2024-05-09T04:35:24.218 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.218+0000 7faf15200640 1 -- 172.21.15.145:0/1286051878 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faef4005280 msgr2=0x7faf101473a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:24.218 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.218+0000 7faf15200640 1 --2- 172.21.15.145:0/1286051878 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faef4005280 0x7faf101473a0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7faf00010860 tx=0x7faf00010d20 comp rx=0 tx=0).stop 2024-05-09T04:35:24.218 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.218+0000 7faf15200640 1 -- 172.21.15.145:0/1286051878 shutdown_connections 2024-05-09T04:35:24.218 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.218+0000 7faf15200640 1 --2- 172.21.15.145:0/1286051878 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7faef0076300 0x7faef00787c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:24.218 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.218+0000 7faf15200640 1 --2- 172.21.15.145:0/1286051878 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faf1014a820 0x7faf1014ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:24.218 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.218+0000 7faf15200640 1 --2- 172.21.15.145:0/1286051878 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf10149010 0x7faf10161610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:24.218 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.218+0000 7faf15200640 1 --2- 172.21.15.145:0/1286051878 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faef4005280 0x7faf101473a0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:24.218 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.218+0000 7faf15200640 1 -- 172.21.15.145:0/1286051878 >> 172.21.15.145:0/1286051878 conn(0x7faf1006cab0 msgr2=0x7faf1006e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:24.220 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.221+0000 7faf15200640 1 -- 172.21.15.145:0/1286051878 shutdown_connections 2024-05-09T04:35:24.220 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.221+0000 7faf15200640 1 -- 172.21.15.145:0/1286051878 wait complete. 2024-05-09T04:35:24.259 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T04:35:24.428 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.428+0000 7f70fde00640 1 Processor -- start 2024-05-09T04:35:24.428 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.428+0000 7f70fde00640 1 -- start start 2024-05-09T04:35:24.429 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.429+0000 7f70fde00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f70f80d22f0 0x7f70f80b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:24.429 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.429+0000 7f70fde00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70f80b53b0 con 0x7f70f80d5ad0 2024-05-09T04:35:24.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.429+0000 7f70fde00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70f80d3e70 con 0x7f70f80d3180 2024-05-09T04:35:24.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.429+0000 7f70fde00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f70f80d4150 con 0x7f70f80d22f0 2024-05-09T04:35:24.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.430+0000 7f70fd400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f70f80d22f0 0x7f70f80b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:24.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.430+0000 7f70fd400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f70f80d22f0 0x7f70f80b4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:52808/0 (socket says 172.21.15.145:52808) 2024-05-09T04:35:24.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.430+0000 7f70fd400640 1 -- 172.21.15.145:0/3344704575 learned_addr learned my addr 172.21.15.145:0/3344704575 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:24.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.430+0000 7f70fd400640 1 -- 172.21.15.145:0/3344704575 >> v1:172.21.15.192:6789/0 conn(0x7f70f80d3180 legacy=0x7f70f80d3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:24.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.430+0000 7f70fd400640 1 -- 172.21.15.145:0/3344704575 >> v1:172.21.15.145:6789/0 conn(0x7f70f80d5ad0 legacy=0x7f70f80b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:24.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.430+0000 7f70fd400640 1 -- 172.21.15.145:0/3344704575 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70f80b6450 con 0x7f70f80d22f0 2024-05-09T04:35:24.431 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.431+0000 7f70fd400640 1 --2- 172.21.15.145:0/3344704575 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f70f80d22f0 0x7f70f80b4e70 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f70f0009e50 tx=0x7f70f002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61b1cb2ab85d263d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:24.431 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.431+0000 7f70f7400640 1 -- 172.21.15.145:0/3344704575 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f0036070 con 0x7f70f80d22f0 2024-05-09T04:35:24.431 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.431+0000 7f70f7400640 1 -- 172.21.15.145:0/3344704575 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70f0031600 con 0x7f70f80d22f0 2024-05-09T04:35:24.431 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.431+0000 7f70f7400640 1 -- 172.21.15.145:0/3344704575 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f00319d0 con 0x7f70f80d22f0 2024-05-09T04:35:24.432 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.432+0000 7f70fde00640 1 -- 172.21.15.145:0/3344704575 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f70f80d22f0 msgr2=0x7f70f80b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:24.432 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.432+0000 7f70fde00640 1 --2- 172.21.15.145:0/3344704575 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f70f80d22f0 0x7f70f80b4e70 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f70f0009e50 tx=0x7f70f002a680 comp rx=0 tx=0).stop 2024-05-09T04:35:24.432 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.432+0000 7f70fde00640 1 -- 172.21.15.145:0/3344704575 shutdown_connections 2024-05-09T04:35:24.432 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.432+0000 7f70fde00640 1 --2- 172.21.15.145:0/3344704575 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f70f80d22f0 0x7f70f80b4e70 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:24.432 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.432+0000 7f70fde00640 1 -- 172.21.15.145:0/3344704575 >> 172.21.15.145:0/3344704575 conn(0x7f70f80d4fb0 msgr2=0x7f70f80d53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:24.432 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.432+0000 7f70fde00640 1 -- 172.21.15.145:0/3344704575 shutdown_connections 2024-05-09T04:35:24.433 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.432+0000 7f70fde00640 1 -- 172.21.15.145:0/3344704575 wait complete. 2024-05-09T04:35:24.433 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.434+0000 7f70fde00640 1 Processor -- start 2024-05-09T04:35:24.433 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.434+0000 7f70fde00640 1 -- start start 2024-05-09T04:35:24.435 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.435+0000 7f70fde00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70f80d22f0 0x7f70f815e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:24.435 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.435+0000 7f70fde00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f70f80d3180 0x7f70f814f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:24.435 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.435+0000 7f70fde00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f70f80d5ad0 0x7f70f815c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:24.435 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.435+0000 7f70fde00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f70f80d1e60 con 0x7f70f80d3180 2024-05-09T04:35:24.436 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.435+0000 7f70fde00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f70f80b6200 con 0x7f70f80d22f0 2024-05-09T04:35:24.436 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.435+0000 7f70fde00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f70f80d2760 con 0x7f70f80d5ad0 2024-05-09T04:35:24.436 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.435+0000 7f70f7e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f70f80d3180 0x7f70f814f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:24.436 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.435+0000 7f70f7e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f70f80d3180 0x7f70f814f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:52814/0 (socket says 172.21.15.145:52814) 2024-05-09T04:35:24.436 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.435+0000 7f70f7e00640 1 -- 172.21.15.145:0/2687628626 learned_addr learned my addr 172.21.15.145:0/2687628626 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:24.436 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.436+0000 7f70f7e00640 1 -- 172.21.15.145:0/2687628626 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f70f80d5ad0 msgr2=0x7f70f815c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:24.436 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.436+0000 7f70f7e00640 1 --2- 172.21.15.145:0/2687628626 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f70f80d5ad0 0x7f70f815c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:24.436 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.436+0000 7f70f7e00640 1 -- 172.21.15.145:0/2687628626 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70f80d22f0 msgr2=0x7f70f815e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:24.436 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.436+0000 7f70f7e00640 1 --2- 172.21.15.145:0/2687628626 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70f80d22f0 0x7f70f815e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:24.436 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.436+0000 7f70f7e00640 1 -- 172.21.15.145:0/2687628626 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70f814f930 con 0x7f70f80d3180 2024-05-09T04:35:24.436 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.436+0000 7f70f7e00640 1 --2- 172.21.15.145:0/2687628626 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f70f80d3180 0x7f70f814f3f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f70e400c3a0 tx=0x7f70e400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:24.437 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.436+0000 7f70f4c00640 1 -- 172.21.15.145:0/2687628626 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e4017070 con 0x7f70f80d3180 2024-05-09T04:35:24.437 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.436+0000 7f70f4c00640 1 -- 172.21.15.145:0/2687628626 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70e40052d0 con 0x7f70f80d3180 2024-05-09T04:35:24.437 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.436+0000 7f70fde00640 1 -- 172.21.15.145:0/2687628626 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70f815d1e0 con 0x7f70f80d3180 2024-05-09T04:35:24.437 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.436+0000 7f70f4c00640 1 -- 172.21.15.145:0/2687628626 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e4003830 con 0x7f70f80d3180 2024-05-09T04:35:24.437 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.436+0000 7f70fde00640 1 -- 172.21.15.145:0/2687628626 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70f815d7a0 con 0x7f70f80d3180 2024-05-09T04:35:24.437 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.437+0000 7f70fde00640 1 -- 172.21.15.145:0/2687628626 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70f806c5d0 con 0x7f70f80d3180 2024-05-09T04:35:24.438 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.439+0000 7f70f4c00640 1 -- 172.21.15.145:0/2687628626 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f70e401a070 con 0x7f70f80d3180 2024-05-09T04:35:24.439 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.439+0000 7f70f4c00640 1 --2- 172.21.15.145:0/2687628626 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f70d8076300 0x7f70d80787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:24.439 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.440+0000 7f70f4c00640 1 -- 172.21.15.145:0/2687628626 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f70e40969a0 con 0x7f70f80d3180 2024-05-09T04:35:24.439 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.440+0000 7f70fca00640 1 --2- 172.21.15.145:0/2687628626 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f70d8076300 0x7f70d80787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:24.440 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.440+0000 7f70fca00640 1 --2- 172.21.15.145:0/2687628626 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f70d8076300 0x7f70d80787c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f70e8001560 tx=0x7f70e8007990 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:24.443 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.444+0000 7f70f4c00640 1 -- 172.21.15.145:0/2687628626 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f70f806c5d0 con 0x7f70f80d3180 2024-05-09T04:35:24.642 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.642+0000 7f70fde00640 1 -- 172.21.15.145:0/2687628626 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f70f806de70 con 0x7f70f80d3180 2024-05-09T04:35:24.644 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.644+0000 7f70f4c00640 1 -- 172.21.15.145:0/2687628626 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+16730 (secure 0 0 0) 0x7f70f814f930 con 0x7f70f80d3180 2024-05-09T04:35:24.644 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:35:24.644 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":46,"fsid":"957ee6c1-073b-44cf-9b93-9491b3fdfc1d","created":"2024-05-09T04:33:26.544367+0000","modified":"2024-05-09T04:35:23.667337+0000","last_up_change":"2024-05-09T04:33:39.039318+0000","last_in_change":"2024-05-09T04:33:29.902361+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T04:33:39.795344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-09T04:35:21.570048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-09T04:35:22.639737+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22e550ad-847b-4952-80e6-63d859ed1a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6801","nonce":3182572912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6803","nonce":3182572912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6807","nonce":3182572912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6805","nonce":3182572912}]},"public_addr":"172.21.15.145:6801/3182572912","cluster_addr":"172.21.15.145:6803/3182572912","heartbeat_back_addr":"172.21.15.145:6807/3182572912","heartbeat_front_addr":"172.21.15.145:6805/3182572912","state":["exists","up"]},{"osd":1,"uuid":"0f6b9fca-0c41-4145-8d74-598ddb42f2a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6825","nonce":1845225555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6827","nonce":1845225555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6831","nonce":1845225555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6829","nonce":1845225555}]},"public_addr":"172.21.15.145:6825/1845225555","cluster_addr":"172.21.15.145:6827/1845225555","heartbeat_back_addr":"172.21.15.145:6831/1845225555","heartbeat_front_addr":"172.21.15.145:6829/1845225555","state":["exists","up"]},{"osd":2,"uuid":"b3b1f67e-5790-483f-8610-e05a5bce4c30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6817","nonce":2168958993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6819","nonce":2168958993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6823","nonce":2168958993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6821","nonce":2168958993}]},"public_addr":"172.21.15.145:6817/2168958993","cluster_addr":"172.21.15.145:6819/2168958993","heartbeat_back_addr":"172.21.15.145:6823/2168958993","heartbeat_front_addr":"172.21.15.145:6821/2168958993","state":["exists","up"]},{"osd":3,"uuid":"884787a5-e037-4d31-9aae-743030eba4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6809","nonce":337009826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6811","nonce":337009826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6815","nonce":337009826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6813","nonce":337009826}]},"public_addr":"172.21.15.145:6809/337009826","cluster_addr":"172.21.15.145:6811/337009826","heartbeat_back_addr":"172.21.15.145:6815/337009826","heartbeat_front_addr":"172.21.15.145:6813/337009826","state":["exists","up"]},{"osd":4,"uuid":"0759fc1e-c5fc-4a2f-86bb-e8eccd83f833","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6818","nonce":3117700995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6821","nonce":3117700995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6827","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6829","nonce":3117700995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6823","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6825","nonce":3117700995}]},"public_addr":"172.21.15.192:6818/3117700995","cluster_addr":"172.21.15.192:6821/3117700995","heartbeat_back_addr":"172.21.15.192:6829/3117700995","heartbeat_front_addr":"172.21.15.192:6825/3117700995","state":["exists","up"]},{"osd":5,"uuid":"9443cc22-9379-46e3-9173-98b592bf2c16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6801","nonce":1328373268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6803","nonce":1328373268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6807","nonce":1328373268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6805","nonce":1328373268}]},"public_addr":"172.21.15.192:6801/1328373268","cluster_addr":"172.21.15.192:6803/1328373268","heartbeat_back_addr":"172.21.15.192:6807/1328373268","heartbeat_front_addr":"172.21.15.192:6805/1328373268","state":["exists","up"]},{"osd":6,"uuid":"d05a3311-505a-45b5-85b9-aa3113e96987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6817","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6820","nonce":1868011057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6824","nonce":1868011057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6831","nonce":1868011057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6828","nonce":1868011057}]},"public_addr":"172.21.15.192:6820/1868011057","cluster_addr":"172.21.15.192:6824/1868011057","heartbeat_back_addr":"172.21.15.192:6831/1868011057","heartbeat_front_addr":"172.21.15.192:6828/1868011057","state":["exists","up"]},{"osd":7,"uuid":"c7c23dbc-2029-43ec-9f53-25f3d58019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6809","nonce":3368512157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6811","nonce":3368512157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6815","nonce":3368512157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6813","nonce":3368512157}]},"public_addr":"172.21.15.192:6809/3368512157","cluster_addr":"172.21.15.192:6811/3368512157","heartbeat_back_addr":"172.21.15.192:6815/3368512157","heartbeat_front_addr":"172.21.15.192:6813/3368512157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.091188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.032567+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:34.797468+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.228839+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.622400+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.473103+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.566092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.349395+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T04:35:24.646 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.646+0000 7f70fde00640 1 -- 172.21.15.145:0/2687628626 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f70d8076300 msgr2=0x7f70d80787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:24.646 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.646+0000 7f70fde00640 1 --2- 172.21.15.145:0/2687628626 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f70d8076300 0x7f70d80787c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f70e8001560 tx=0x7f70e8007990 comp rx=0 tx=0).stop 2024-05-09T04:35:24.646 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.646+0000 7f70fde00640 1 -- 172.21.15.145:0/2687628626 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f70f80d3180 msgr2=0x7f70f814f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:24.647 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.646+0000 7f70fde00640 1 --2- 172.21.15.145:0/2687628626 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f70f80d3180 0x7f70f814f3f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f70e400c3a0 tx=0x7f70e400c860 comp rx=0 tx=0).stop 2024-05-09T04:35:24.647 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.646+0000 7f70fde00640 1 -- 172.21.15.145:0/2687628626 shutdown_connections 2024-05-09T04:35:24.647 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.646+0000 7f70fde00640 1 --2- 172.21.15.145:0/2687628626 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f70d8076300 0x7f70d80787c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:24.647 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.646+0000 7f70fde00640 1 --2- 172.21.15.145:0/2687628626 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f70f80d5ad0 0x7f70f815c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:24.647 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.646+0000 7f70fde00640 1 --2- 172.21.15.145:0/2687628626 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f70f80d3180 0x7f70f814f3f0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:24.647 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.646+0000 7f70fde00640 1 --2- 172.21.15.145:0/2687628626 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f70f80d22f0 0x7f70f815e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:24.647 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.646+0000 7f70fde00640 1 -- 172.21.15.145:0/2687628626 >> 172.21.15.145:0/2687628626 conn(0x7f70f80d4fb0 msgr2=0x7f70f806e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:24.647 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.647+0000 7f70fde00640 1 -- 172.21.15.145:0/2687628626 shutdown_connections 2024-05-09T04:35:24.647 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.647+0000 7f70fde00640 1 -- 172.21.15.145:0/2687628626 wait complete. 2024-05-09T04:35:24.665 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T04:35:24.860 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.860+0000 7f4c92c00640 1 Processor -- start 2024-05-09T04:35:24.860 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.860+0000 7f4c92c00640 1 -- start start 2024-05-09T04:35:24.861 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.862+0000 7f4c92c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c8c0b5920 0x7f4c8c167db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:24.861 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.862+0000 7f4c92c00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c8c1615c0 con 0x7f4c8c06e860 2024-05-09T04:35:24.861 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.862+0000 7f4c92c00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c8c1618a0 con 0x7f4c8c15d780 2024-05-09T04:35:24.862 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.862+0000 7f4c92c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f4c8c161360 con 0x7f4c8c0b5920 2024-05-09T04:35:24.862 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.862+0000 7f4c8be00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f4c8c06e860 0x7f4c8c06ec40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:55152/0 (socket says 172.21.15.145:55152) 2024-05-09T04:35:24.862 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.862+0000 7f4c8be00640 1 -- 172.21.15.145:0/196614572 learned_addr learned my addr 172.21.15.145:0/196614572 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:35:24.864 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.862+0000 7f4c8aa00640 1 -- 172.21.15.145:0/196614572 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1056927505 0 0) 0x7f4c8c1615c0 con 0x7f4c8c06e860 2024-05-09T04:35:24.864 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.862+0000 7f4c8aa00640 1 -- 172.21.15.145:0/196614572 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4c70003520 con 0x7f4c8c06e860 2024-05-09T04:35:24.865 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.862+0000 7f4c90c00640 1 --2- 172.21.15.145:0/196614572 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c8c0b5920 0x7f4c8c167db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:24.865 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.863+0000 7f4c8aa00640 1 -- 172.21.15.145:0/196614572 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3493404897 0 0) 0x7f4c70003520 con 0x7f4c8c06e860 2024-05-09T04:35:24.865 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.863+0000 7f4c8aa00640 1 -- 172.21.15.145:0/196614572 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c8c0b5920 msgr2=0x7f4c8c167db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:24.865 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.863+0000 7f4c8aa00640 1 --2- 172.21.15.145:0/196614572 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c8c0b5920 0x7f4c8c167db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:24.865 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.863+0000 7f4c8aa00640 1 -- 172.21.15.145:0/196614572 >> v1:172.21.15.192:6789/0 conn(0x7f4c8c15d780 legacy=0x7f4c8c160c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:24.865 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.863+0000 7f4c8aa00640 1 -- 172.21.15.145:0/196614572 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c8c0b4110 con 0x7f4c8c06e860 2024-05-09T04:35:24.865 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.863+0000 7f4c8aa00640 1 -- 172.21.15.145:0/196614572 <== mon.0 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3474413261 0 0) 0x7f4c7c001710 con 0x7f4c8c06e860 2024-05-09T04:35:24.865 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.863+0000 7f4c8aa00640 1 -- 172.21.15.145:0/196614572 >> v1:172.21.15.145:6789/0 conn(0x7f4c8c06e860 legacy=0x7f4c8c06ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:24.865 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.863+0000 7f4c8aa00640 1 --2- 172.21.15.145:0/196614572 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c8c0b5920 0x7f4c700039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:24.865 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.863+0000 7f4c8aa00640 1 --2- 172.21.15.145:0/196614572 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4c70004040 0x7f4c70004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:24.865 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.863+0000 7f4c8aa00640 1 --2- 172.21.15.145:0/196614572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c70004d40 0x7f4c700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:24.865 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.863+0000 7f4c8aa00640 1 -- 172.21.15.145:0/196614572 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8c161360 con 0x7f4c70004040 2024-05-09T04:35:24.865 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.863+0000 7f4c8aa00640 1 -- 172.21.15.145:0/196614572 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7000bbb0 con 0x7f4c70004d40 2024-05-09T04:35:24.865 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.863+0000 7f4c8aa00640 1 -- 172.21.15.145:0/196614572 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f4c7000bdb0 con 0x7f4c8c0b5920 2024-05-09T04:35:24.866 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.863+0000 7f4c8b400640 1 --2- 172.21.15.145:0/196614572 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4c70004040 0x7f4c70004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:24.866 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.863+0000 7f4c8be00640 1 --2- 172.21.15.145:0/196614572 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c8c0b5920 0x7f4c700039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:24.866 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.864+0000 7f4c90c00640 1 --2- 172.21.15.145:0/196614572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c70004d40 0x7f4c700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:24.866 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.864+0000 7f4c8b400640 1 -- 172.21.15.145:0/196614572 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c8c0b5920 msgr2=0x7f4c700039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:24.866 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.864+0000 7f4c8b400640 1 --2- 172.21.15.145:0/196614572 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c8c0b5920 0x7f4c700039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:24.866 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.864+0000 7f4c8b400640 1 -- 172.21.15.145:0/196614572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c70004d40 msgr2=0x7f4c700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:24.866 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.864+0000 7f4c8b400640 1 --2- 172.21.15.145:0/196614572 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c70004d40 0x7f4c700051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:24.866 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.864+0000 7f4c8b400640 1 -- 172.21.15.145:0/196614572 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c70003520 con 0x7f4c70004040 2024-05-09T04:35:24.866 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.864+0000 7f4c8be00640 1 --2- 172.21.15.145:0/196614572 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c8c0b5920 0x7f4c700039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:35:24.866 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.865+0000 7f4c8b400640 1 --2- 172.21.15.145:0/196614572 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4c70004040 0x7f4c70004800 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f4c78003db0 tx=0x7f4c70005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92a2965515ef5586 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:24.866 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.865+0000 7f4c8aa00640 1 -- 172.21.15.145:0/196614572 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c78037070 con 0x7f4c70004040 2024-05-09T04:35:24.866 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.865+0000 7f4c8aa00640 1 -- 172.21.15.145:0/196614572 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c7802aa00 con 0x7f4c70004040 2024-05-09T04:35:24.866 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.865+0000 7f4c8aa00640 1 -- 172.21.15.145:0/196614572 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c78003980 con 0x7f4c70004040 2024-05-09T04:35:24.867 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.865+0000 7f4c92c00640 1 -- 172.21.15.145:0/196614572 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4c70004040 msgr2=0x7f4c70004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:24.867 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.865+0000 7f4c92c00640 1 --2- 172.21.15.145:0/196614572 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4c70004040 0x7f4c70004800 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f4c78003db0 tx=0x7f4c70005830 comp rx=0 tx=0).stop 2024-05-09T04:35:24.868 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.865+0000 7f4c90c00640 1 -- 172.21.15.145:0/196614572 reap_dead start 2024-05-09T04:35:24.868 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.866+0000 7f4c92c00640 1 -- 172.21.15.145:0/196614572 shutdown_connections 2024-05-09T04:35:24.868 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.866+0000 7f4c92c00640 1 -- 172.21.15.145:0/196614572 >> 172.21.15.145:0/196614572 conn(0x7f4c8c06c870 msgr2=0x7f4c8c06cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:24.868 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.866+0000 7f4c92c00640 1 -- 172.21.15.145:0/196614572 shutdown_connections 2024-05-09T04:35:24.868 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.866+0000 7f4c92c00640 1 -- 172.21.15.145:0/196614572 wait complete. 2024-05-09T04:35:24.868 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.867+0000 7f4c92c00640 1 Processor -- start 2024-05-09T04:35:24.868 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.868+0000 7f4c92c00640 1 -- start start 2024-05-09T04:35:24.870 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.868+0000 7f4c92c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c70004040 0x7f4c8c138fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:24.870 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.868+0000 7f4c92c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4c8c1602b0 0x7f4c8c15db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:24.871 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.868+0000 7f4c92c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c8c139510 0x7f4c8c15e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:24.871 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.868+0000 7f4c92c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8c146d80 con 0x7f4c8c1602b0 2024-05-09T04:35:24.871 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.868+0000 7f4c92c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8c146c00 con 0x7f4c8c139510 2024-05-09T04:35:24.871 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.868+0000 7f4c92c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f4c8c072220 con 0x7f4c70004040 2024-05-09T04:35:24.871 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.869+0000 7f4c8b400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4c8c1602b0 0x7f4c8c15db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:24.871 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.869+0000 7f4c8b400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4c8c1602b0 0x7f4c8c15db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:52844/0 (socket says 172.21.15.145:52844) 2024-05-09T04:35:24.871 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.869+0000 7f4c8b400640 1 -- 172.21.15.145:0/1005549218 learned_addr learned my addr 172.21.15.145:0/1005549218 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:24.871 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.869+0000 7f4c8be00640 1 --2- 172.21.15.145:0/1005549218 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c70004040 0x7f4c8c138fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:24.871 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.869+0000 7f4c90c00640 1 --2- 172.21.15.145:0/1005549218 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c8c139510 0x7f4c8c15e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:24.871 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.869+0000 7f4c8b400640 1 -- 172.21.15.145:0/1005549218 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c70004040 msgr2=0x7f4c8c138fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:24.871 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.869+0000 7f4c8b400640 1 --2- 172.21.15.145:0/1005549218 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c70004040 0x7f4c8c138fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:24.871 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.869+0000 7f4c8b400640 1 -- 172.21.15.145:0/1005549218 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c8c139510 msgr2=0x7f4c8c15e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:24.871 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.869+0000 7f4c8b400640 1 --2- 172.21.15.145:0/1005549218 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c8c139510 0x7f4c8c15e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:24.871 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.869+0000 7f4c8b400640 1 -- 172.21.15.145:0/1005549218 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c8c139a20 con 0x7f4c8c1602b0 2024-05-09T04:35:24.872 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.869+0000 7f4c8be00640 1 --2- 172.21.15.145:0/1005549218 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c70004040 0x7f4c8c138fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:35:24.872 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.869+0000 7f4c8b400640 1 --2- 172.21.15.145:0/1005549218 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4c8c1602b0 0x7f4c8c15db90 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f4c78035040 tx=0x7f4c780036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:24.872 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.870+0000 7f4c83e00640 1 -- 172.21.15.145:0/1005549218 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c78037040 con 0x7f4c8c1602b0 2024-05-09T04:35:24.872 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.870+0000 7f4c83e00640 1 -- 172.21.15.145:0/1005549218 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c78006e50 con 0x7f4c8c1602b0 2024-05-09T04:35:24.872 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.870+0000 7f4c83e00640 1 -- 172.21.15.145:0/1005549218 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c78007260 con 0x7f4c8c1602b0 2024-05-09T04:35:24.872 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.870+0000 7f4c92c00640 1 -- 172.21.15.145:0/1005549218 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c8c139d00 con 0x7f4c8c1602b0 2024-05-09T04:35:24.872 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.870+0000 7f4c92c00640 1 -- 172.21.15.145:0/1005549218 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c8c13b700 con 0x7f4c8c1602b0 2024-05-09T04:35:24.872 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.871+0000 7f4c92c00640 1 -- 172.21.15.145:0/1005549218 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c8c06a310 con 0x7f4c8c1602b0 2024-05-09T04:35:24.873 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.872+0000 7f4c83e00640 1 -- 172.21.15.145:0/1005549218 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4c78007400 con 0x7f4c8c1602b0 2024-05-09T04:35:24.873 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.873+0000 7f4c83e00640 1 --2- 172.21.15.145:0/1005549218 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f4c6c076300 0x7f4c6c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:24.873 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.873+0000 7f4c83e00640 1 -- 172.21.15.145:0/1005549218 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f4c780b7f10 con 0x7f4c8c1602b0 2024-05-09T04:35:24.873 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.873+0000 7f4c8be00640 1 --2- 172.21.15.145:0/1005549218 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f4c6c076300 0x7f4c6c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:24.873 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.874+0000 7f4c8be00640 1 --2- 172.21.15.145:0/1005549218 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f4c6c076300 0x7f4c6c0787c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c001880 tx=0x7f4c7c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:24.877 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:24.878+0000 7f4c83e00640 1 -- 172.21.15.145:0/1005549218 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f4c78081860 con 0x7f4c8c1602b0 2024-05-09T04:35:25.082 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.082+0000 7f4c92c00640 1 -- 172.21.15.145:0/1005549218 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4c8c143110 con 0x7f4c8c1602b0 2024-05-09T04:35:25.085 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.085+0000 7f4c83e00640 1 -- 172.21.15.145:0/1005549218 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+16730 (secure 0 0 0) 0x7f4c78081160 con 0x7f4c8c1602b0 2024-05-09T04:35:25.085 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:35:25.086 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":46,"fsid":"957ee6c1-073b-44cf-9b93-9491b3fdfc1d","created":"2024-05-09T04:33:26.544367+0000","modified":"2024-05-09T04:35:23.667337+0000","last_up_change":"2024-05-09T04:33:39.039318+0000","last_in_change":"2024-05-09T04:33:29.902361+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T04:33:39.795344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-09T04:35:21.570048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-09T04:35:22.639737+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22e550ad-847b-4952-80e6-63d859ed1a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6801","nonce":3182572912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6803","nonce":3182572912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6807","nonce":3182572912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6805","nonce":3182572912}]},"public_addr":"172.21.15.145:6801/3182572912","cluster_addr":"172.21.15.145:6803/3182572912","heartbeat_back_addr":"172.21.15.145:6807/3182572912","heartbeat_front_addr":"172.21.15.145:6805/3182572912","state":["exists","up"]},{"osd":1,"uuid":"0f6b9fca-0c41-4145-8d74-598ddb42f2a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6825","nonce":1845225555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6827","nonce":1845225555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6831","nonce":1845225555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6829","nonce":1845225555}]},"public_addr":"172.21.15.145:6825/1845225555","cluster_addr":"172.21.15.145:6827/1845225555","heartbeat_back_addr":"172.21.15.145:6831/1845225555","heartbeat_front_addr":"172.21.15.145:6829/1845225555","state":["exists","up"]},{"osd":2,"uuid":"b3b1f67e-5790-483f-8610-e05a5bce4c30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6817","nonce":2168958993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6819","nonce":2168958993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6823","nonce":2168958993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6821","nonce":2168958993}]},"public_addr":"172.21.15.145:6817/2168958993","cluster_addr":"172.21.15.145:6819/2168958993","heartbeat_back_addr":"172.21.15.145:6823/2168958993","heartbeat_front_addr":"172.21.15.145:6821/2168958993","state":["exists","up"]},{"osd":3,"uuid":"884787a5-e037-4d31-9aae-743030eba4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6809","nonce":337009826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6811","nonce":337009826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6815","nonce":337009826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6813","nonce":337009826}]},"public_addr":"172.21.15.145:6809/337009826","cluster_addr":"172.21.15.145:6811/337009826","heartbeat_back_addr":"172.21.15.145:6815/337009826","heartbeat_front_addr":"172.21.15.145:6813/337009826","state":["exists","up"]},{"osd":4,"uuid":"0759fc1e-c5fc-4a2f-86bb-e8eccd83f833","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6818","nonce":3117700995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6821","nonce":3117700995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6827","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6829","nonce":3117700995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6823","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6825","nonce":3117700995}]},"public_addr":"172.21.15.192:6818/3117700995","cluster_addr":"172.21.15.192:6821/3117700995","heartbeat_back_addr":"172.21.15.192:6829/3117700995","heartbeat_front_addr":"172.21.15.192:6825/3117700995","state":["exists","up"]},{"osd":5,"uuid":"9443cc22-9379-46e3-9173-98b592bf2c16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6801","nonce":1328373268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6803","nonce":1328373268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6807","nonce":1328373268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6805","nonce":1328373268}]},"public_addr":"172.21.15.192:6801/1328373268","cluster_addr":"172.21.15.192:6803/1328373268","heartbeat_back_addr":"172.21.15.192:6807/1328373268","heartbeat_front_addr":"172.21.15.192:6805/1328373268","state":["exists","up"]},{"osd":6,"uuid":"d05a3311-505a-45b5-85b9-aa3113e96987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6817","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6820","nonce":1868011057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6824","nonce":1868011057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6831","nonce":1868011057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6828","nonce":1868011057}]},"public_addr":"172.21.15.192:6820/1868011057","cluster_addr":"172.21.15.192:6824/1868011057","heartbeat_back_addr":"172.21.15.192:6831/1868011057","heartbeat_front_addr":"172.21.15.192:6828/1868011057","state":["exists","up"]},{"osd":7,"uuid":"c7c23dbc-2029-43ec-9f53-25f3d58019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6809","nonce":3368512157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6811","nonce":3368512157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6815","nonce":3368512157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6813","nonce":3368512157}]},"public_addr":"172.21.15.192:6809/3368512157","cluster_addr":"172.21.15.192:6811/3368512157","heartbeat_back_addr":"172.21.15.192:6815/3368512157","heartbeat_front_addr":"172.21.15.192:6813/3368512157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.091188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.032567+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:34.797468+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.228839+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.622400+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.473103+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.566092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.349395+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T04:35:25.088 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.088+0000 7f4c92c00640 1 -- 172.21.15.145:0/1005549218 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f4c6c076300 msgr2=0x7f4c6c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:25.088 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.088+0000 7f4c92c00640 1 --2- 172.21.15.145:0/1005549218 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f4c6c076300 0x7f4c6c0787c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c001880 tx=0x7f4c7c009040 comp rx=0 tx=0).stop 2024-05-09T04:35:25.088 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.088+0000 7f4c92c00640 1 -- 172.21.15.145:0/1005549218 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4c8c1602b0 msgr2=0x7f4c8c15db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:25.088 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.088+0000 7f4c92c00640 1 --2- 172.21.15.145:0/1005549218 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4c8c1602b0 0x7f4c8c15db90 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f4c78035040 tx=0x7f4c780036f0 comp rx=0 tx=0).stop 2024-05-09T04:35:25.089 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.088+0000 7f4c92c00640 1 -- 172.21.15.145:0/1005549218 shutdown_connections 2024-05-09T04:35:25.089 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.088+0000 7f4c92c00640 1 --2- 172.21.15.145:0/1005549218 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f4c6c076300 0x7f4c6c0787c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:25.089 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.088+0000 7f4c92c00640 1 --2- 172.21.15.145:0/1005549218 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c8c139510 0x7f4c8c15e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:25.089 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.088+0000 7f4c92c00640 1 --2- 172.21.15.145:0/1005549218 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4c8c1602b0 0x7f4c8c15db90 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:25.089 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.088+0000 7f4c92c00640 1 --2- 172.21.15.145:0/1005549218 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c70004040 0x7f4c8c138fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:25.089 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.088+0000 7f4c92c00640 1 -- 172.21.15.145:0/1005549218 >> 172.21.15.145:0/1005549218 conn(0x7f4c8c06c870 msgr2=0x7f4c8c0ce790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:25.089 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.089+0000 7f4c92c00640 1 -- 172.21.15.145:0/1005549218 shutdown_connections 2024-05-09T04:35:25.089 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.089+0000 7f4c92c00640 1 -- 172.21.15.145:0/1005549218 wait complete. 2024-05-09T04:35:25.108 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-09T04:35:25.314 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.314+0000 7f0722c00640 1 Processor -- start 2024-05-09T04:35:25.314 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.314+0000 7f0722c00640 1 -- start start 2024-05-09T04:35:25.316 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.316+0000 7f0722c00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f071c144c00 0x7f071c0ce640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:25.316 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.316+0000 7f0722c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f071c145440 0x7f071c1458a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:25.316 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.316+0000 7f0722c00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f071c15ffa0 con 0x7f071c06f5d0 2024-05-09T04:35:25.316 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.316+0000 7f0722c00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f071c160280 con 0x7f071c144c00 2024-05-09T04:35:25.316 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.316+0000 7f0722c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f071c160560 con 0x7f071c145440 2024-05-09T04:35:25.316 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.316+0000 7f071be00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f071c144c00 0x7f071c0ce640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:25.316 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.316+0000 7f071be00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f071c144c00 0x7f071c0ce640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:52848/0 (socket says 172.21.15.145:52848) 2024-05-09T04:35:25.317 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.316+0000 7f0720c00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f071c06f5d0 0x7f071c072200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:55154/0 (socket says 172.21.15.145:55154) 2024-05-09T04:35:25.317 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.316+0000 7f071be00640 1 -- 172.21.15.145:0/1700999597 learned_addr learned my addr 172.21.15.145:0/1700999597 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:25.318 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.317+0000 7f071b400640 1 --2- 172.21.15.145:0/1700999597 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f071c145440 0x7f071c1458a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:25.318 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.317+0000 7f071be00640 1 -- 172.21.15.145:0/1700999597 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f071c145440 msgr2=0x7f071c1458a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:25.318 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.317+0000 7f071be00640 1 --2- 172.21.15.145:0/1700999597 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f071c145440 0x7f071c1458a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:25.318 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.317+0000 7f071be00640 1 -- 172.21.15.145:0/1700999597 >> v1:172.21.15.145:6789/0 conn(0x7f071c06f5d0 legacy=0x7f071c072200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:25.318 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.317+0000 7f071aa00640 1 -- 172.21.15.145:0/1700999597 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2955060331 0 0) 0x7f071c15ffa0 con 0x7f071c06f5d0 2024-05-09T04:35:25.318 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.317+0000 7f071be00640 1 -- 172.21.15.145:0/1700999597 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f071c160840 con 0x7f071c144c00 2024-05-09T04:35:25.318 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.317+0000 7f071b400640 1 --2- 172.21.15.145:0/1700999597 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f071c145440 0x7f071c1458a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:35:25.319 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.318+0000 7f071be00640 1 --2- 172.21.15.145:0/1700999597 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f071c144c00 0x7f071c0ce640 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f070c009870 tx=0x7f070c02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88fee14207123d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:25.319 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.318+0000 7f071aa00640 1 -- 172.21.15.145:0/1700999597 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f070c037070 con 0x7f071c144c00 2024-05-09T04:35:25.319 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.318+0000 7f071aa00640 1 -- 172.21.15.145:0/1700999597 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f070c032440 con 0x7f071c144c00 2024-05-09T04:35:25.319 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.318+0000 7f071aa00640 1 -- 172.21.15.145:0/1700999597 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f070c032810 con 0x7f071c144c00 2024-05-09T04:35:25.320 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.319+0000 7f0722c00640 1 -- 172.21.15.145:0/1700999597 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f071c144c00 msgr2=0x7f071c0ce640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:25.320 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.319+0000 7f0722c00640 1 --2- 172.21.15.145:0/1700999597 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f071c144c00 0x7f071c0ce640 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f070c009870 tx=0x7f070c02a250 comp rx=0 tx=0).stop 2024-05-09T04:35:25.320 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.319+0000 7f0722c00640 1 -- 172.21.15.145:0/1700999597 shutdown_connections 2024-05-09T04:35:25.320 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.319+0000 7f0722c00640 1 --2- 172.21.15.145:0/1700999597 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f071c145440 0x7f071c1458a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:25.320 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.319+0000 7f0722c00640 1 --2- 172.21.15.145:0/1700999597 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f071c144c00 0x7f071c0ce640 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:25.320 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.319+0000 7f0722c00640 1 -- 172.21.15.145:0/1700999597 >> 172.21.15.145:0/1700999597 conn(0x7f071c06cab0 msgr2=0x7f071c0d0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:25.320 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.319+0000 7f0722c00640 1 -- 172.21.15.145:0/1700999597 shutdown_connections 2024-05-09T04:35:25.320 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.319+0000 7f0722c00640 1 -- 172.21.15.145:0/1700999597 wait complete. 2024-05-09T04:35:25.323 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.321+0000 7f0722c00640 1 Processor -- start 2024-05-09T04:35:25.323 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.321+0000 7f0722c00640 1 -- start start 2024-05-09T04:35:25.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.322+0000 7f0722c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f071c06f5d0 0x7f071c0ca210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:25.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.322+0000 7f0722c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f071c144c00 0x7f071c1431e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:25.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.322+0000 7f0722c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f071c145440 0x7f071c13d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:25.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.322+0000 7f0722c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f071c144fe0 con 0x7f071c06f5d0 2024-05-09T04:35:25.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.322+0000 7f0722c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f071c15fc30 con 0x7f071c144c00 2024-05-09T04:35:25.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.322+0000 7f0722c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f071c145160 con 0x7f071c145440 2024-05-09T04:35:25.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.322+0000 7f071be00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f071c06f5d0 0x7f071c0ca210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:25.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.322+0000 7f071be00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f071c06f5d0 0x7f071c0ca210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:52860/0 (socket says 172.21.15.145:52860) 2024-05-09T04:35:25.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.322+0000 7f071be00640 1 -- 172.21.15.145:0/3794131310 learned_addr learned my addr 172.21.15.145:0/3794131310 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:25.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.323+0000 7f071be00640 1 -- 172.21.15.145:0/3794131310 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f071c145440 msgr2=0x7f071c13d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:25.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.323+0000 7f071be00640 1 --2- 172.21.15.145:0/3794131310 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f071c145440 0x7f071c13d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:25.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.323+0000 7f071be00640 1 -- 172.21.15.145:0/3794131310 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f071c144c00 msgr2=0x7f071c1431e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:25.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.323+0000 7f071be00640 1 --2- 172.21.15.145:0/3794131310 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f071c144c00 0x7f071c1431e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:25.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.323+0000 7f071be00640 1 -- 172.21.15.145:0/3794131310 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f071c0caa40 con 0x7f071c06f5d0 2024-05-09T04:35:25.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.323+0000 7f071be00640 1 --2- 172.21.15.145:0/3794131310 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f071c06f5d0 0x7f071c0ca210 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f070c000c00 tx=0x7f070c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:25.325 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.323+0000 7f0713e00640 1 -- 172.21.15.145:0/3794131310 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f070c037040 con 0x7f071c06f5d0 2024-05-09T04:35:25.325 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.323+0000 7f0713e00640 1 -- 172.21.15.145:0/3794131310 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f070c042070 con 0x7f071c06f5d0 2024-05-09T04:35:25.325 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.323+0000 7f0713e00640 1 -- 172.21.15.145:0/3794131310 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f070c030bd0 con 0x7f071c06f5d0 2024-05-09T04:35:25.325 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.323+0000 7f0722c00640 1 -- 172.21.15.145:0/3794131310 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f071c0cad20 con 0x7f071c06f5d0 2024-05-09T04:35:25.325 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.324+0000 7f0722c00640 1 -- 172.21.15.145:0/3794131310 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f071c0cb5c0 con 0x7f071c06f5d0 2024-05-09T04:35:25.325 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.325+0000 7f0722c00640 1 -- 172.21.15.145:0/3794131310 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f071c0caef0 con 0x7f071c06f5d0 2024-05-09T04:35:25.325 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.326+0000 7f0713e00640 1 -- 172.21.15.145:0/3794131310 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f070c047460 con 0x7f071c06f5d0 2024-05-09T04:35:25.326 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.327+0000 7f0713e00640 1 --2- 172.21.15.145:0/3794131310 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f06f80762d0 0x7f06f8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:25.327 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.327+0000 7f0713e00640 1 -- 172.21.15.145:0/3794131310 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f070c0b8220 con 0x7f071c06f5d0 2024-05-09T04:35:25.327 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.327+0000 7f071b400640 1 --2- 172.21.15.145:0/3794131310 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f06f80762d0 0x7f06f8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:25.327 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.328+0000 7f071b400640 1 --2- 172.21.15.145:0/3794131310 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f06f80762d0 0x7f06f8078790 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0704006fd0 tx=0x7f0704008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:25.331 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.331+0000 7f0713e00640 1 -- 172.21.15.145:0/3794131310 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f070c081b80 con 0x7f071c06f5d0 2024-05-09T04:35:25.560 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:25.560+0000 7f0722c00640 1 -- 172.21.15.145:0/3794131310 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f071c06a310 con 0x7f071c06f5d0 2024-05-09T04:35:26.174 INFO:tasks.ceph.mds.c.smithi145.stderr:2024-05-09T04:35:26.174+0000 7fcbd3a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-09T04:35:26.253 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.253+0000 7f0713e00640 1 -- 172.21.15.145:0/3794131310 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v26) ==== 118+0+0 (secure 0 0 0) 0x7f070c081480 con 0x7f071c06f5d0 2024-05-09T04:35:26.257 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.256+0000 7f0722c00640 1 -- 172.21.15.145:0/3794131310 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f06f80762d0 msgr2=0x7f06f8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:26.257 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.257+0000 7f0722c00640 1 --2- 172.21.15.145:0/3794131310 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f06f80762d0 0x7f06f8078790 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0704006fd0 tx=0x7f0704008040 comp rx=0 tx=0).stop 2024-05-09T04:35:26.257 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.257+0000 7f0722c00640 1 -- 172.21.15.145:0/3794131310 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f071c06f5d0 msgr2=0x7f071c0ca210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:26.258 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.257+0000 7f0722c00640 1 --2- 172.21.15.145:0/3794131310 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f071c06f5d0 0x7f071c0ca210 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f070c000c00 tx=0x7f070c0046c0 comp rx=0 tx=0).stop 2024-05-09T04:35:26.258 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.257+0000 7f0722c00640 1 -- 172.21.15.145:0/3794131310 shutdown_connections 2024-05-09T04:35:26.258 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.257+0000 7f0722c00640 1 --2- 172.21.15.145:0/3794131310 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f06f80762d0 0x7f06f8078790 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:26.258 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.257+0000 7f0722c00640 1 --2- 172.21.15.145:0/3794131310 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f071c145440 0x7f071c13d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:26.258 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.257+0000 7f0722c00640 1 --2- 172.21.15.145:0/3794131310 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f071c144c00 0x7f071c1431e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:26.258 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.257+0000 7f0722c00640 1 --2- 172.21.15.145:0/3794131310 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f071c06f5d0 0x7f071c0ca210 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:26.259 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.257+0000 7f0722c00640 1 -- 172.21.15.145:0/3794131310 >> 172.21.15.145:0/3794131310 conn(0x7f071c06cab0 msgr2=0x7f071c13ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:26.259 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.257+0000 7f0722c00640 1 -- 172.21.15.145:0/3794131310 shutdown_connections 2024-05-09T04:35:26.259 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.257+0000 7f0722c00640 1 -- 172.21.15.145:0/3794131310 wait complete. 2024-05-09T04:35:26.280 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-05-09T04:35:26.280 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-09T04:35:26.483 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.483+0000 7f10d7a00640 1 Processor -- start 2024-05-09T04:35:26.483 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.484+0000 7f10d7a00640 1 -- start start 2024-05-09T04:35:26.486 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.485+0000 7f10d7a00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f10d00b0580 0x7f10d01603c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:26.486 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.485+0000 7f10d7a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f10d00b4930 0x7f10d00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:26.486 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.485+0000 7f10d7a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10d0162f30 0x7f10d01633c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:26.486 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.485+0000 7f10d7a00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f10d00b52d0 con 0x7f10d00b4930 2024-05-09T04:35:26.486 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.485+0000 7f10d7a00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f10d00b0960 con 0x7f10d00b0580 2024-05-09T04:35:26.486 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.485+0000 7f10d7a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f10d0073520 con 0x7f10d0162f30 2024-05-09T04:35:26.486 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.486+0000 7f10cfe00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f10d00b4930 0x7f10d00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:26.486 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.486+0000 7f10cfe00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f10d00b4930 0x7f10d00b4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:52870/0 (socket says 172.21.15.145:52870) 2024-05-09T04:35:26.486 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.486+0000 7f10cfe00640 1 -- 172.21.15.145:0/4111661694 learned_addr learned my addr 172.21.15.145:0/4111661694 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:26.487 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.486+0000 7f10d5000640 1 --2- 172.21.15.145:0/4111661694 >> v2:172.21.15.192:3300/0 conn(0x7f10d00b0580 0x7f10d01603c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:26.487 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.486+0000 7f10d5a00640 1 --2- 172.21.15.145:0/4111661694 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10d0162f30 0x7f10d01633c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:26.487 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.486+0000 7f10cfe00640 1 -- 172.21.15.145:0/4111661694 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10d0162f30 msgr2=0x7f10d01633c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:26.487 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.486+0000 7f10cfe00640 1 --2- 172.21.15.145:0/4111661694 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10d0162f30 0x7f10d01633c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:26.487 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.486+0000 7f10cfe00640 1 -- 172.21.15.145:0/4111661694 >> v2:172.21.15.192:3300/0 conn(0x7f10d00b0580 msgr2=0x7f10d01603c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:26.487 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.486+0000 7f10cfe00640 1 --2- 172.21.15.145:0/4111661694 >> v2:172.21.15.192:3300/0 conn(0x7f10d00b0580 0x7f10d01603c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:26.487 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.486+0000 7f10cfe00640 1 -- 172.21.15.145:0/4111661694 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10d0072910 con 0x7f10d00b4930 2024-05-09T04:35:26.487 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.486+0000 7f10d5000640 1 --2- 172.21.15.145:0/4111661694 >> v2:172.21.15.192:3300/0 conn(0x7f10d00b0580 0x7f10d01603c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:35:26.488 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.487+0000 7f10cfe00640 1 --2- 172.21.15.145:0/4111661694 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10d00b4930 0x7f10d00b4d90 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f10d0070310 tx=0x7f10b802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa981486c27ad735 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:26.488 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.487+0000 7f10cf400640 1 -- 172.21.15.145:0/4111661694 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b8036070 con 0x7f10d00b4930 2024-05-09T04:35:26.489 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.488+0000 7f10cf400640 1 -- 172.21.15.145:0/4111661694 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10b8031600 con 0x7f10d00b4930 2024-05-09T04:35:26.489 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.488+0000 7f10cf400640 1 -- 172.21.15.145:0/4111661694 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b80319d0 con 0x7f10d00b4930 2024-05-09T04:35:26.489 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.488+0000 7f10d7a00640 1 -- 172.21.15.145:0/4111661694 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10d00b4930 msgr2=0x7f10d00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:26.489 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.488+0000 7f10d7a00640 1 --2- 172.21.15.145:0/4111661694 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10d00b4930 0x7f10d00b4d90 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f10d0070310 tx=0x7f10b802a6a0 comp rx=0 tx=0).stop 2024-05-09T04:35:26.489 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.488+0000 7f10d7a00640 1 -- 172.21.15.145:0/4111661694 shutdown_connections 2024-05-09T04:35:26.489 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.488+0000 7f10d7a00640 1 --2- 172.21.15.145:0/4111661694 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10d0162f30 0x7f10d01633c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:26.489 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.488+0000 7f10d7a00640 1 --2- 172.21.15.145:0/4111661694 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10d00b4930 0x7f10d00b4d90 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:26.489 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.488+0000 7f10d7a00640 1 --2- 172.21.15.145:0/4111661694 >> v2:172.21.15.192:3300/0 conn(0x7f10d00b0580 0x7f10d01603c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:26.489 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.488+0000 7f10d7a00640 1 -- 172.21.15.145:0/4111661694 >> 172.21.15.145:0/4111661694 conn(0x7f10d006cab0 msgr2=0x7f10d00d0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:26.489 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.488+0000 7f10d7a00640 1 -- 172.21.15.145:0/4111661694 shutdown_connections 2024-05-09T04:35:26.489 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.488+0000 7f10d7a00640 1 -- 172.21.15.145:0/4111661694 wait complete. 2024-05-09T04:35:26.490 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.490+0000 7f10d7a00640 1 Processor -- start 2024-05-09T04:35:26.490 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.490+0000 7f10d7a00640 1 -- start start 2024-05-09T04:35:26.492 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.491+0000 7f10d7a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10d00b0580 0x7f10d00cd950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:26.492 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.491+0000 7f10d7a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10d0162f30 0x7f10d00afa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:26.492 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.491+0000 7f10d7a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10d0161f50 0x7f10d01594f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:26.492 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.491+0000 7f10d7a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f10d00731b0 con 0x7f10d0161f50 2024-05-09T04:35:26.492 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.491+0000 7f10d7a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f10d0072ed0 con 0x7f10d00b0580 2024-05-09T04:35:26.492 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.491+0000 7f10d7a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f10d00739b0 con 0x7f10d0162f30 2024-05-09T04:35:26.493 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.491+0000 7f10d5a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10d0161f50 0x7f10d01594f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:26.493 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.491+0000 7f10d5a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10d0161f50 0x7f10d01594f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:52874/0 (socket says 172.21.15.145:52874) 2024-05-09T04:35:26.493 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.491+0000 7f10d5a00640 1 -- 172.21.15.145:0/1541391361 learned_addr learned my addr 172.21.15.145:0/1541391361 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:26.493 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.492+0000 7f10d5000640 1 --2- 172.21.15.145:0/1541391361 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10d00b0580 0x7f10d00cd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:26.493 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.492+0000 7f10cfe00640 1 --2- 172.21.15.145:0/1541391361 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10d0162f30 0x7f10d00afa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:26.493 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.492+0000 7f10d5a00640 1 -- 172.21.15.145:0/1541391361 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10d0162f30 msgr2=0x7f10d00afa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:26.493 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.492+0000 7f10d5a00640 1 --2- 172.21.15.145:0/1541391361 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10d0162f30 0x7f10d00afa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:26.493 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.492+0000 7f10d5a00640 1 -- 172.21.15.145:0/1541391361 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10d00b0580 msgr2=0x7f10d00cd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:26.493 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.492+0000 7f10d5a00640 1 --2- 172.21.15.145:0/1541391361 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10d00b0580 0x7f10d00cd950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:26.493 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.492+0000 7f10d5a00640 1 -- 172.21.15.145:0/1541391361 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10d0161190 con 0x7f10d0161f50 2024-05-09T04:35:26.493 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.492+0000 7f10d5000640 1 --2- 172.21.15.145:0/1541391361 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10d00b0580 0x7f10d00cd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:35:26.493 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.492+0000 7f10d5a00640 1 --2- 172.21.15.145:0/1541391361 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10d0161f50 0x7f10d01594f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f10c800c820 tx=0x7f10c800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:26.493 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.492+0000 7f10ccc00640 1 -- 172.21.15.145:0/1541391361 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c8015070 con 0x7f10d0161f50 2024-05-09T04:35:26.493 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.492+0000 7f10ccc00640 1 -- 172.21.15.145:0/1541391361 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10c8004020 con 0x7f10d0161f50 2024-05-09T04:35:26.493 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.492+0000 7f10d7a00640 1 -- 172.21.15.145:0/1541391361 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10d0161470 con 0x7f10d0161f50 2024-05-09T04:35:26.494 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.492+0000 7f10ccc00640 1 -- 172.21.15.145:0/1541391361 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c8004410 con 0x7f10d0161f50 2024-05-09T04:35:26.494 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.492+0000 7f10d7a00640 1 -- 172.21.15.145:0/1541391361 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10d0161a30 con 0x7f10d0161f50 2024-05-09T04:35:26.494 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.494+0000 7f10d7a00640 1 -- 172.21.15.145:0/1541391361 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10d006a310 con 0x7f10d0161f50 2024-05-09T04:35:26.494 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.495+0000 7f10ccc00640 1 -- 172.21.15.145:0/1541391361 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f10c8007830 con 0x7f10d0161f50 2024-05-09T04:35:26.495 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.496+0000 7f10ccc00640 1 --2- 172.21.15.145:0/1541391361 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f10ac0762d0 0x7f10ac078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:26.496 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.496+0000 7f10ccc00640 1 -- 172.21.15.145:0/1541391361 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f10c8097400 con 0x7f10d0161f50 2024-05-09T04:35:26.496 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.496+0000 7f10d5000640 1 --2- 172.21.15.145:0/1541391361 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f10ac0762d0 0x7f10ac078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:26.496 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.497+0000 7f10d5000640 1 --2- 172.21.15.145:0/1541391361 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f10ac0762d0 0x7f10ac078790 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f10d00b4f10 tx=0x7f10c0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:26.501 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.501+0000 7f10ccc00640 1 -- 172.21.15.145:0/1541391361 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f10c8060d50 con 0x7f10d0161f50 2024-05-09T04:35:26.748 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:26.748+0000 7f10d7a00640 1 -- 172.21.15.145:0/1541391361 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f10d00b6150 con 0x7f10d0161f50 2024-05-09T04:35:27.257 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.257+0000 7f10ccc00640 1 -- 172.21.15.145:0/1541391361 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v27) ==== 122+0+0 (secure 0 0 0) 0x7f10c8060650 con 0x7f10d0161f50 2024-05-09T04:35:27.260 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.260+0000 7f10d7a00640 1 -- 172.21.15.145:0/1541391361 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f10ac0762d0 msgr2=0x7f10ac078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:27.260 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.260+0000 7f10d7a00640 1 --2- 172.21.15.145:0/1541391361 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f10ac0762d0 0x7f10ac078790 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f10d00b4f10 tx=0x7f10c0008040 comp rx=0 tx=0).stop 2024-05-09T04:35:27.260 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.260+0000 7f10d7a00640 1 -- 172.21.15.145:0/1541391361 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10d0161f50 msgr2=0x7f10d01594f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:27.260 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.260+0000 7f10d7a00640 1 --2- 172.21.15.145:0/1541391361 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10d0161f50 0x7f10d01594f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f10c800c820 tx=0x7f10c800cce0 comp rx=0 tx=0).stop 2024-05-09T04:35:27.261 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.260+0000 7f10d7a00640 1 -- 172.21.15.145:0/1541391361 shutdown_connections 2024-05-09T04:35:27.261 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.260+0000 7f10d7a00640 1 --2- 172.21.15.145:0/1541391361 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f10ac0762d0 0x7f10ac078790 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:27.261 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.260+0000 7f10d7a00640 1 --2- 172.21.15.145:0/1541391361 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10d0161f50 0x7f10d01594f0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:27.261 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.260+0000 7f10d7a00640 1 --2- 172.21.15.145:0/1541391361 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10d0162f30 0x7f10d00afa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:27.261 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.260+0000 7f10d7a00640 1 --2- 172.21.15.145:0/1541391361 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10d00b0580 0x7f10d00cd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:27.261 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.260+0000 7f10d7a00640 1 -- 172.21.15.145:0/1541391361 >> 172.21.15.145:0/1541391361 conn(0x7f10d006cab0 msgr2=0x7f10d006eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:27.261 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.260+0000 7f10d7a00640 1 -- 172.21.15.145:0/1541391361 shutdown_connections 2024-05-09T04:35:27.261 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.260+0000 7f10d7a00640 1 -- 172.21.15.145:0/1541391361 wait complete. 2024-05-09T04:35:27.283 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-09T04:35:27.284 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T04:35:27.484 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.484+0000 7f18d8c00640 1 Processor -- start 2024-05-09T04:35:27.484 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.484+0000 7f18d8c00640 1 -- start start 2024-05-09T04:35:27.486 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.486+0000 7f18d8c00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f18d40d0640 0x7f18d415deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:27.486 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.486+0000 7f18d8c00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18d40aecd0 con 0x7f18d41572c0 2024-05-09T04:35:27.486 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.486+0000 7f18d8c00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18d40aefb0 con 0x7f18d40ae710 2024-05-09T04:35:27.486 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.486+0000 7f18d8c00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f18d40b6290 con 0x7f18d40d0640 2024-05-09T04:35:27.486 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.486+0000 7f18d3e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f18d40d0640 0x7f18d415deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:27.486 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.486+0000 7f18d3400640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f18d41572c0 0x7f18d4161220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:55160/0 (socket says 172.21.15.145:55160) 2024-05-09T04:35:27.486 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.486+0000 7f18d3400640 1 -- 172.21.15.145:0/260134908 learned_addr learned my addr 172.21.15.145:0/260134908 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:35:27.487 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.487+0000 7f18d3e00640 1 -- 172.21.15.145:0/260134908 >> v1:172.21.15.192:6789/0 conn(0x7f18d40ae710 legacy=0x7f18d40cff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:27.487 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.487+0000 7f18d3e00640 1 -- 172.21.15.145:0/260134908 >> v1:172.21.15.145:6789/0 conn(0x7f18d41572c0 legacy=0x7f18d4161220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:27.487 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.487+0000 7f18d3e00640 1 -- 172.21.15.145:0/260134908 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18d40af290 con 0x7f18d40d0640 2024-05-09T04:35:27.487 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.487+0000 7f18d2a00640 1 -- 172.21.15.145:0/260134908 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1749062977 0 0) 0x7f18d40aecd0 con 0x7f18d41572c0 2024-05-09T04:35:27.487 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.488+0000 7f18d3e00640 1 --2- 172.21.15.145:0/260134908 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f18d40d0640 0x7f18d415deb0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f18cc009d90 tx=0x7f18cc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fe0a9cb6dd602bd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:27.488 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.488+0000 7f18d2a00640 1 -- 172.21.15.145:0/260134908 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18cc007740 con 0x7f18d40d0640 2024-05-09T04:35:27.488 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.488+0000 7f18d2a00640 1 -- 172.21.15.145:0/260134908 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18cc0078e0 con 0x7f18d40d0640 2024-05-09T04:35:27.488 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.488+0000 7f18d2a00640 1 -- 172.21.15.145:0/260134908 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18cc007cb0 con 0x7f18d40d0640 2024-05-09T04:35:27.489 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.489+0000 7f18d8c00640 1 -- 172.21.15.145:0/260134908 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f18d40d0640 msgr2=0x7f18d415deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:27.489 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.489+0000 7f18d8c00640 1 --2- 172.21.15.145:0/260134908 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f18d40d0640 0x7f18d415deb0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f18cc009d90 tx=0x7f18cc02a660 comp rx=0 tx=0).stop 2024-05-09T04:35:27.489 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.489+0000 7f18d8c00640 1 -- 172.21.15.145:0/260134908 shutdown_connections 2024-05-09T04:35:27.489 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.489+0000 7f18d8c00640 1 --2- 172.21.15.145:0/260134908 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f18d40d0640 0x7f18d415deb0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:27.489 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.489+0000 7f18d8c00640 1 -- 172.21.15.145:0/260134908 >> 172.21.15.145:0/260134908 conn(0x7f18d406c8a0 msgr2=0x7f18d406ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:27.489 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.489+0000 7f18d8c00640 1 -- 172.21.15.145:0/260134908 shutdown_connections 2024-05-09T04:35:27.489 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.489+0000 7f18d8c00640 1 -- 172.21.15.145:0/260134908 wait complete. 2024-05-09T04:35:27.490 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.491+0000 7f18d8c00640 1 Processor -- start 2024-05-09T04:35:27.490 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.491+0000 7f18d8c00640 1 -- start start 2024-05-09T04:35:27.492 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.492+0000 7f18d8c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18d40ae710 0x7f18d4150a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:27.492 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.492+0000 7f18d8c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f18d40d0640 0x7f18d41f2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:27.492 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.492+0000 7f18d8c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18d41572c0 0x7f18d41e8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:27.492 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.492+0000 7f18d8c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f18d4160d00 con 0x7f18d40d0640 2024-05-09T04:35:27.492 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.492+0000 7f18d8c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f18d4161930 con 0x7f18d40ae710 2024-05-09T04:35:27.492 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.492+0000 7f18d8c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f18d41576a0 con 0x7f18d41572c0 2024-05-09T04:35:27.492 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.492+0000 7f18cbe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f18d40d0640 0x7f18d41f2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:27.492 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.492+0000 7f18cbe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f18d40d0640 0x7f18d41f2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:52886/0 (socket says 172.21.15.145:52886) 2024-05-09T04:35:27.493 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.492+0000 7f18cbe00640 1 -- 172.21.15.145:0/1062410976 learned_addr learned my addr 172.21.15.145:0/1062410976 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:27.493 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.492+0000 7f18cbe00640 1 -- 172.21.15.145:0/1062410976 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18d41572c0 msgr2=0x7f18d41e8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:27.493 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.492+0000 7f18cbe00640 1 --2- 172.21.15.145:0/1062410976 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18d41572c0 0x7f18d41e8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:27.493 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.492+0000 7f18cbe00640 1 -- 172.21.15.145:0/1062410976 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18d40ae710 msgr2=0x7f18d4150a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:27.493 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.492+0000 7f18cbe00640 1 --2- 172.21.15.145:0/1062410976 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18d40ae710 0x7f18d4150a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:27.493 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.492+0000 7f18cbe00640 1 -- 172.21.15.145:0/1062410976 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18d40b1c80 con 0x7f18d40d0640 2024-05-09T04:35:27.493 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.493+0000 7f18cbe00640 1 --2- 172.21.15.145:0/1062410976 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f18d40d0640 0x7f18d41f2a10 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f18bc00b420 tx=0x7f18bc00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:27.494 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.493+0000 7f18cb400640 1 -- 172.21.15.145:0/1062410976 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18bc011830 con 0x7f18d40d0640 2024-05-09T04:35:27.494 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.493+0000 7f18cb400640 1 -- 172.21.15.145:0/1062410976 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18bc016070 con 0x7f18d40d0640 2024-05-09T04:35:27.494 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.493+0000 7f18cb400640 1 -- 172.21.15.145:0/1062410976 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18bc011c10 con 0x7f18d40d0640 2024-05-09T04:35:27.494 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.493+0000 7f18d8c00640 1 -- 172.21.15.145:0/1062410976 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18d41e8fe0 con 0x7f18d40d0640 2024-05-09T04:35:27.495 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.493+0000 7f18d8c00640 1 -- 172.21.15.145:0/1062410976 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18d40b21e0 con 0x7f18d40d0640 2024-05-09T04:35:27.495 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.494+0000 7f18d8c00640 1 -- 172.21.15.145:0/1062410976 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18d406a310 con 0x7f18d40d0640 2024-05-09T04:35:27.495 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.495+0000 7f18cb400640 1 -- 172.21.15.145:0/1062410976 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f18bc01b460 con 0x7f18d40d0640 2024-05-09T04:35:27.496 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.496+0000 7f18cb400640 1 --2- 172.21.15.145:0/1062410976 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f18b00762d0 0x7f18b0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:27.496 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.496+0000 7f18cb400640 1 -- 172.21.15.145:0/1062410976 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f18bc0970e0 con 0x7f18d40d0640 2024-05-09T04:35:27.496 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.496+0000 7f18d3400640 1 --2- 172.21.15.145:0/1062410976 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f18b00762d0 0x7f18b0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:27.496 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.497+0000 7f18d3400640 1 --2- 172.21.15.145:0/1062410976 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f18b00762d0 0x7f18b0078790 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f18c40015d0 tx=0x7f18c40078d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:27.500 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.501+0000 7f18cb400640 1 -- 172.21.15.145:0/1062410976 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f18bc020020 con 0x7f18d40d0640 2024-05-09T04:35:27.748 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.748+0000 7f18d8c00640 1 -- 172.21.15.145:0/1062410976 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f18d40b45d0 con 0x7f18d40d0640 2024-05-09T04:35:27.749 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.750+0000 7f18cb400640 1 -- 172.21.15.145:0/1062410976 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5187 (secure 0 0 0) 0x7f18d40b1c80 con 0x7f18d40d0640 2024-05-09T04:35:27.750 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:35:27.750 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4661,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6834/1033874179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":1033874179},{"type":"v1","addr":"172.21.15.192:6834","nonce":1033874179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4664,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1787962921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6833","nonce":1787962921},{"type":"v1","addr":"172.21.15.192:6835","nonce":1787962921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5313,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6836/65063948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":65063948},{"type":"v1","addr":"172.21.15.145:6836","nonce":65063948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T04:35:23.667303+0000","modified":"2024-05-09T04:35:26.750569+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5316},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5316":{"gid":5316,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.145:6837/2593908438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6835","nonce":2593908438},{"type":"v1","addr":"172.21.15.145:6837","nonce":2593908438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5316,"qdb_cluster":[5316]},"id":3}]} 2024-05-09T04:35:27.750 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 27 2024-05-09T04:35:27.753 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.752+0000 7f18d8c00640 1 -- 172.21.15.145:0/1062410976 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f18b00762d0 msgr2=0x7f18b0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:27.753 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.752+0000 7f18d8c00640 1 --2- 172.21.15.145:0/1062410976 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f18b00762d0 0x7f18b0078790 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f18c40015d0 tx=0x7f18c40078d0 comp rx=0 tx=0).stop 2024-05-09T04:35:27.753 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.753+0000 7f18d8c00640 1 -- 172.21.15.145:0/1062410976 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f18d40d0640 msgr2=0x7f18d41f2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:27.753 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.753+0000 7f18d8c00640 1 --2- 172.21.15.145:0/1062410976 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f18d40d0640 0x7f18d41f2a10 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f18bc00b420 tx=0x7f18bc00b8e0 comp rx=0 tx=0).stop 2024-05-09T04:35:27.753 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.753+0000 7f18d8c00640 1 -- 172.21.15.145:0/1062410976 shutdown_connections 2024-05-09T04:35:27.753 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.753+0000 7f18d8c00640 1 --2- 172.21.15.145:0/1062410976 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f18b00762d0 0x7f18b0078790 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:27.753 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.753+0000 7f18d8c00640 1 --2- 172.21.15.145:0/1062410976 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18d41572c0 0x7f18d41e8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:27.753 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.753+0000 7f18d8c00640 1 --2- 172.21.15.145:0/1062410976 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f18d40d0640 0x7f18d41f2a10 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:27.754 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.753+0000 7f18d8c00640 1 --2- 172.21.15.145:0/1062410976 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18d40ae710 0x7f18d4150a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:27.754 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.753+0000 7f18d8c00640 1 -- 172.21.15.145:0/1062410976 >> 172.21.15.145:0/1062410976 conn(0x7f18d406c8a0 msgr2=0x7f18d406cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:27.754 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.753+0000 7f18d8c00640 1 -- 172.21.15.145:0/1062410976 shutdown_connections 2024-05-09T04:35:27.754 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.753+0000 7f18d8c00640 1 -- 172.21.15.145:0/1062410976 wait complete. 2024-05-09T04:35:27.772 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T04:35:27.964 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.964+0000 7f2d93200640 1 Processor -- start 2024-05-09T04:35:27.964 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.964+0000 7f2d93200640 1 -- start start 2024-05-09T04:35:27.965 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.965+0000 7f2d93200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f2d940d22f0 0x7f2d940b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:27.965 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.965+0000 7f2d93200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d940b53b0 con 0x7f2d940d5ad0 2024-05-09T04:35:27.965 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.965+0000 7f2d93200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d940d3e70 con 0x7f2d940d3180 2024-05-09T04:35:27.965 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.965+0000 7f2d93200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f2d940d4150 con 0x7f2d940d22f0 2024-05-09T04:35:27.965 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.966+0000 7f2d91400640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f2d940d5ad0 0x7f2d940b4760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:55176/0 (socket says 172.21.15.145:55176) 2024-05-09T04:35:27.965 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.966+0000 7f2d91400640 1 -- 172.21.15.145:0/1866729590 learned_addr learned my addr 172.21.15.145:0/1866729590 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:35:27.968 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.966+0000 7f2d92800640 1 --2- 172.21.15.145:0/1866729590 >> v2:172.21.15.192:3300/0 conn(0x7f2d940d22f0 0x7f2d940b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:27.968 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.966+0000 7f2d90a00640 1 -- 172.21.15.145:0/1866729590 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3193940716 0 0) 0x7f2d940b53b0 con 0x7f2d940d5ad0 2024-05-09T04:35:27.968 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.966+0000 7f2d90a00640 1 -- 172.21.15.145:0/1866729590 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d74003520 con 0x7f2d940d5ad0 2024-05-09T04:35:27.968 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.966+0000 7f2d90a00640 1 -- 172.21.15.145:0/1866729590 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2337495733 0 0) 0x7f2d74003520 con 0x7f2d940d5ad0 2024-05-09T04:35:27.969 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.966+0000 7f2d90a00640 1 -- 172.21.15.145:0/1866729590 >> v2:172.21.15.192:3300/0 conn(0x7f2d940d22f0 msgr2=0x7f2d940b4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:27.969 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.966+0000 7f2d90a00640 1 --2- 172.21.15.145:0/1866729590 >> v2:172.21.15.192:3300/0 conn(0x7f2d940d22f0 0x7f2d940b4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:27.969 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.966+0000 7f2d90a00640 1 -- 172.21.15.145:0/1866729590 >> v1:172.21.15.192:6789/0 conn(0x7f2d940d3180 legacy=0x7f2d940d3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:27.969 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.966+0000 7f2d90a00640 1 -- 172.21.15.145:0/1866729590 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d940b6450 con 0x7f2d940d5ad0 2024-05-09T04:35:27.969 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.967+0000 7f2d90a00640 1 -- 172.21.15.145:0/1866729590 <== mon.0 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3474413261 0 0) 0x7f2d7c0016f0 con 0x7f2d940d5ad0 2024-05-09T04:35:27.969 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.967+0000 7f2d90a00640 1 -- 172.21.15.145:0/1866729590 >> v1:172.21.15.145:6789/0 conn(0x7f2d940d5ad0 legacy=0x7f2d940b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:27.969 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.967+0000 7f2d90a00640 1 --2- 172.21.15.145:0/1866729590 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d74003a00 0x7f2d74003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:27.969 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.967+0000 7f2d90a00640 1 --2- 172.21.15.145:0/1866729590 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2d74004800 0x7f2d74004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:27.969 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.967+0000 7f2d90a00640 1 --2- 172.21.15.145:0/1866729590 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d74005210 0x7f2d7400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:27.969 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.967+0000 7f2d90a00640 1 -- 172.21.15.145:0/1866729590 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7400c110 con 0x7f2d74004800 2024-05-09T04:35:27.969 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.967+0000 7f2d90a00640 1 -- 172.21.15.145:0/1866729590 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7400c2c0 con 0x7f2d74005210 2024-05-09T04:35:27.969 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.967+0000 7f2d90a00640 1 -- 172.21.15.145:0/1866729590 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f2d7400c4c0 con 0x7f2d74003a00 2024-05-09T04:35:27.969 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.967+0000 7f2d91400640 1 --2- 172.21.15.145:0/1866729590 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2d74004800 0x7f2d74004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:27.969 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.967+0000 7f2d91e00640 1 --2- 172.21.15.145:0/1866729590 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d74003a00 0x7f2d74003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:27.970 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.967+0000 7f2d91400640 1 -- 172.21.15.145:0/1866729590 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d74003a00 msgr2=0x7f2d74003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:27.970 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.967+0000 7f2d91400640 1 --2- 172.21.15.145:0/1866729590 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d74003a00 0x7f2d74003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:27.970 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.967+0000 7f2d91400640 1 -- 172.21.15.145:0/1866729590 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d74005210 msgr2=0x7f2d7400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:27.970 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.967+0000 7f2d91400640 1 --2- 172.21.15.145:0/1866729590 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d74005210 0x7f2d7400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:27.970 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.967+0000 7f2d91400640 1 -- 172.21.15.145:0/1866729590 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d74003520 con 0x7f2d74004800 2024-05-09T04:35:27.970 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.967+0000 7f2d92800640 1 -- 172.21.15.145:0/1866729590 reap_dead start 2024-05-09T04:35:27.970 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.967+0000 7f2d91e00640 1 --2- 172.21.15.145:0/1866729590 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d74003a00 0x7f2d74003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:35:27.970 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.968+0000 7f2d91400640 1 --2- 172.21.15.145:0/1866729590 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2d74004800 0x7f2d74004c80 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c00c290 tx=0x7f2d74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9fc86f55c115f05 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:27.970 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.969+0000 7f2d90a00640 1 -- 172.21.15.145:0/1866729590 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c007d50 con 0x7f2d74004800 2024-05-09T04:35:27.970 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.969+0000 7f2d90a00640 1 -- 172.21.15.145:0/1866729590 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d7c005ad0 con 0x7f2d74004800 2024-05-09T04:35:27.970 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.969+0000 7f2d90a00640 1 -- 172.21.15.145:0/1866729590 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c007070 con 0x7f2d74004800 2024-05-09T04:35:27.971 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.969+0000 7f2d93200640 1 -- 172.21.15.145:0/1866729590 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2d74004800 msgr2=0x7f2d74004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:27.971 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.969+0000 7f2d93200640 1 --2- 172.21.15.145:0/1866729590 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2d74004800 0x7f2d74004c80 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c00c290 tx=0x7f2d74005830 comp rx=0 tx=0).stop 2024-05-09T04:35:27.971 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.969+0000 7f2d93200640 1 -- 172.21.15.145:0/1866729590 shutdown_connections 2024-05-09T04:35:27.972 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.969+0000 7f2d93200640 1 --2- 172.21.15.145:0/1866729590 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2d74004800 0x7f2d74004c80 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:27.972 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.969+0000 7f2d93200640 1 -- 172.21.15.145:0/1866729590 >> 172.21.15.145:0/1866729590 conn(0x7f2d940d4fb0 msgr2=0x7f2d940d53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:27.972 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.969+0000 7f2d93200640 1 -- 172.21.15.145:0/1866729590 shutdown_connections 2024-05-09T04:35:27.972 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.969+0000 7f2d93200640 1 -- 172.21.15.145:0/1866729590 wait complete. 2024-05-09T04:35:27.972 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.971+0000 7f2d93200640 1 Processor -- start 2024-05-09T04:35:27.972 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.971+0000 7f2d93200640 1 -- start start 2024-05-09T04:35:27.974 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.972+0000 7f2d93200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d74004800 0x7f2d941f44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:27.974 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.972+0000 7f2d93200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d9415b400 0x7f2d940cc080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:27.974 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.972+0000 7f2d93200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2d9415a7d0 0x7f2d9415ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:27.974 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.972+0000 7f2d93200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2d940d1e60 con 0x7f2d9415a7d0 2024-05-09T04:35:27.974 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.972+0000 7f2d93200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f2d940b6200 con 0x7f2d9415b400 2024-05-09T04:35:27.974 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.972+0000 7f2d93200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f2d940d2760 con 0x7f2d74004800 2024-05-09T04:35:27.974 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.972+0000 7f2d92800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2d9415a7d0 0x7f2d9415ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:27.974 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.973+0000 7f2d92800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2d9415a7d0 0x7f2d9415ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:52906/0 (socket says 172.21.15.145:52906) 2024-05-09T04:35:27.974 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.973+0000 7f2d92800640 1 -- 172.21.15.145:0/29452797 learned_addr learned my addr 172.21.15.145:0/29452797 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:27.975 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.973+0000 7f2d91e00640 1 --2- 172.21.15.145:0/29452797 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d74004800 0x7f2d941f44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:27.975 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.973+0000 7f2d91400640 1 --2- 172.21.15.145:0/29452797 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d9415b400 0x7f2d940cc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:27.975 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.973+0000 7f2d92800640 1 -- 172.21.15.145:0/29452797 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d74004800 msgr2=0x7f2d941f44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:27.975 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.973+0000 7f2d92800640 1 --2- 172.21.15.145:0/29452797 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d74004800 0x7f2d941f44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:27.975 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.973+0000 7f2d92800640 1 -- 172.21.15.145:0/29452797 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d9415b400 msgr2=0x7f2d940cc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:27.975 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.973+0000 7f2d92800640 1 --2- 172.21.15.145:0/29452797 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d9415b400 0x7f2d940cc080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:27.975 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.973+0000 7f2d92800640 1 -- 172.21.15.145:0/29452797 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d940cc9b0 con 0x7f2d9415a7d0 2024-05-09T04:35:27.975 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.973+0000 7f2d91e00640 1 --2- 172.21.15.145:0/29452797 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d74004800 0x7f2d941f44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:35:27.975 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.973+0000 7f2d91400640 1 --2- 172.21.15.145:0/29452797 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d9415b400 0x7f2d940cc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:35:27.975 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.973+0000 7f2d92800640 1 --2- 172.21.15.145:0/29452797 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2d9415a7d0 0x7f2d9415ac50 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f2d9408e470 tx=0x7f2d8c00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:27.975 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.973+0000 7f2d8a000640 1 -- 172.21.15.145:0/29452797 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c004030 con 0x7f2d9415a7d0 2024-05-09T04:35:27.975 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.974+0000 7f2d8a000640 1 -- 172.21.15.145:0/29452797 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d8c019070 con 0x7f2d9415a7d0 2024-05-09T04:35:27.975 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.974+0000 7f2d8a000640 1 -- 172.21.15.145:0/29452797 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c004410 con 0x7f2d9415a7d0 2024-05-09T04:35:27.975 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.974+0000 7f2d93200640 1 -- 172.21.15.145:0/29452797 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d940ccc90 con 0x7f2d9415a7d0 2024-05-09T04:35:27.976 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.974+0000 7f2d93200640 1 -- 172.21.15.145:0/29452797 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d940cd250 con 0x7f2d9415a7d0 2024-05-09T04:35:27.976 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.975+0000 7f2d93200640 1 -- 172.21.15.145:0/29452797 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d9406a310 con 0x7f2d9415a7d0 2024-05-09T04:35:27.976 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.976+0000 7f2d8a000640 1 -- 172.21.15.145:0/29452797 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2d8c00c680 con 0x7f2d9415a7d0 2024-05-09T04:35:27.977 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.977+0000 7f2d8a000640 1 --2- 172.21.15.145:0/29452797 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f2d700762d0 0x7f2d70078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:27.977 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.978+0000 7f2d8a000640 1 -- 172.21.15.145:0/29452797 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f2d8c097380 con 0x7f2d9415a7d0 2024-05-09T04:35:27.977 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.978+0000 7f2d91e00640 1 --2- 172.21.15.145:0/29452797 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f2d700762d0 0x7f2d70078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:27.978 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.978+0000 7f2d91e00640 1 --2- 172.21.15.145:0/29452797 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f2d700762d0 0x7f2d70078790 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2d9406be20 tx=0x7f2d84009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:27.982 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:27.982+0000 7f2d8a000640 1 -- 172.21.15.145:0/29452797 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2d8c023020 con 0x7f2d9415a7d0 2024-05-09T04:35:28.188 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.188+0000 7f2d93200640 1 -- 172.21.15.145:0/29452797 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2d940d0250 con 0x7f2d9415a7d0 2024-05-09T04:35:28.191 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.191+0000 7f2d8a000640 1 -- 172.21.15.145:0/29452797 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+16717 (secure 0 0 0) 0x7f2d8c060c40 con 0x7f2d9415a7d0 2024-05-09T04:35:28.191 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:35:28.191 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":47,"fsid":"957ee6c1-073b-44cf-9b93-9491b3fdfc1d","created":"2024-05-09T04:33:26.544367+0000","modified":"2024-05-09T04:35:25.231876+0000","last_up_change":"2024-05-09T04:33:39.039318+0000","last_in_change":"2024-05-09T04:33:29.902361+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T04:33:39.795344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-09T04:35:21.570048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-09T04:35:22.639737+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22e550ad-847b-4952-80e6-63d859ed1a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6801","nonce":3182572912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6803","nonce":3182572912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6807","nonce":3182572912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6805","nonce":3182572912}]},"public_addr":"172.21.15.145:6801/3182572912","cluster_addr":"172.21.15.145:6803/3182572912","heartbeat_back_addr":"172.21.15.145:6807/3182572912","heartbeat_front_addr":"172.21.15.145:6805/3182572912","state":["exists","up"]},{"osd":1,"uuid":"0f6b9fca-0c41-4145-8d74-598ddb42f2a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6825","nonce":1845225555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6827","nonce":1845225555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6831","nonce":1845225555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6829","nonce":1845225555}]},"public_addr":"172.21.15.145:6825/1845225555","cluster_addr":"172.21.15.145:6827/1845225555","heartbeat_back_addr":"172.21.15.145:6831/1845225555","heartbeat_front_addr":"172.21.15.145:6829/1845225555","state":["exists","up"]},{"osd":2,"uuid":"b3b1f67e-5790-483f-8610-e05a5bce4c30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6817","nonce":2168958993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6819","nonce":2168958993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6823","nonce":2168958993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6821","nonce":2168958993}]},"public_addr":"172.21.15.145:6817/2168958993","cluster_addr":"172.21.15.145:6819/2168958993","heartbeat_back_addr":"172.21.15.145:6823/2168958993","heartbeat_front_addr":"172.21.15.145:6821/2168958993","state":["exists","up"]},{"osd":3,"uuid":"884787a5-e037-4d31-9aae-743030eba4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6809","nonce":337009826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6811","nonce":337009826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6815","nonce":337009826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6813","nonce":337009826}]},"public_addr":"172.21.15.145:6809/337009826","cluster_addr":"172.21.15.145:6811/337009826","heartbeat_back_addr":"172.21.15.145:6815/337009826","heartbeat_front_addr":"172.21.15.145:6813/337009826","state":["exists","up"]},{"osd":4,"uuid":"0759fc1e-c5fc-4a2f-86bb-e8eccd83f833","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6818","nonce":3117700995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6821","nonce":3117700995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6827","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6829","nonce":3117700995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6823","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6825","nonce":3117700995}]},"public_addr":"172.21.15.192:6818/3117700995","cluster_addr":"172.21.15.192:6821/3117700995","heartbeat_back_addr":"172.21.15.192:6829/3117700995","heartbeat_front_addr":"172.21.15.192:6825/3117700995","state":["exists","up"]},{"osd":5,"uuid":"9443cc22-9379-46e3-9173-98b592bf2c16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6801","nonce":1328373268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6803","nonce":1328373268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6807","nonce":1328373268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6805","nonce":1328373268}]},"public_addr":"172.21.15.192:6801/1328373268","cluster_addr":"172.21.15.192:6803/1328373268","heartbeat_back_addr":"172.21.15.192:6807/1328373268","heartbeat_front_addr":"172.21.15.192:6805/1328373268","state":["exists","up"]},{"osd":6,"uuid":"d05a3311-505a-45b5-85b9-aa3113e96987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6817","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6820","nonce":1868011057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6824","nonce":1868011057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6831","nonce":1868011057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6828","nonce":1868011057}]},"public_addr":"172.21.15.192:6820/1868011057","cluster_addr":"172.21.15.192:6824/1868011057","heartbeat_back_addr":"172.21.15.192:6831/1868011057","heartbeat_front_addr":"172.21.15.192:6828/1868011057","state":["exists","up"]},{"osd":7,"uuid":"c7c23dbc-2029-43ec-9f53-25f3d58019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6809","nonce":3368512157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6811","nonce":3368512157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6815","nonce":3368512157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6813","nonce":3368512157}]},"public_addr":"172.21.15.192:6809/3368512157","cluster_addr":"172.21.15.192:6811/3368512157","heartbeat_back_addr":"172.21.15.192:6815/3368512157","heartbeat_front_addr":"172.21.15.192:6813/3368512157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.091188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.032567+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:34.797468+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.228839+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.622400+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.473103+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.566092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.349395+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T04:35:28.194 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.193+0000 7f2d93200640 1 -- 172.21.15.145:0/29452797 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f2d700762d0 msgr2=0x7f2d70078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:28.194 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.193+0000 7f2d93200640 1 --2- 172.21.15.145:0/29452797 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f2d700762d0 0x7f2d70078790 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2d9406be20 tx=0x7f2d84009040 comp rx=0 tx=0).stop 2024-05-09T04:35:28.194 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.194+0000 7f2d93200640 1 -- 172.21.15.145:0/29452797 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2d9415a7d0 msgr2=0x7f2d9415ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:28.194 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.194+0000 7f2d93200640 1 --2- 172.21.15.145:0/29452797 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2d9415a7d0 0x7f2d9415ac50 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f2d9408e470 tx=0x7f2d8c00ec00 comp rx=0 tx=0).stop 2024-05-09T04:35:28.194 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.194+0000 7f2d93200640 1 -- 172.21.15.145:0/29452797 shutdown_connections 2024-05-09T04:35:28.194 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.194+0000 7f2d93200640 1 --2- 172.21.15.145:0/29452797 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f2d700762d0 0x7f2d70078790 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:28.194 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.194+0000 7f2d93200640 1 --2- 172.21.15.145:0/29452797 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f2d9415a7d0 0x7f2d9415ac50 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:28.194 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.194+0000 7f2d93200640 1 --2- 172.21.15.145:0/29452797 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d9415b400 0x7f2d940cc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:28.194 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.194+0000 7f2d93200640 1 --2- 172.21.15.145:0/29452797 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d74004800 0x7f2d941f44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:28.194 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.194+0000 7f2d93200640 1 -- 172.21.15.145:0/29452797 >> 172.21.15.145:0/29452797 conn(0x7f2d940d4fb0 msgr2=0x7f2d94158530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:28.194 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.194+0000 7f2d93200640 1 -- 172.21.15.145:0/29452797 shutdown_connections 2024-05-09T04:35:28.194 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.194+0000 7f2d93200640 1 -- 172.21.15.145:0/29452797 wait complete. 2024-05-09T04:35:28.213 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-09T04:35:28.213 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T04:35:28.406 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.406+0000 7f9c1c200640 1 Processor -- start 2024-05-09T04:35:28.406 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.407+0000 7f9c1c200640 1 -- start start 2024-05-09T04:35:28.408 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.408+0000 7f9c1c200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f9c1415d780 0x7f9c14160c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:28.408 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.408+0000 7f9c1c200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9c140b4910 0x7f9c140b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:28.409 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.408+0000 7f9c1c200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c141615c0 con 0x7f9c1406e860 2024-05-09T04:35:28.409 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.408+0000 7f9c1c200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f9c141618a0 con 0x7f9c1415d780 2024-05-09T04:35:28.409 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.408+0000 7f9c1c200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9c140b4110 con 0x7f9c140b4910 2024-05-09T04:35:28.409 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.408+0000 7f9c19800640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f9c1406e860 0x7f9c1406ec40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:55182/0 (socket says 172.21.15.145:55182) 2024-05-09T04:35:28.409 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.408+0000 7f9c19800640 1 -- 172.21.15.145:0/2905872984 learned_addr learned my addr 172.21.15.145:0/2905872984 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:35:28.409 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.408+0000 7f9c18e00640 1 --2- 172.21.15.145:0/2905872984 >> v2:172.21.15.145:3300/0 conn(0x7f9c1415d780 0x7f9c14160c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:28.409 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.409+0000 7f9c1a200640 1 --2- 172.21.15.145:0/2905872984 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9c140b4910 0x7f9c140b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:28.409 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.409+0000 7f9c13e00640 1 -- 172.21.15.145:0/2905872984 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2377405988 0 0) 0x7f9c141615c0 con 0x7f9c1406e860 2024-05-09T04:35:28.409 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.409+0000 7f9c18e00640 1 -- 172.21.15.145:0/2905872984 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9c140b4910 msgr2=0x7f9c140b4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:28.409 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.409+0000 7f9c18e00640 1 --2- 172.21.15.145:0/2905872984 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9c140b4910 0x7f9c140b4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:28.409 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.409+0000 7f9c18e00640 1 -- 172.21.15.145:0/2905872984 >> v1:172.21.15.145:6789/0 conn(0x7f9c1406e860 legacy=0x7f9c1406ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:28.409 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.409+0000 7f9c18e00640 1 -- 172.21.15.145:0/2905872984 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c140b43f0 con 0x7f9c1415d780 2024-05-09T04:35:28.409 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.409+0000 7f9c1a200640 1 --2- 172.21.15.145:0/2905872984 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9c140b4910 0x7f9c140b4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:35:28.410 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.410+0000 7f9c18e00640 1 --2- 172.21.15.145:0/2905872984 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9c1415d780 0x7f9c14160c50 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9c000097a0 tx=0x7f9c0002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=758de00e14abaedf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:28.411 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.410+0000 7f9c13e00640 1 -- 172.21.15.145:0/2905872984 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c00037070 con 0x7f9c1415d780 2024-05-09T04:35:28.411 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.410+0000 7f9c13e00640 1 -- 172.21.15.145:0/2905872984 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c00032440 con 0x7f9c1415d780 2024-05-09T04:35:28.411 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.410+0000 7f9c13e00640 1 -- 172.21.15.145:0/2905872984 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c00032810 con 0x7f9c1415d780 2024-05-09T04:35:28.411 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.410+0000 7f9c1c200640 1 -- 172.21.15.145:0/2905872984 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9c1415d780 msgr2=0x7f9c14160c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:28.411 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.410+0000 7f9c1c200640 1 --2- 172.21.15.145:0/2905872984 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9c1415d780 0x7f9c14160c50 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9c000097a0 tx=0x7f9c0002a180 comp rx=0 tx=0).stop 2024-05-09T04:35:28.411 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.411+0000 7f9c1c200640 1 -- 172.21.15.145:0/2905872984 shutdown_connections 2024-05-09T04:35:28.411 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.411+0000 7f9c1c200640 1 --2- 172.21.15.145:0/2905872984 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9c140b4910 0x7f9c140b4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:28.411 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.411+0000 7f9c1c200640 1 --2- 172.21.15.145:0/2905872984 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9c1415d780 0x7f9c14160c50 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:28.411 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.411+0000 7f9c1c200640 1 -- 172.21.15.145:0/2905872984 >> 172.21.15.145:0/2905872984 conn(0x7f9c1406c870 msgr2=0x7f9c1406cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:28.411 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.411+0000 7f9c1c200640 1 -- 172.21.15.145:0/2905872984 shutdown_connections 2024-05-09T04:35:28.411 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.411+0000 7f9c1c200640 1 -- 172.21.15.145:0/2905872984 wait complete. 2024-05-09T04:35:28.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.412+0000 7f9c1c200640 1 Processor -- start 2024-05-09T04:35:28.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.413+0000 7f9c1c200640 1 -- start start 2024-05-09T04:35:28.414 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.413+0000 7f9c1c200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9c1406e860 0x7f9c141449e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:28.415 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.414+0000 7f9c1c200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c140b4910 0x7f9c1413a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:28.415 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.414+0000 7f9c1c200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9c1415d780 0x7f9c140ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:28.415 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.414+0000 7f9c1c200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9c14161190 con 0x7f9c1406e860 2024-05-09T04:35:28.415 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.414+0000 7f9c1c200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9c14146e10 con 0x7f9c140b4910 2024-05-09T04:35:28.415 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.414+0000 7f9c1c200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9c14161310 con 0x7f9c1415d780 2024-05-09T04:35:28.415 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.414+0000 7f9c19800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9c1406e860 0x7f9c141449e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:28.415 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.414+0000 7f9c19800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9c1406e860 0x7f9c141449e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:52926/0 (socket says 172.21.15.145:52926) 2024-05-09T04:35:28.415 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.414+0000 7f9c19800640 1 -- 172.21.15.145:0/3912612110 learned_addr learned my addr 172.21.15.145:0/3912612110 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:28.415 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.414+0000 7f9c19800640 1 -- 172.21.15.145:0/3912612110 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9c1415d780 msgr2=0x7f9c140ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:28.415 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.414+0000 7f9c19800640 1 --2- 172.21.15.145:0/3912612110 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9c1415d780 0x7f9c140ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:28.415 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.414+0000 7f9c19800640 1 -- 172.21.15.145:0/3912612110 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c140b4910 msgr2=0x7f9c1413a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:28.415 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.414+0000 7f9c19800640 1 --2- 172.21.15.145:0/3912612110 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c140b4910 0x7f9c1413a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:28.415 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.414+0000 7f9c19800640 1 -- 172.21.15.145:0/3912612110 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c14140fa0 con 0x7f9c1406e860 2024-05-09T04:35:28.416 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.414+0000 7f9c19800640 1 --2- 172.21.15.145:0/3912612110 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9c1406e860 0x7f9c141449e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f9c0800c410 tx=0x7f9c0800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:28.416 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.415+0000 7f9c11600640 1 -- 172.21.15.145:0/3912612110 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c08017070 con 0x7f9c1406e860 2024-05-09T04:35:28.416 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.415+0000 7f9c11600640 1 -- 172.21.15.145:0/3912612110 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c08005d00 con 0x7f9c1406e860 2024-05-09T04:35:28.416 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.415+0000 7f9c11600640 1 -- 172.21.15.145:0/3912612110 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c08007310 con 0x7f9c1406e860 2024-05-09T04:35:28.416 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.415+0000 7f9c1c200640 1 -- 172.21.15.145:0/3912612110 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c141413d0 con 0x7f9c1406e860 2024-05-09T04:35:28.416 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.415+0000 7f9c1c200640 1 -- 172.21.15.145:0/3912612110 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c14141990 con 0x7f9c1406e860 2024-05-09T04:35:28.416 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.416+0000 7f9c1c200640 1 -- 172.21.15.145:0/3912612110 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c1406a310 con 0x7f9c1406e860 2024-05-09T04:35:28.417 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.417+0000 7f9c11600640 1 -- 172.21.15.145:0/3912612110 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9c0801a070 con 0x7f9c1406e860 2024-05-09T04:35:28.419 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.419+0000 7f9c11600640 1 --2- 172.21.15.145:0/3912612110 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f9bf40762d0 0x7f9bf4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:28.419 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.419+0000 7f9c11600640 1 -- 172.21.15.145:0/3912612110 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f9c08096a60 con 0x7f9c1406e860 2024-05-09T04:35:28.419 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.419+0000 7f9c18e00640 1 --2- 172.21.15.145:0/3912612110 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f9bf40762d0 0x7f9bf4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:28.419 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.420+0000 7f9c18e00640 1 --2- 172.21.15.145:0/3912612110 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f9bf40762d0 0x7f9bf4078790 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9c00034000 tx=0x7f9c0002c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:28.423 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.424+0000 7f9c11600640 1 -- 172.21.15.145:0/3912612110 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f9c080603b0 con 0x7f9c1406e860 2024-05-09T04:35:28.628 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.628+0000 7f9c1c200640 1 -- 172.21.15.145:0/3912612110 --> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9c1406f5f0 con 0x7f9bf40762d0 2024-05-09T04:35:28.638 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.638+0000 7f9c11600640 1 -- 172.21.15.145:0/3912612110 <== mgr.4104 v2:172.21.15.145:6832/2725855396 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251875 (secure 0 0 0) 0x7f9c1406f5f0 con 0x7f9bf40762d0 2024-05-09T04:35:28.641 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:35:28.643 INFO:teuthology.orchestra.run.smithi145.stderr:dumped all 2024-05-09T04:35:28.644 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.643+0000 7f9c1c200640 1 -- 172.21.15.145:0/3912612110 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f9bf40762d0 msgr2=0x7f9bf4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:28.644 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.643+0000 7f9c1c200640 1 --2- 172.21.15.145:0/3912612110 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f9bf40762d0 0x7f9bf4078790 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9c00034000 tx=0x7f9c0002c000 comp rx=0 tx=0).stop 2024-05-09T04:35:28.645 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.643+0000 7f9c1c200640 1 -- 172.21.15.145:0/3912612110 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9c1406e860 msgr2=0x7f9c141449e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:28.645 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.643+0000 7f9c1c200640 1 --2- 172.21.15.145:0/3912612110 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9c1406e860 0x7f9c141449e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f9c0800c410 tx=0x7f9c0800c8d0 comp rx=0 tx=0).stop 2024-05-09T04:35:28.645 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.643+0000 7f9c1c200640 1 -- 172.21.15.145:0/3912612110 shutdown_connections 2024-05-09T04:35:28.645 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.643+0000 7f9c1c200640 1 --2- 172.21.15.145:0/3912612110 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f9bf40762d0 0x7f9bf4078790 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:28.645 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.643+0000 7f9c1c200640 1 --2- 172.21.15.145:0/3912612110 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9c1415d780 0x7f9c140ce7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:28.645 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.643+0000 7f9c1c200640 1 --2- 172.21.15.145:0/3912612110 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c140b4910 0x7f9c1413a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:28.645 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.643+0000 7f9c1c200640 1 --2- 172.21.15.145:0/3912612110 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9c1406e860 0x7f9c141449e0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:28.645 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.643+0000 7f9c1c200640 1 -- 172.21.15.145:0/3912612110 >> 172.21.15.145:0/3912612110 conn(0x7f9c1406c870 msgr2=0x7f9c140b3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:28.645 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.643+0000 7f9c1c200640 1 -- 172.21.15.145:0/3912612110 shutdown_connections 2024-05-09T04:35:28.645 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.643+0000 7f9c1c200640 1 -- 172.21.15.145:0/3912612110 wait complete. 2024-05-09T04:35:28.662 INFO:teuthology.orchestra.run.smithi145.stdout:{"pg_ready":true,"pg_map":{"version":94,"stamp":"2024-05-09T04:35:27.784210+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":229980,"kb_used_data":5452,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754744740,"statfs":{"total":773094113280,"available":772858613760,"internally_reserved":0,"allocated":5582848,"data_stored":2972099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":229756465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.598465"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.258715+0000","last_change":"2024-05-09T04:35:24.253736+0000","last_active":"2024-05-09T04:35:24.258715+0000","last_peered":"2024-05-09T04:35:24.258715+0000","last_clean":"2024-05-09T04:35:24.258715+0000","last_became_active":"2024-05-09T04:35:24.252217+0000","last_became_peered":"2024-05-09T04:35:24.252217+0000","last_unstale":"2024-05-09T04:35:24.258715+0000","last_undegraded":"2024-05-09T04:35:24.258715+0000","last_fullsized":"2024-05-09T04:35:24.258715+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:06:11.802530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.262929+0000","last_change":"2024-05-09T04:35:24.257063+0000","last_active":"2024-05-09T04:35:24.262929+0000","last_peered":"2024-05-09T04:35:24.262929+0000","last_clean":"2024-05-09T04:35:24.262929+0000","last_became_active":"2024-05-09T04:35:24.256800+0000","last_became_peered":"2024-05-09T04:35:24.256800+0000","last_unstale":"2024-05-09T04:35:24.262929+0000","last_undegraded":"2024-05-09T04:35:24.262929+0000","last_fullsized":"2024-05-09T04:35:24.262929+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:03:08.472325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.257620+0000","last_change":"2024-05-09T04:35:24.252124+0000","last_active":"2024-05-09T04:35:24.257620+0000","last_peered":"2024-05-09T04:35:24.257620+0000","last_clean":"2024-05-09T04:35:24.257620+0000","last_became_active":"2024-05-09T04:35:24.251901+0000","last_became_peered":"2024-05-09T04:35:24.251901+0000","last_unstale":"2024-05-09T04:35:24.257620+0000","last_undegraded":"2024-05-09T04:35:24.257620+0000","last_fullsized":"2024-05-09T04:35:24.257620+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:09:14.673363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.271430+0000","last_change":"2024-05-09T04:35:24.269122+0000","last_active":"2024-05-09T04:35:24.271430+0000","last_peered":"2024-05-09T04:35:24.271430+0000","last_clean":"2024-05-09T04:35:24.271430+0000","last_became_active":"2024-05-09T04:35:24.268794+0000","last_became_peered":"2024-05-09T04:35:24.268794+0000","last_unstale":"2024-05-09T04:35:24.271430+0000","last_undegraded":"2024-05-09T04:35:24.271430+0000","last_fullsized":"2024-05-09T04:35:24.271430+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:24:08.787200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.258302+0000","last_change":"2024-05-09T04:35:24.255247+0000","last_active":"2024-05-09T04:35:24.258302+0000","last_peered":"2024-05-09T04:35:24.258302+0000","last_clean":"2024-05-09T04:35:24.258302+0000","last_became_active":"2024-05-09T04:35:24.254540+0000","last_became_peered":"2024-05-09T04:35:24.254540+0000","last_unstale":"2024-05-09T04:35:24.258302+0000","last_undegraded":"2024-05-09T04:35:24.258302+0000","last_fullsized":"2024-05-09T04:35:24.258302+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:10:10.774894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.258656+0000","last_change":"2024-05-09T04:35:24.254553+0000","last_active":"2024-05-09T04:35:24.258656+0000","last_peered":"2024-05-09T04:35:24.258656+0000","last_clean":"2024-05-09T04:35:24.258656+0000","last_became_active":"2024-05-09T04:35:24.254351+0000","last_became_peered":"2024-05-09T04:35:24.254351+0000","last_unstale":"2024-05-09T04:35:24.258656+0000","last_undegraded":"2024-05-09T04:35:24.258656+0000","last_fullsized":"2024-05-09T04:35:24.258656+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:48:53.260192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.255475+0000","last_change":"2024-05-09T04:35:24.244839+0000","last_active":"2024-05-09T04:35:24.255475+0000","last_peered":"2024-05-09T04:35:24.255475+0000","last_clean":"2024-05-09T04:35:24.255475+0000","last_became_active":"2024-05-09T04:35:24.244566+0000","last_became_peered":"2024-05-09T04:35:24.244566+0000","last_unstale":"2024-05-09T04:35:24.255475+0000","last_undegraded":"2024-05-09T04:35:24.255475+0000","last_fullsized":"2024-05-09T04:35:24.255475+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:11:24.172582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.268337+0000","last_change":"2024-05-09T04:35:24.260670+0000","last_active":"2024-05-09T04:35:24.268337+0000","last_peered":"2024-05-09T04:35:24.268337+0000","last_clean":"2024-05-09T04:35:24.268337+0000","last_became_active":"2024-05-09T04:35:24.257218+0000","last_became_peered":"2024-05-09T04:35:24.257218+0000","last_unstale":"2024-05-09T04:35:24.268337+0000","last_undegraded":"2024-05-09T04:35:24.268337+0000","last_fullsized":"2024-05-09T04:35:24.268337+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:04:25.956768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.268664+0000","last_change":"2024-05-09T04:35:24.267325+0000","last_active":"2024-05-09T04:35:24.268664+0000","last_peered":"2024-05-09T04:35:24.268664+0000","last_clean":"2024-05-09T04:35:24.268664+0000","last_became_active":"2024-05-09T04:35:24.265960+0000","last_became_peered":"2024-05-09T04:35:24.265960+0000","last_unstale":"2024-05-09T04:35:24.268664+0000","last_undegraded":"2024-05-09T04:35:24.268664+0000","last_fullsized":"2024-05-09T04:35:24.268664+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:39:13.548453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.244809+0000","last_change":"2024-05-09T04:35:24.240356+0000","last_active":"2024-05-09T04:35:24.244809+0000","last_peered":"2024-05-09T04:35:24.244809+0000","last_clean":"2024-05-09T04:35:24.244809+0000","last_became_active":"2024-05-09T04:35:24.239958+0000","last_became_peered":"2024-05-09T04:35:24.239958+0000","last_unstale":"2024-05-09T04:35:24.244809+0000","last_undegraded":"2024-05-09T04:35:24.244809+0000","last_fullsized":"2024-05-09T04:35:24.244809+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:50:40.529895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.261964+0000","last_change":"2024-05-09T04:35:24.255043+0000","last_active":"2024-05-09T04:35:24.261964+0000","last_peered":"2024-05-09T04:35:24.261964+0000","last_clean":"2024-05-09T04:35:24.261964+0000","last_became_active":"2024-05-09T04:35:24.254124+0000","last_became_peered":"2024-05-09T04:35:24.254124+0000","last_unstale":"2024-05-09T04:35:24.261964+0000","last_undegraded":"2024-05-09T04:35:24.261964+0000","last_fullsized":"2024-05-09T04:35:24.261964+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:55:13.620010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.268670+0000","last_change":"2024-05-09T04:35:24.264561+0000","last_active":"2024-05-09T04:35:24.268670+0000","last_peered":"2024-05-09T04:35:24.268670+0000","last_clean":"2024-05-09T04:35:24.268670+0000","last_became_active":"2024-05-09T04:35:24.263961+0000","last_became_peered":"2024-05-09T04:35:24.263961+0000","last_unstale":"2024-05-09T04:35:24.268670+0000","last_undegraded":"2024-05-09T04:35:24.268670+0000","last_fullsized":"2024-05-09T04:35:24.268670+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:32:06.767185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.251060+0000","last_change":"2024-05-09T04:35:24.246799+0000","last_active":"2024-05-09T04:35:24.251060+0000","last_peered":"2024-05-09T04:35:24.251060+0000","last_clean":"2024-05-09T04:35:24.251060+0000","last_became_active":"2024-05-09T04:35:24.245105+0000","last_became_peered":"2024-05-09T04:35:24.245105+0000","last_unstale":"2024-05-09T04:35:24.251060+0000","last_undegraded":"2024-05-09T04:35:24.251060+0000","last_fullsized":"2024-05-09T04:35:24.251060+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:48:01.703927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.258281+0000","last_change":"2024-05-09T04:35:24.250234+0000","last_active":"2024-05-09T04:35:24.258281+0000","last_peered":"2024-05-09T04:35:24.258281+0000","last_clean":"2024-05-09T04:35:24.258281+0000","last_became_active":"2024-05-09T04:35:24.249914+0000","last_became_peered":"2024-05-09T04:35:24.249914+0000","last_unstale":"2024-05-09T04:35:24.258281+0000","last_undegraded":"2024-05-09T04:35:24.258281+0000","last_fullsized":"2024-05-09T04:35:24.258281+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:20:57.932878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.268335+0000","last_change":"2024-05-09T04:35:24.264230+0000","last_active":"2024-05-09T04:35:24.268335+0000","last_peered":"2024-05-09T04:35:24.268335+0000","last_clean":"2024-05-09T04:35:24.268335+0000","last_became_active":"2024-05-09T04:35:24.263356+0000","last_became_peered":"2024-05-09T04:35:24.263356+0000","last_unstale":"2024-05-09T04:35:24.268335+0000","last_undegraded":"2024-05-09T04:35:24.268335+0000","last_fullsized":"2024-05-09T04:35:24.268335+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:29:15.109166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.243385+0000","last_change":"2024-05-09T04:35:24.236902+0000","last_active":"2024-05-09T04:35:24.243385+0000","last_peered":"2024-05-09T04:35:24.243385+0000","last_clean":"2024-05-09T04:35:24.243385+0000","last_became_active":"2024-05-09T04:35:24.234449+0000","last_became_peered":"2024-05-09T04:35:24.234449+0000","last_unstale":"2024-05-09T04:35:24.243385+0000","last_undegraded":"2024-05-09T04:35:24.243385+0000","last_fullsized":"2024-05-09T04:35:24.243385+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:38:58.731420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.259963+0000","last_change":"2024-05-09T04:35:24.245669+0000","last_active":"2024-05-09T04:35:24.259963+0000","last_peered":"2024-05-09T04:35:24.259963+0000","last_clean":"2024-05-09T04:35:24.259963+0000","last_became_active":"2024-05-09T04:35:24.242139+0000","last_became_peered":"2024-05-09T04:35:24.242139+0000","last_unstale":"2024-05-09T04:35:24.259963+0000","last_undegraded":"2024-05-09T04:35:24.259963+0000","last_fullsized":"2024-05-09T04:35:24.259963+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:48:01.703927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.263690+0000","last_change":"2024-05-09T04:35:24.257226+0000","last_active":"2024-05-09T04:35:24.263690+0000","last_peered":"2024-05-09T04:35:24.263690+0000","last_clean":"2024-05-09T04:35:24.263690+0000","last_became_active":"2024-05-09T04:35:24.256968+0000","last_became_peered":"2024-05-09T04:35:24.256968+0000","last_unstale":"2024-05-09T04:35:24.263690+0000","last_undegraded":"2024-05-09T04:35:24.263690+0000","last_fullsized":"2024-05-09T04:35:24.263690+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:19:27.566627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.257347+0000","last_change":"2024-05-09T04:35:24.247805+0000","last_active":"2024-05-09T04:35:24.257347+0000","last_peered":"2024-05-09T04:35:24.257347+0000","last_clean":"2024-05-09T04:35:24.257347+0000","last_became_active":"2024-05-09T04:35:24.247459+0000","last_became_peered":"2024-05-09T04:35:24.247459+0000","last_unstale":"2024-05-09T04:35:24.257347+0000","last_undegraded":"2024-05-09T04:35:24.257347+0000","last_fullsized":"2024-05-09T04:35:24.257347+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:15:49.152245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.262799+0000","last_change":"2024-05-09T04:35:24.258292+0000","last_active":"2024-05-09T04:35:24.262799+0000","last_peered":"2024-05-09T04:35:24.262799+0000","last_clean":"2024-05-09T04:35:24.262799+0000","last_became_active":"2024-05-09T04:35:24.257967+0000","last_became_peered":"2024-05-09T04:35:24.257967+0000","last_unstale":"2024-05-09T04:35:24.262799+0000","last_undegraded":"2024-05-09T04:35:24.262799+0000","last_fullsized":"2024-05-09T04:35:24.262799+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:03:01.497601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.262244+0000","last_change":"2024-05-09T04:35:24.259180+0000","last_active":"2024-05-09T04:35:24.262244+0000","last_peered":"2024-05-09T04:35:24.262244+0000","last_clean":"2024-05-09T04:35:24.262244+0000","last_became_active":"2024-05-09T04:35:24.258884+0000","last_became_peered":"2024-05-09T04:35:24.258884+0000","last_unstale":"2024-05-09T04:35:24.262244+0000","last_undegraded":"2024-05-09T04:35:24.262244+0000","last_fullsized":"2024-05-09T04:35:24.262244+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:28:54.503580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.251797+0000","last_change":"2024-05-09T04:35:24.249166+0000","last_active":"2024-05-09T04:35:24.251797+0000","last_peered":"2024-05-09T04:35:24.251797+0000","last_clean":"2024-05-09T04:35:24.251797+0000","last_became_active":"2024-05-09T04:35:24.246642+0000","last_became_peered":"2024-05-09T04:35:24.246642+0000","last_unstale":"2024-05-09T04:35:24.251797+0000","last_undegraded":"2024-05-09T04:35:24.251797+0000","last_fullsized":"2024-05-09T04:35:24.251797+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:24:08.787200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.261943+0000","last_change":"2024-05-09T04:35:24.254719+0000","last_active":"2024-05-09T04:35:24.261943+0000","last_peered":"2024-05-09T04:35:24.261943+0000","last_clean":"2024-05-09T04:35:24.261943+0000","last_became_active":"2024-05-09T04:35:24.254495+0000","last_became_peered":"2024-05-09T04:35:24.254495+0000","last_unstale":"2024-05-09T04:35:24.261943+0000","last_undegraded":"2024-05-09T04:35:24.261943+0000","last_fullsized":"2024-05-09T04:35:24.261943+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:38:26.054002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.247016+0000","last_change":"2024-05-09T04:35:24.239761+0000","last_active":"2024-05-09T04:35:24.247016+0000","last_peered":"2024-05-09T04:35:24.247016+0000","last_clean":"2024-05-09T04:35:24.247016+0000","last_became_active":"2024-05-09T04:35:24.237890+0000","last_became_peered":"2024-05-09T04:35:24.237890+0000","last_unstale":"2024-05-09T04:35:24.247016+0000","last_undegraded":"2024-05-09T04:35:24.247016+0000","last_fullsized":"2024-05-09T04:35:24.247016+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:17:18.226996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.265125+0000","last_change":"2024-05-09T04:35:24.261402+0000","last_active":"2024-05-09T04:35:24.265125+0000","last_peered":"2024-05-09T04:35:24.265125+0000","last_clean":"2024-05-09T04:35:24.265125+0000","last_became_active":"2024-05-09T04:35:24.259786+0000","last_became_peered":"2024-05-09T04:35:24.259786+0000","last_unstale":"2024-05-09T04:35:24.265125+0000","last_undegraded":"2024-05-09T04:35:24.265125+0000","last_fullsized":"2024-05-09T04:35:24.265125+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:22:35.619585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.266247+0000","last_change":"2024-05-09T04:35:24.253325+0000","last_active":"2024-05-09T04:35:24.266247+0000","last_peered":"2024-05-09T04:35:24.266247+0000","last_clean":"2024-05-09T04:35:24.266247+0000","last_became_active":"2024-05-09T04:35:24.253073+0000","last_became_peered":"2024-05-09T04:35:24.253073+0000","last_unstale":"2024-05-09T04:35:24.266247+0000","last_undegraded":"2024-05-09T04:35:24.266247+0000","last_fullsized":"2024-05-09T04:35:24.266247+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:54:30.099491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.238834+0000","last_change":"2024-05-09T04:35:24.235550+0000","last_active":"2024-05-09T04:35:24.238834+0000","last_peered":"2024-05-09T04:35:24.238834+0000","last_clean":"2024-05-09T04:35:24.238834+0000","last_became_active":"2024-05-09T04:35:24.235259+0000","last_became_peered":"2024-05-09T04:35:24.235259+0000","last_unstale":"2024-05-09T04:35:24.238834+0000","last_undegraded":"2024-05-09T04:35:24.238834+0000","last_fullsized":"2024-05-09T04:35:24.238834+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:10:10.774894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.263024+0000","last_change":"2024-05-09T04:35:24.262004+0000","last_active":"2024-05-09T04:35:24.263024+0000","last_peered":"2024-05-09T04:35:24.263024+0000","last_clean":"2024-05-09T04:35:24.263024+0000","last_became_active":"2024-05-09T04:35:24.261701+0000","last_became_peered":"2024-05-09T04:35:24.261701+0000","last_unstale":"2024-05-09T04:35:24.263024+0000","last_undegraded":"2024-05-09T04:35:24.263024+0000","last_fullsized":"2024-05-09T04:35:24.263024+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:24:12.119694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.263475+0000","last_change":"2024-05-09T04:35:24.256667+0000","last_active":"2024-05-09T04:35:24.263475+0000","last_peered":"2024-05-09T04:35:24.263475+0000","last_clean":"2024-05-09T04:35:24.263475+0000","last_became_active":"2024-05-09T04:35:24.256403+0000","last_became_peered":"2024-05-09T04:35:24.256403+0000","last_unstale":"2024-05-09T04:35:24.263475+0000","last_undegraded":"2024-05-09T04:35:24.263475+0000","last_fullsized":"2024-05-09T04:35:24.263475+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:08:43.697014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.260132+0000","last_change":"2024-05-09T04:35:24.254911+0000","last_active":"2024-05-09T04:35:24.260132+0000","last_peered":"2024-05-09T04:35:24.260132+0000","last_clean":"2024-05-09T04:35:24.260132+0000","last_became_active":"2024-05-09T04:35:24.254599+0000","last_became_peered":"2024-05-09T04:35:24.254599+0000","last_unstale":"2024-05-09T04:35:24.260132+0000","last_undegraded":"2024-05-09T04:35:24.260132+0000","last_fullsized":"2024-05-09T04:35:24.260132+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:22:35.619585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.255360+0000","last_change":"2024-05-09T04:35:24.244616+0000","last_active":"2024-05-09T04:35:24.255360+0000","last_peered":"2024-05-09T04:35:24.255360+0000","last_clean":"2024-05-09T04:35:24.255360+0000","last_became_active":"2024-05-09T04:35:24.244333+0000","last_became_peered":"2024-05-09T04:35:24.244333+0000","last_unstale":"2024-05-09T04:35:24.255360+0000","last_undegraded":"2024-05-09T04:35:24.255360+0000","last_fullsized":"2024-05-09T04:35:24.255360+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:50:40.529895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.263548+0000","last_change":"2024-05-09T04:35:24.260896+0000","last_active":"2024-05-09T04:35:24.263548+0000","last_peered":"2024-05-09T04:35:24.263548+0000","last_clean":"2024-05-09T04:35:24.263548+0000","last_became_active":"2024-05-09T04:35:24.260576+0000","last_became_peered":"2024-05-09T04:35:24.260576+0000","last_unstale":"2024-05-09T04:35:24.263548+0000","last_undegraded":"2024-05-09T04:35:24.263548+0000","last_fullsized":"2024-05-09T04:35:24.263548+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:47:36.113715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.268328+0000","last_change":"2024-05-09T04:35:24.263898+0000","last_active":"2024-05-09T04:35:24.268328+0000","last_peered":"2024-05-09T04:35:24.268328+0000","last_clean":"2024-05-09T04:35:24.268328+0000","last_became_active":"2024-05-09T04:35:24.263595+0000","last_became_peered":"2024-05-09T04:35:24.263595+0000","last_unstale":"2024-05-09T04:35:24.268328+0000","last_undegraded":"2024-05-09T04:35:24.268328+0000","last_fullsized":"2024-05-09T04:35:24.268328+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:51:11.277081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"46'4","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.257864+0000","last_change":"2024-05-09T04:35:23.232422+0000","last_active":"2024-05-09T04:35:24.257864+0000","last_peered":"2024-05-09T04:35:24.257864+0000","last_clean":"2024-05-09T04:35:24.257864+0000","last_became_active":"2024-05-09T04:35:23.231940+0000","last_became_peered":"2024-05-09T04:35:23.231940+0000","last_unstale":"2024-05-09T04:35:24.257864+0000","last_undegraded":"2024-05-09T04:35:24.257864+0000","last_fullsized":"2024-05-09T04:35:24.257864+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:13:43.609611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.254868+0000","last_change":"2024-05-09T04:35:24.250337+0000","last_active":"2024-05-09T04:35:24.254868+0000","last_peered":"2024-05-09T04:35:24.254868+0000","last_clean":"2024-05-09T04:35:24.254868+0000","last_became_active":"2024-05-09T04:35:24.250066+0000","last_became_peered":"2024-05-09T04:35:24.250066+0000","last_unstale":"2024-05-09T04:35:24.254868+0000","last_undegraded":"2024-05-09T04:35:24.254868+0000","last_fullsized":"2024-05-09T04:35:24.254868+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:20:57.932878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"46'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.251125+0000","last_change":"2024-05-09T04:35:23.224183+0000","last_active":"2024-05-09T04:35:24.251125+0000","last_peered":"2024-05-09T04:35:24.251125+0000","last_clean":"2024-05-09T04:35:24.251125+0000","last_became_active":"2024-05-09T04:35:23.223901+0000","last_became_peered":"2024-05-09T04:35:23.223901+0000","last_unstale":"2024-05-09T04:35:24.251125+0000","last_undegraded":"2024-05-09T04:35:24.251125+0000","last_fullsized":"2024-05-09T04:35:24.251125+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:45:18.026556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.262281+0000","last_change":"2024-05-09T04:35:24.256033+0000","last_active":"2024-05-09T04:35:24.262281+0000","last_peered":"2024-05-09T04:35:24.262281+0000","last_clean":"2024-05-09T04:35:24.262281+0000","last_became_active":"2024-05-09T04:35:24.255706+0000","last_became_peered":"2024-05-09T04:35:24.255706+0000","last_unstale":"2024-05-09T04:35:24.262281+0000","last_undegraded":"2024-05-09T04:35:24.262281+0000","last_fullsized":"2024-05-09T04:35:24.262281+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:28:45.183673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.258464+0000","last_change":"2024-05-09T04:35:23.230924+0000","last_active":"2024-05-09T04:35:24.258464+0000","last_peered":"2024-05-09T04:35:24.258464+0000","last_clean":"2024-05-09T04:35:24.258464+0000","last_became_active":"2024-05-09T04:35:23.230679+0000","last_became_peered":"2024-05-09T04:35:23.230679+0000","last_unstale":"2024-05-09T04:35:24.258464+0000","last_undegraded":"2024-05-09T04:35:24.258464+0000","last_fullsized":"2024-05-09T04:35:24.258464+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:44.211487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.260485+0000","last_change":"2024-05-09T04:35:24.252740+0000","last_active":"2024-05-09T04:35:24.260485+0000","last_peered":"2024-05-09T04:35:24.260485+0000","last_clean":"2024-05-09T04:35:24.260485+0000","last_became_active":"2024-05-09T04:35:24.252258+0000","last_became_peered":"2024-05-09T04:35:24.252258+0000","last_unstale":"2024-05-09T04:35:24.260485+0000","last_undegraded":"2024-05-09T04:35:24.260485+0000","last_fullsized":"2024-05-09T04:35:24.260485+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:18:04.045651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.255620+0000","last_change":"2024-05-09T04:35:23.223959+0000","last_active":"2024-05-09T04:35:24.255620+0000","last_peered":"2024-05-09T04:35:24.255620+0000","last_clean":"2024-05-09T04:35:24.255620+0000","last_became_active":"2024-05-09T04:35:23.223672+0000","last_became_peered":"2024-05-09T04:35:23.223672+0000","last_unstale":"2024-05-09T04:35:24.255620+0000","last_undegraded":"2024-05-09T04:35:24.255620+0000","last_fullsized":"2024-05-09T04:35:24.255620+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:48:00.689997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.253334+0000","last_change":"2024-05-09T04:35:24.249572+0000","last_active":"2024-05-09T04:35:24.253334+0000","last_peered":"2024-05-09T04:35:24.253334+0000","last_clean":"2024-05-09T04:35:24.253334+0000","last_became_active":"2024-05-09T04:35:24.249301+0000","last_became_peered":"2024-05-09T04:35:24.249301+0000","last_unstale":"2024-05-09T04:35:24.253334+0000","last_undegraded":"2024-05-09T04:35:24.253334+0000","last_fullsized":"2024-05-09T04:35:24.253334+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:45.225417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"46'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.250819+0000","last_change":"2024-05-09T04:35:23.215932+0000","last_active":"2024-05-09T04:35:24.250819+0000","last_peered":"2024-05-09T04:35:24.250819+0000","last_clean":"2024-05-09T04:35:24.250819+0000","last_became_active":"2024-05-09T04:35:23.215635+0000","last_became_peered":"2024-05-09T04:35:23.215635+0000","last_unstale":"2024-05-09T04:35:24.250819+0000","last_undegraded":"2024-05-09T04:35:24.250819+0000","last_fullsized":"2024-05-09T04:35:24.250819+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:11:53.567574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.263408+0000","last_change":"2024-05-09T04:35:24.259303+0000","last_active":"2024-05-09T04:35:24.263408+0000","last_peered":"2024-05-09T04:35:24.263408+0000","last_clean":"2024-05-09T04:35:24.263408+0000","last_became_active":"2024-05-09T04:35:24.259044+0000","last_became_peered":"2024-05-09T04:35:24.259044+0000","last_unstale":"2024-05-09T04:35:24.263408+0000","last_undegraded":"2024-05-09T04:35:24.263408+0000","last_fullsized":"2024-05-09T04:35:24.263408+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:47:00.435558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"46'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.257747+0000","last_change":"2024-05-09T04:35:23.232488+0000","last_active":"2024-05-09T04:35:24.257747+0000","last_peered":"2024-05-09T04:35:24.257747+0000","last_clean":"2024-05-09T04:35:24.257747+0000","last_became_active":"2024-05-09T04:35:23.231585+0000","last_became_peered":"2024-05-09T04:35:23.231585+0000","last_unstale":"2024-05-09T04:35:24.257747+0000","last_undegraded":"2024-05-09T04:35:24.257747+0000","last_fullsized":"2024-05-09T04:35:24.257747+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:57:42.352879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.268435+0000","last_change":"2024-05-09T04:35:24.259706+0000","last_active":"2024-05-09T04:35:24.268435+0000","last_peered":"2024-05-09T04:35:24.268435+0000","last_clean":"2024-05-09T04:35:24.268435+0000","last_became_active":"2024-05-09T04:35:24.254043+0000","last_became_peered":"2024-05-09T04:35:24.254043+0000","last_unstale":"2024-05-09T04:35:24.268435+0000","last_undegraded":"2024-05-09T04:35:24.268435+0000","last_fullsized":"2024-05-09T04:35:24.268435+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:36:44.724956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.246917+0000","last_change":"2024-05-09T04:35:23.230199+0000","last_active":"2024-05-09T04:35:24.246917+0000","last_peered":"2024-05-09T04:35:24.246917+0000","last_clean":"2024-05-09T04:35:24.246917+0000","last_became_active":"2024-05-09T04:35:23.229830+0000","last_became_peered":"2024-05-09T04:35:23.229830+0000","last_unstale":"2024-05-09T04:35:24.246917+0000","last_undegraded":"2024-05-09T04:35:24.246917+0000","last_fullsized":"2024-05-09T04:35:24.246917+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:40:59.071549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.265617+0000","last_change":"2024-05-09T04:35:24.262731+0000","last_active":"2024-05-09T04:35:24.265617+0000","last_peered":"2024-05-09T04:35:24.265617+0000","last_clean":"2024-05-09T04:35:24.265617+0000","last_became_active":"2024-05-09T04:35:24.262401+0000","last_became_peered":"2024-05-09T04:35:24.262401+0000","last_unstale":"2024-05-09T04:35:24.265617+0000","last_undegraded":"2024-05-09T04:35:24.265617+0000","last_fullsized":"2024-05-09T04:35:24.265617+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:56:30.472788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.230656+0000","last_change":"2024-05-09T04:35:23.228702+0000","last_active":"2024-05-09T04:35:24.230656+0000","last_peered":"2024-05-09T04:35:24.230656+0000","last_clean":"2024-05-09T04:35:24.230656+0000","last_became_active":"2024-05-09T04:35:23.228026+0000","last_became_peered":"2024-05-09T04:35:23.228026+0000","last_unstale":"2024-05-09T04:35:24.230656+0000","last_undegraded":"2024-05-09T04:35:24.230656+0000","last_fullsized":"2024-05-09T04:35:24.230656+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:45:18.026556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.265896+0000","last_change":"2024-05-09T04:35:24.261499+0000","last_active":"2024-05-09T04:35:24.265896+0000","last_peered":"2024-05-09T04:35:24.265896+0000","last_clean":"2024-05-09T04:35:24.265896+0000","last_became_active":"2024-05-09T04:35:24.260127+0000","last_became_peered":"2024-05-09T04:35:24.260127+0000","last_unstale":"2024-05-09T04:35:24.265896+0000","last_undegraded":"2024-05-09T04:35:24.265896+0000","last_fullsized":"2024-05-09T04:35:24.265896+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:38:58.731420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.248850+0000","last_change":"2024-05-09T04:35:23.230669+0000","last_active":"2024-05-09T04:35:24.248850+0000","last_peered":"2024-05-09T04:35:24.248850+0000","last_clean":"2024-05-09T04:35:24.248850+0000","last_became_active":"2024-05-09T04:35:23.230146+0000","last_became_peered":"2024-05-09T04:35:23.230146+0000","last_unstale":"2024-05-09T04:35:24.248850+0000","last_undegraded":"2024-05-09T04:35:24.248850+0000","last_fullsized":"2024-05-09T04:35:24.248850+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:47:35.099785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.252405+0000","last_change":"2024-05-09T04:35:24.245039+0000","last_active":"2024-05-09T04:35:24.252405+0000","last_peered":"2024-05-09T04:35:24.252405+0000","last_clean":"2024-05-09T04:35:24.252405+0000","last_became_active":"2024-05-09T04:35:24.241594+0000","last_became_peered":"2024-05-09T04:35:24.241594+0000","last_unstale":"2024-05-09T04:35:24.252405+0000","last_undegraded":"2024-05-09T04:35:24.252405+0000","last_fullsized":"2024-05-09T04:35:24.252405+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:06:11.802530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.239272+0000","last_change":"2024-05-09T04:35:23.208131+0000","last_active":"2024-05-09T04:35:24.239272+0000","last_peered":"2024-05-09T04:35:24.239272+0000","last_clean":"2024-05-09T04:35:24.239272+0000","last_became_active":"2024-05-09T04:35:23.207166+0000","last_became_peered":"2024-05-09T04:35:23.207166+0000","last_unstale":"2024-05-09T04:35:24.239272+0000","last_undegraded":"2024-05-09T04:35:24.239272+0000","last_fullsized":"2024-05-09T04:35:24.239272+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:02.848544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.263494+0000","last_change":"2024-05-09T04:35:24.258823+0000","last_active":"2024-05-09T04:35:24.263494+0000","last_peered":"2024-05-09T04:35:24.263494+0000","last_clean":"2024-05-09T04:35:24.263494+0000","last_became_active":"2024-05-09T04:35:24.258458+0000","last_became_peered":"2024-05-09T04:35:24.258458+0000","last_unstale":"2024-05-09T04:35:24.263494+0000","last_undegraded":"2024-05-09T04:35:24.263494+0000","last_fullsized":"2024-05-09T04:35:24.263494+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:43:22.117586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"46'3","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.260347+0000","last_change":"2024-05-09T04:35:23.231022+0000","last_active":"2024-05-09T04:35:24.260347+0000","last_peered":"2024-05-09T04:35:24.260347+0000","last_clean":"2024-05-09T04:35:24.260347+0000","last_became_active":"2024-05-09T04:35:23.230720+0000","last_became_peered":"2024-05-09T04:35:23.230720+0000","last_unstale":"2024-05-09T04:35:24.260347+0000","last_undegraded":"2024-05-09T04:35:24.260347+0000","last_fullsized":"2024-05-09T04:35:24.260347+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:43:21.103656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.246040+0000","last_change":"2024-05-09T04:35:24.241846+0000","last_active":"2024-05-09T04:35:24.246040+0000","last_peered":"2024-05-09T04:35:24.246040+0000","last_clean":"2024-05-09T04:35:24.246040+0000","last_became_active":"2024-05-09T04:35:24.240937+0000","last_became_peered":"2024-05-09T04:35:24.240937+0000","last_unstale":"2024-05-09T04:35:24.246040+0000","last_undegraded":"2024-05-09T04:35:24.246040+0000","last_fullsized":"2024-05-09T04:35:24.246040+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:11:54.581504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"46'6","reported_seq":18,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.268716+0000","last_change":"2024-05-09T04:35:23.220710+0000","last_active":"2024-05-09T04:35:24.268716+0000","last_peered":"2024-05-09T04:35:24.268716+0000","last_clean":"2024-05-09T04:35:24.268716+0000","last_became_active":"2024-05-09T04:35:23.219368+0000","last_became_peered":"2024-05-09T04:35:23.219368+0000","last_unstale":"2024-05-09T04:35:24.268716+0000","last_undegraded":"2024-05-09T04:35:24.268716+0000","last_fullsized":"2024-05-09T04:35:24.268716+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:56:29.458858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.261263+0000","last_change":"2024-05-09T04:35:24.255664+0000","last_active":"2024-05-09T04:35:24.261263+0000","last_peered":"2024-05-09T04:35:24.261263+0000","last_clean":"2024-05-09T04:35:24.261263+0000","last_became_active":"2024-05-09T04:35:24.255068+0000","last_became_peered":"2024-05-09T04:35:24.255068+0000","last_unstale":"2024-05-09T04:35:24.261263+0000","last_undegraded":"2024-05-09T04:35:24.261263+0000","last_fullsized":"2024-05-09T04:35:24.261263+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:08:43.697014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.259256+0000","last_change":"2024-05-09T04:35:23.226565+0000","last_active":"2024-05-09T04:35:24.259256+0000","last_peered":"2024-05-09T04:35:24.259256+0000","last_clean":"2024-05-09T04:35:24.259256+0000","last_became_active":"2024-05-09T04:35:23.226303+0000","last_became_peered":"2024-05-09T04:35:23.226303+0000","last_unstale":"2024-05-09T04:35:24.259256+0000","last_undegraded":"2024-05-09T04:35:24.259256+0000","last_fullsized":"2024-05-09T04:35:24.259256+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:44.211487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.243236+0000","last_change":"2024-05-09T04:35:24.236331+0000","last_active":"2024-05-09T04:35:24.243236+0000","last_peered":"2024-05-09T04:35:24.243236+0000","last_clean":"2024-05-09T04:35:24.243236+0000","last_became_active":"2024-05-09T04:35:24.236058+0000","last_became_peered":"2024-05-09T04:35:24.236058+0000","last_unstale":"2024-05-09T04:35:24.243236+0000","last_undegraded":"2024-05-09T04:35:24.243236+0000","last_fullsized":"2024-05-09T04:35:24.243236+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:38:58.731420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.237899+0000","last_change":"2024-05-09T04:35:23.229353+0000","last_active":"2024-05-09T04:35:24.237899+0000","last_peered":"2024-05-09T04:35:24.237899+0000","last_clean":"2024-05-09T04:35:24.237899+0000","last_became_active":"2024-05-09T04:35:23.228801+0000","last_became_peered":"2024-05-09T04:35:23.228801+0000","last_unstale":"2024-05-09T04:35:24.237899+0000","last_undegraded":"2024-05-09T04:35:24.237899+0000","last_fullsized":"2024-05-09T04:35:24.237899+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:14:35.671258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.268289+0000","last_change":"2024-05-09T04:35:24.259077+0000","last_active":"2024-05-09T04:35:24.268289+0000","last_peered":"2024-05-09T04:35:24.268289+0000","last_clean":"2024-05-09T04:35:24.268289+0000","last_became_active":"2024-05-09T04:35:24.258266+0000","last_became_peered":"2024-05-09T04:35:24.258266+0000","last_unstale":"2024-05-09T04:35:24.268289+0000","last_undegraded":"2024-05-09T04:35:24.268289+0000","last_fullsized":"2024-05-09T04:35:24.268289+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:07:02.781092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.245780+0000","last_change":"2024-05-09T04:35:23.233816+0000","last_active":"2024-05-09T04:35:24.245780+0000","last_peered":"2024-05-09T04:35:24.245780+0000","last_clean":"2024-05-09T04:35:24.245780+0000","last_became_active":"2024-05-09T04:35:23.233096+0000","last_became_peered":"2024-05-09T04:35:23.233096+0000","last_unstale":"2024-05-09T04:35:24.245780+0000","last_undegraded":"2024-05-09T04:35:24.245780+0000","last_fullsized":"2024-05-09T04:35:24.245780+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:08:42.683084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.261362+0000","last_change":"2024-05-09T04:35:24.248388+0000","last_active":"2024-05-09T04:35:24.261362+0000","last_peered":"2024-05-09T04:35:24.261362+0000","last_clean":"2024-05-09T04:35:24.261362+0000","last_became_active":"2024-05-09T04:35:24.248132+0000","last_became_peered":"2024-05-09T04:35:24.248132+0000","last_unstale":"2024-05-09T04:35:24.261362+0000","last_undegraded":"2024-05-09T04:35:24.261362+0000","last_fullsized":"2024-05-09T04:35:24.261362+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:06:11.802530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"46'8","reported_seq":20,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.269539+0000","last_change":"2024-05-09T04:35:23.215879+0000","last_active":"2024-05-09T04:35:24.269539+0000","last_peered":"2024-05-09T04:35:24.269539+0000","last_clean":"2024-05-09T04:35:24.269539+0000","last_became_active":"2024-05-09T04:35:23.214959+0000","last_became_peered":"2024-05-09T04:35:23.214959+0000","last_unstale":"2024-05-09T04:35:24.269539+0000","last_undegraded":"2024-05-09T04:35:24.269539+0000","last_fullsized":"2024-05-09T04:35:24.269539+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:19:26.552697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.268522+0000","last_change":"2024-05-09T04:35:24.263955+0000","last_active":"2024-05-09T04:35:24.268522+0000","last_peered":"2024-05-09T04:35:24.268522+0000","last_clean":"2024-05-09T04:35:24.268522+0000","last_became_active":"2024-05-09T04:35:24.262715+0000","last_became_peered":"2024-05-09T04:35:24.262715+0000","last_unstale":"2024-05-09T04:35:24.268522+0000","last_undegraded":"2024-05-09T04:35:24.268522+0000","last_fullsized":"2024-05-09T04:35:24.268522+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:43:34.179388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"46'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.264475+0000","last_change":"2024-05-09T04:35:23.218612+0000","last_active":"2024-05-09T04:35:24.264475+0000","last_peered":"2024-05-09T04:35:24.264475+0000","last_clean":"2024-05-09T04:35:24.264475+0000","last_became_active":"2024-05-09T04:35:23.218258+0000","last_became_peered":"2024-05-09T04:35:23.218258+0000","last_unstale":"2024-05-09T04:35:24.264475+0000","last_undegraded":"2024-05-09T04:35:24.264475+0000","last_fullsized":"2024-05-09T04:35:24.264475+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:05:43.481549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.244673+0000","last_change":"2024-05-09T04:35:24.242891+0000","last_active":"2024-05-09T04:35:24.244673+0000","last_peered":"2024-05-09T04:35:24.244673+0000","last_clean":"2024-05-09T04:35:24.244673+0000","last_became_active":"2024-05-09T04:35:24.241067+0000","last_became_peered":"2024-05-09T04:35:24.241067+0000","last_unstale":"2024-05-09T04:35:24.244673+0000","last_undegraded":"2024-05-09T04:35:24.244673+0000","last_fullsized":"2024-05-09T04:35:24.244673+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:11:24.172582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.248889+0000","last_change":"2024-05-09T04:35:23.223741+0000","last_active":"2024-05-09T04:35:24.248889+0000","last_peered":"2024-05-09T04:35:24.248889+0000","last_clean":"2024-05-09T04:35:24.248889+0000","last_became_active":"2024-05-09T04:35:23.223426+0000","last_became_peered":"2024-05-09T04:35:23.223426+0000","last_unstale":"2024-05-09T04:35:24.248889+0000","last_undegraded":"2024-05-09T04:35:24.248889+0000","last_fullsized":"2024-05-09T04:35:24.248889+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:40:30.949891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.245708+0000","last_change":"2024-05-09T04:35:24.240282+0000","last_active":"2024-05-09T04:35:24.245708+0000","last_peered":"2024-05-09T04:35:24.245708+0000","last_clean":"2024-05-09T04:35:24.245708+0000","last_became_active":"2024-05-09T04:35:24.239867+0000","last_became_peered":"2024-05-09T04:35:24.239867+0000","last_unstale":"2024-05-09T04:35:24.245708+0000","last_undegraded":"2024-05-09T04:35:24.245708+0000","last_fullsized":"2024-05-09T04:35:24.245708+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:45:19.040487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"46'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.256441+0000","last_change":"2024-05-09T04:35:23.233478+0000","last_active":"2024-05-09T04:35:24.256441+0000","last_peered":"2024-05-09T04:35:24.256441+0000","last_clean":"2024-05-09T04:35:24.256441+0000","last_became_active":"2024-05-09T04:35:23.233123+0000","last_became_peered":"2024-05-09T04:35:23.233123+0000","last_unstale":"2024-05-09T04:35:24.256441+0000","last_undegraded":"2024-05-09T04:35:24.256441+0000","last_fullsized":"2024-05-09T04:35:24.256441+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:50:54.527868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.258590+0000","last_change":"2024-05-09T04:35:24.252157+0000","last_active":"2024-05-09T04:35:24.258590+0000","last_peered":"2024-05-09T04:35:24.258590+0000","last_clean":"2024-05-09T04:35:24.258590+0000","last_became_active":"2024-05-09T04:35:24.251826+0000","last_became_peered":"2024-05-09T04:35:24.251826+0000","last_unstale":"2024-05-09T04:35:24.258590+0000","last_undegraded":"2024-05-09T04:35:24.258590+0000","last_fullsized":"2024-05-09T04:35:24.258590+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:15:49.152245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.249106+0000","last_change":"2024-05-09T04:35:23.230027+0000","last_active":"2024-05-09T04:35:24.249106+0000","last_peered":"2024-05-09T04:35:24.249106+0000","last_clean":"2024-05-09T04:35:24.249106+0000","last_became_active":"2024-05-09T04:35:23.227428+0000","last_became_peered":"2024-05-09T04:35:23.227428+0000","last_unstale":"2024-05-09T04:35:24.249106+0000","last_undegraded":"2024-05-09T04:35:24.249106+0000","last_fullsized":"2024-05-09T04:35:24.249106+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:32:14.116321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.255534+0000","last_change":"2024-05-09T04:35:24.249839+0000","last_active":"2024-05-09T04:35:24.255534+0000","last_peered":"2024-05-09T04:35:24.255534+0000","last_clean":"2024-05-09T04:35:24.255534+0000","last_became_active":"2024-05-09T04:35:24.249526+0000","last_became_peered":"2024-05-09T04:35:24.249526+0000","last_unstale":"2024-05-09T04:35:24.255534+0000","last_undegraded":"2024-05-09T04:35:24.255534+0000","last_fullsized":"2024-05-09T04:35:24.255534+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:48:53.260192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.251685+0000","last_change":"2024-05-09T04:35:23.228101+0000","last_active":"2024-05-09T04:35:24.251685+0000","last_peered":"2024-05-09T04:35:24.251685+0000","last_clean":"2024-05-09T04:35:24.251685+0000","last_became_active":"2024-05-09T04:35:23.227279+0000","last_became_peered":"2024-05-09T04:35:23.227279+0000","last_unstale":"2024-05-09T04:35:24.251685+0000","last_undegraded":"2024-05-09T04:35:24.251685+0000","last_fullsized":"2024-05-09T04:35:24.251685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:11:53.567574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.262210+0000","last_change":"2024-05-09T04:35:24.256874+0000","last_active":"2024-05-09T04:35:24.262210+0000","last_peered":"2024-05-09T04:35:24.262210+0000","last_clean":"2024-05-09T04:35:24.262210+0000","last_became_active":"2024-05-09T04:35:24.256492+0000","last_became_peered":"2024-05-09T04:35:24.256492+0000","last_unstale":"2024-05-09T04:35:24.262210+0000","last_undegraded":"2024-05-09T04:35:24.262210+0000","last_fullsized":"2024-05-09T04:35:24.262210+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:45:10.725727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.227046+0000","last_change":"2024-05-09T04:35:23.216804+0000","last_active":"2024-05-09T04:35:24.227046+0000","last_peered":"2024-05-09T04:35:24.227046+0000","last_clean":"2024-05-09T04:35:24.227046+0000","last_became_active":"2024-05-09T04:35:23.216518+0000","last_became_peered":"2024-05-09T04:35:23.216518+0000","last_unstale":"2024-05-09T04:35:24.227046+0000","last_undegraded":"2024-05-09T04:35:24.227046+0000","last_fullsized":"2024-05-09T04:35:24.227046+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:03:07.458395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.263814+0000","last_change":"2024-05-09T04:35:24.260012+0000","last_active":"2024-05-09T04:35:24.263814+0000","last_peered":"2024-05-09T04:35:24.263814+0000","last_clean":"2024-05-09T04:35:24.263814+0000","last_became_active":"2024-05-09T04:35:24.259675+0000","last_became_peered":"2024-05-09T04:35:24.259675+0000","last_unstale":"2024-05-09T04:35:24.263814+0000","last_undegraded":"2024-05-09T04:35:24.263814+0000","last_fullsized":"2024-05-09T04:35:24.263814+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:41:23.533288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.228776+0000","last_change":"2024-05-09T04:35:23.223222+0000","last_active":"2024-05-09T04:35:24.228776+0000","last_peered":"2024-05-09T04:35:24.228776+0000","last_clean":"2024-05-09T04:35:24.228776+0000","last_became_active":"2024-05-09T04:35:23.222919+0000","last_became_peered":"2024-05-09T04:35:23.222919+0000","last_unstale":"2024-05-09T04:35:24.228776+0000","last_undegraded":"2024-05-09T04:35:24.228776+0000","last_fullsized":"2024-05-09T04:35:24.228776+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:02.848544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.260815+0000","last_change":"2024-05-09T04:35:24.255432+0000","last_active":"2024-05-09T04:35:24.260815+0000","last_peered":"2024-05-09T04:35:24.260815+0000","last_clean":"2024-05-09T04:35:24.260815+0000","last_became_active":"2024-05-09T04:35:24.255142+0000","last_became_peered":"2024-05-09T04:35:24.255142+0000","last_unstale":"2024-05-09T04:35:24.260815+0000","last_undegraded":"2024-05-09T04:35:24.260815+0000","last_fullsized":"2024-05-09T04:35:24.260815+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:20:57.932878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.219637+0000","last_change":"2024-05-09T04:35:23.228776+0000","last_active":"2024-05-09T04:35:24.219637+0000","last_peered":"2024-05-09T04:35:24.219637+0000","last_clean":"2024-05-09T04:35:24.219637+0000","last_became_active":"2024-05-09T04:35:23.227388+0000","last_became_peered":"2024-05-09T04:35:23.227388+0000","last_unstale":"2024-05-09T04:35:24.219637+0000","last_undegraded":"2024-05-09T04:35:24.219637+0000","last_fullsized":"2024-05-09T04:35:24.219637+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:36:13.725524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.263526+0000","last_change":"2024-05-09T04:35:24.259870+0000","last_active":"2024-05-09T04:35:24.263526+0000","last_peered":"2024-05-09T04:35:24.263526+0000","last_clean":"2024-05-09T04:35:24.263526+0000","last_became_active":"2024-05-09T04:35:24.259548+0000","last_became_peered":"2024-05-09T04:35:24.259548+0000","last_unstale":"2024-05-09T04:35:24.263526+0000","last_undegraded":"2024-05-09T04:35:24.263526+0000","last_fullsized":"2024-05-09T04:35:24.263526+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:43:22.117586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.261595+0000","last_change":"2024-05-09T04:35:23.235093+0000","last_active":"2024-05-09T04:35:24.261595+0000","last_peered":"2024-05-09T04:35:24.261595+0000","last_clean":"2024-05-09T04:35:24.261595+0000","last_became_active":"2024-05-09T04:35:23.234833+0000","last_became_peered":"2024-05-09T04:35:23.234833+0000","last_unstale":"2024-05-09T04:35:24.261595+0000","last_undegraded":"2024-05-09T04:35:24.261595+0000","last_fullsized":"2024-05-09T04:35:24.261595+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:08:02.005553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.243514+0000","last_change":"2024-05-09T04:35:24.238100+0000","last_active":"2024-05-09T04:35:24.243514+0000","last_peered":"2024-05-09T04:35:24.243514+0000","last_clean":"2024-05-09T04:35:24.243514+0000","last_became_active":"2024-05-09T04:35:24.237649+0000","last_became_peered":"2024-05-09T04:35:24.237649+0000","last_unstale":"2024-05-09T04:35:24.243514+0000","last_undegraded":"2024-05-09T04:35:24.243514+0000","last_fullsized":"2024-05-09T04:35:24.243514+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:17:18.226996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.249024+0000","last_change":"2024-05-09T04:35:23.229778+0000","last_active":"2024-05-09T04:35:24.249024+0000","last_peered":"2024-05-09T04:35:24.249024+0000","last_clean":"2024-05-09T04:35:24.249024+0000","last_became_active":"2024-05-09T04:35:23.229538+0000","last_became_peered":"2024-05-09T04:35:23.229538+0000","last_unstale":"2024-05-09T04:35:24.249024+0000","last_undegraded":"2024-05-09T04:35:24.249024+0000","last_fullsized":"2024-05-09T04:35:24.249024+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:41:22.519358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.260463+0000","last_change":"2024-05-09T04:35:24.252509+0000","last_active":"2024-05-09T04:35:24.260463+0000","last_peered":"2024-05-09T04:35:24.260463+0000","last_clean":"2024-05-09T04:35:24.260463+0000","last_became_active":"2024-05-09T04:35:24.251843+0000","last_became_peered":"2024-05-09T04:35:24.251843+0000","last_unstale":"2024-05-09T04:35:24.260463+0000","last_undegraded":"2024-05-09T04:35:24.260463+0000","last_fullsized":"2024-05-09T04:35:24.260463+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:44:42.597472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.228498+0000","last_change":"2024-05-09T04:35:23.219904+0000","last_active":"2024-05-09T04:35:24.228498+0000","last_peered":"2024-05-09T04:35:24.228498+0000","last_clean":"2024-05-09T04:35:24.228498+0000","last_became_active":"2024-05-09T04:35:23.219605+0000","last_became_peered":"2024-05-09T04:35:23.219605+0000","last_unstale":"2024-05-09T04:35:24.228498+0000","last_undegraded":"2024-05-09T04:35:24.228498+0000","last_fullsized":"2024-05-09T04:35:24.228498+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:57:42.352879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.263517+0000","last_change":"2024-05-09T04:35:24.259603+0000","last_active":"2024-05-09T04:35:24.263517+0000","last_peered":"2024-05-09T04:35:24.263517+0000","last_clean":"2024-05-09T04:35:24.263517+0000","last_became_active":"2024-05-09T04:35:24.259350+0000","last_became_peered":"2024-05-09T04:35:24.259350+0000","last_unstale":"2024-05-09T04:35:24.263517+0000","last_undegraded":"2024-05-09T04:35:24.263517+0000","last_fullsized":"2024-05-09T04:35:24.263517+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:47:36.113715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.233706+0000","last_change":"2024-05-09T04:35:23.214211+0000","last_active":"2024-05-09T04:35:24.233706+0000","last_peered":"2024-05-09T04:35:24.233706+0000","last_clean":"2024-05-09T04:35:24.233706+0000","last_became_active":"2024-05-09T04:35:23.213905+0000","last_became_peered":"2024-05-09T04:35:23.213905+0000","last_unstale":"2024-05-09T04:35:24.233706+0000","last_undegraded":"2024-05-09T04:35:24.233706+0000","last_fullsized":"2024-05-09T04:35:24.233706+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:51:10.263151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.258539+0000","last_change":"2024-05-09T04:35:24.252148+0000","last_active":"2024-05-09T04:35:24.258539+0000","last_peered":"2024-05-09T04:35:24.258539+0000","last_clean":"2024-05-09T04:35:24.258539+0000","last_became_active":"2024-05-09T04:35:24.251778+0000","last_became_peered":"2024-05-09T04:35:24.251778+0000","last_unstale":"2024-05-09T04:35:24.258539+0000","last_undegraded":"2024-05-09T04:35:24.258539+0000","last_fullsized":"2024-05-09T04:35:24.258539+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:09:14.673363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.227258+0000","last_change":"2024-05-09T04:35:23.221479+0000","last_active":"2024-05-09T04:35:24.227258+0000","last_peered":"2024-05-09T04:35:24.227258+0000","last_clean":"2024-05-09T04:35:24.227258+0000","last_became_active":"2024-05-09T04:35:23.220945+0000","last_became_peered":"2024-05-09T04:35:23.220945+0000","last_unstale":"2024-05-09T04:35:24.227258+0000","last_undegraded":"2024-05-09T04:35:24.227258+0000","last_fullsized":"2024-05-09T04:35:24.227258+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:46:59.421628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.255940+0000","last_change":"2024-05-09T04:35:24.249909+0000","last_active":"2024-05-09T04:35:24.255940+0000","last_peered":"2024-05-09T04:35:24.255940+0000","last_clean":"2024-05-09T04:35:24.255940+0000","last_became_active":"2024-05-09T04:35:24.249495+0000","last_became_peered":"2024-05-09T04:35:24.249495+0000","last_unstale":"2024-05-09T04:35:24.255940+0000","last_undegraded":"2024-05-09T04:35:24.255940+0000","last_fullsized":"2024-05-09T04:35:24.255940+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:48:53.260192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.233400+0000","last_change":"2024-05-09T04:35:23.218454+0000","last_active":"2024-05-09T04:35:24.233400+0000","last_peered":"2024-05-09T04:35:24.233400+0000","last_clean":"2024-05-09T04:35:24.233400+0000","last_became_active":"2024-05-09T04:35:23.218150+0000","last_became_peered":"2024-05-09T04:35:23.218150+0000","last_unstale":"2024-05-09T04:35:24.233400+0000","last_undegraded":"2024-05-09T04:35:24.233400+0000","last_fullsized":"2024-05-09T04:35:24.233400+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:05:43.481549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.252856+0000","last_change":"2024-05-09T04:35:24.248678+0000","last_active":"2024-05-09T04:35:24.252856+0000","last_peered":"2024-05-09T04:35:24.252856+0000","last_clean":"2024-05-09T04:35:24.252856+0000","last_became_active":"2024-05-09T04:35:24.246998+0000","last_became_peered":"2024-05-09T04:35:24.246998+0000","last_unstale":"2024-05-09T04:35:24.252856+0000","last_undegraded":"2024-05-09T04:35:24.252856+0000","last_fullsized":"2024-05-09T04:35:24.252856+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:29:15.109166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.225907+0000","last_change":"2024-05-09T04:35:23.227744+0000","last_active":"2024-05-09T04:35:24.225907+0000","last_peered":"2024-05-09T04:35:24.225907+0000","last_clean":"2024-05-09T04:35:24.225907+0000","last_became_active":"2024-05-09T04:35:23.227333+0000","last_became_peered":"2024-05-09T04:35:23.227333+0000","last_unstale":"2024-05-09T04:35:24.225907+0000","last_undegraded":"2024-05-09T04:35:24.225907+0000","last_fullsized":"2024-05-09T04:35:24.225907+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:11:23.158652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.262180+0000","last_change":"2024-05-09T04:35:24.256432+0000","last_active":"2024-05-09T04:35:24.262180+0000","last_peered":"2024-05-09T04:35:24.262180+0000","last_clean":"2024-05-09T04:35:24.262180+0000","last_became_active":"2024-05-09T04:35:24.256139+0000","last_became_peered":"2024-05-09T04:35:24.256139+0000","last_unstale":"2024-05-09T04:35:24.262180+0000","last_undegraded":"2024-05-09T04:35:24.262180+0000","last_fullsized":"2024-05-09T04:35:24.262180+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:39:34.024712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.232838+0000","last_change":"2024-05-09T04:35:23.229344+0000","last_active":"2024-05-09T04:35:24.232838+0000","last_peered":"2024-05-09T04:35:24.232838+0000","last_clean":"2024-05-09T04:35:24.232838+0000","last_became_active":"2024-05-09T04:35:23.227086+0000","last_became_peered":"2024-05-09T04:35:23.227086+0000","last_unstale":"2024-05-09T04:35:24.232838+0000","last_undegraded":"2024-05-09T04:35:24.232838+0000","last_fullsized":"2024-05-09T04:35:24.232838+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:10:09.760964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":158,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.244258+0000","last_change":"2024-05-09T04:33:41.092063+0000","last_active":"2024-05-09T04:35:24.244258+0000","last_peered":"2024-05-09T04:35:24.244258+0000","last_clean":"2024-05-09T04:35:24.244258+0000","last_became_active":"2024-05-09T04:33:41.091194+0000","last_became_peered":"2024-05-09T04:33:41.091194+0000","last_unstale":"2024-05-09T04:35:24.244258+0000","last_undegraded":"2024-05-09T04:35:24.244258+0000","last_fullsized":"2024-05-09T04:35:24.244258+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:40.045339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:40.045339+0000","last_clean_scrub_stamp":"2024-05-09T04:33:40.045339+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:04.908456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28144,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343696,"statfs":{"total":96636764160,"available":96607944704,"internally_reserved":0,"allocated":1146880,"data_stored":814861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":552960,"data_stored":225021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27560,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344280,"statfs":{"total":96636764160,"available":96608542720,"internally_reserved":0,"allocated":544768,"data_stored":224369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27564,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344276,"statfs":{"total":96636764160,"available":96608538624,"internally_reserved":0,"allocated":548864,"data_stored":224951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27544,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":532480,"data_stored":217594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27572,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344268,"statfs":{"total":96636764160,"available":96608530432,"internally_reserved":0,"allocated":557056,"data_stored":225981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31724,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340116,"statfs":{"total":96636764160,"available":96604278784,"internally_reserved":0,"allocated":552960,"data_stored":224529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32304,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339536,"statfs":{"total":96636764160,"available":96603684864,"internally_reserved":0,"allocated":1146880,"data_stored":814793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T04:35:28.667 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T04:35:28.902 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.902+0000 7fcc8dc00640 1 Processor -- start 2024-05-09T04:35:28.902 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.903+0000 7fcc8dc00640 1 -- start start 2024-05-09T04:35:28.904 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.904+0000 7fcc8dc00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc8815bae0 0x7fcc880721b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:28.904 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.904+0000 7fcc8dc00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fcc880cfe30 0x7fcc880d02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:28.904 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.904+0000 7fcc8dc00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc8815ffb0 con 0x7fcc8815bec0 2024-05-09T04:35:28.904 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.904+0000 7fcc8dc00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fcc88160840 con 0x7fcc880cfe30 2024-05-09T04:35:28.904 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.904+0000 7fcc8dc00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fcc880b4190 con 0x7fcc8815bae0 2024-05-09T04:35:28.904 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.904+0000 7fcc8d200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fcc880cfe30 0x7fcc880d02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:28.904 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.904+0000 7fcc8d200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fcc880cfe30 0x7fcc880d02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:52938/0 (socket says 172.21.15.145:52938) 2024-05-09T04:35:28.905 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.904+0000 7fcc8d200640 1 -- 172.21.15.145:0/359135100 learned_addr learned my addr 172.21.15.145:0/359135100 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:28.905 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.904+0000 7fcc7f400640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7fcc8815bec0 0x7fcc8815f8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:55196/0 (socket says 172.21.15.145:55196) 2024-05-09T04:35:28.905 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.905+0000 7fcc87e00640 1 --2- 172.21.15.145:0/359135100 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc8815bae0 0x7fcc880721b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:28.906 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.905+0000 7fcc8d200640 1 -- 172.21.15.145:0/359135100 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc8815bae0 msgr2=0x7fcc880721b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:28.906 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.905+0000 7fcc8d200640 1 --2- 172.21.15.145:0/359135100 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc8815bae0 0x7fcc880721b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:28.906 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.905+0000 7fcc8d200640 1 -- 172.21.15.145:0/359135100 >> v1:172.21.15.145:6789/0 conn(0x7fcc8815bec0 legacy=0x7fcc8815f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:28.906 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.905+0000 7fcc8d200640 1 -- 172.21.15.145:0/359135100 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc88160280 con 0x7fcc880cfe30 2024-05-09T04:35:28.906 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.905+0000 7fcc87400640 1 -- 172.21.15.145:0/359135100 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1996245225 0 0) 0x7fcc8815ffb0 con 0x7fcc8815bec0 2024-05-09T04:35:28.906 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.906+0000 7fcc8d200640 1 --2- 172.21.15.145:0/359135100 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc880cfe30 0x7fcc880d02c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fcc80009e50 tx=0x7fcc8002a670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d743e99b7a2f86bf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:28.906 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.906+0000 7fcc87400640 1 -- 172.21.15.145:0/359135100 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc80036070 con 0x7fcc880cfe30 2024-05-09T04:35:28.906 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.906+0000 7fcc87400640 1 -- 172.21.15.145:0/359135100 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc80031600 con 0x7fcc880cfe30 2024-05-09T04:35:28.906 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.906+0000 7fcc87400640 1 -- 172.21.15.145:0/359135100 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc800319d0 con 0x7fcc880cfe30 2024-05-09T04:35:28.907 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.907+0000 7fcc8dc00640 1 -- 172.21.15.145:0/359135100 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc880cfe30 msgr2=0x7fcc880d02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:28.907 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.907+0000 7fcc8dc00640 1 --2- 172.21.15.145:0/359135100 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc880cfe30 0x7fcc880d02c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fcc80009e50 tx=0x7fcc8002a670 comp rx=0 tx=0).stop 2024-05-09T04:35:28.907 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.907+0000 7fcc8dc00640 1 -- 172.21.15.145:0/359135100 shutdown_connections 2024-05-09T04:35:28.907 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.907+0000 7fcc8dc00640 1 --2- 172.21.15.145:0/359135100 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc880cfe30 0x7fcc880d02c0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:28.908 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.907+0000 7fcc8dc00640 1 --2- 172.21.15.145:0/359135100 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc8815bae0 0x7fcc880721b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:28.908 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.907+0000 7fcc8dc00640 1 -- 172.21.15.145:0/359135100 >> 172.21.15.145:0/359135100 conn(0x7fcc8806c8a0 msgr2=0x7fcc8806ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:28.908 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.907+0000 7fcc8dc00640 1 -- 172.21.15.145:0/359135100 shutdown_connections 2024-05-09T04:35:28.908 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.907+0000 7fcc8dc00640 1 -- 172.21.15.145:0/359135100 wait complete. 2024-05-09T04:35:28.908 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.909+0000 7fcc8dc00640 1 Processor -- start 2024-05-09T04:35:28.908 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.909+0000 7fcc8dc00640 1 -- start start 2024-05-09T04:35:28.911 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.910+0000 7fcc8dc00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc880cfe30 0x7fcc88157600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:28.911 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.910+0000 7fcc8dc00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc8815bae0 0x7fcc880b4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:28.911 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.910+0000 7fcc8dc00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc8815bec0 0x7fcc88142350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:28.911 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.910+0000 7fcc8dc00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fcc880726f0 con 0x7fcc8815bec0 2024-05-09T04:35:28.912 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.910+0000 7fcc8dc00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fcc880b3e20 con 0x7fcc8815bae0 2024-05-09T04:35:28.912 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.910+0000 7fcc8dc00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fcc880aded0 con 0x7fcc880cfe30 2024-05-09T04:35:28.912 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.910+0000 7fcc8d200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc8815bec0 0x7fcc88142350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:28.912 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.910+0000 7fcc8d200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc8815bec0 0x7fcc88142350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:52942/0 (socket says 172.21.15.145:52942) 2024-05-09T04:35:28.912 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.910+0000 7fcc8d200640 1 -- 172.21.15.145:0/4054749161 learned_addr learned my addr 172.21.15.145:0/4054749161 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:28.912 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.910+0000 7fcc87e00640 1 --2- 172.21.15.145:0/4054749161 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc880cfe30 0x7fcc88157600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:28.912 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.911+0000 7fcc7f400640 1 --2- 172.21.15.145:0/4054749161 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc8815bae0 0x7fcc880b4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:28.912 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.911+0000 7fcc8d200640 1 -- 172.21.15.145:0/4054749161 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc880cfe30 msgr2=0x7fcc88157600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:28.912 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.911+0000 7fcc8d200640 1 --2- 172.21.15.145:0/4054749161 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc880cfe30 0x7fcc88157600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:28.912 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.911+0000 7fcc8d200640 1 -- 172.21.15.145:0/4054749161 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc8815bae0 msgr2=0x7fcc880b4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:28.912 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.911+0000 7fcc8d200640 1 --2- 172.21.15.145:0/4054749161 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc8815bae0 0x7fcc880b4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:28.912 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.911+0000 7fcc8d200640 1 -- 172.21.15.145:0/4054749161 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc88154ab0 con 0x7fcc8815bec0 2024-05-09T04:35:28.912 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.911+0000 7fcc8d200640 1 --2- 172.21.15.145:0/4054749161 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc8815bec0 0x7fcc88142350 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fcc80034040 tx=0x7fcc80004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:28.912 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.911+0000 7fcc84c00640 1 -- 172.21.15.145:0/4054749161 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc80036050 con 0x7fcc8815bec0 2024-05-09T04:35:28.913 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.911+0000 7fcc84c00640 1 -- 172.21.15.145:0/4054749161 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc80042070 con 0x7fcc8815bec0 2024-05-09T04:35:28.913 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.911+0000 7fcc84c00640 1 -- 172.21.15.145:0/4054749161 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc80009510 con 0x7fcc8815bec0 2024-05-09T04:35:28.913 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.911+0000 7fcc8dc00640 1 -- 172.21.15.145:0/4054749161 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc881594d0 con 0x7fcc8815bec0 2024-05-09T04:35:28.913 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.911+0000 7fcc8dc00640 1 -- 172.21.15.145:0/4054749161 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc88159a90 con 0x7fcc8815bec0 2024-05-09T04:35:28.913 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.912+0000 7fcc8dc00640 1 -- 172.21.15.145:0/4054749161 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc8806a310 con 0x7fcc8815bec0 2024-05-09T04:35:28.913 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.913+0000 7fcc84c00640 1 -- 172.21.15.145:0/4054749161 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fcc80007bd0 con 0x7fcc8815bec0 2024-05-09T04:35:28.914 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.914+0000 7fcc84c00640 1 --2- 172.21.15.145:0/4054749161 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fcc640762d0 0x7fcc64078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:28.914 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.915+0000 7fcc84c00640 1 -- 172.21.15.145:0/4054749161 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7fcc800b84f0 con 0x7fcc8815bec0 2024-05-09T04:35:28.914 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.915+0000 7fcc87e00640 1 --2- 172.21.15.145:0/4054749161 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fcc640762d0 0x7fcc64078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:28.915 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.915+0000 7fcc87e00640 1 --2- 172.21.15.145:0/4054749161 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fcc640762d0 0x7fcc64078790 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fcc78006a90 tx=0x7fcc78006a20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:28.918 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:28.919+0000 7fcc84c00640 1 -- 172.21.15.145:0/4054749161 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fcc80081db0 con 0x7fcc8815bec0 2024-05-09T04:35:29.124 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.124+0000 7fcc8dc00640 1 -- 172.21.15.145:0/4054749161 --> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcc88070360 con 0x7fcc640762d0 2024-05-09T04:35:29.141 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.141+0000 7fcc84c00640 1 -- 172.21.15.145:0/4054749161 <== mgr.4104 v2:172.21.15.145:6832/2725855396 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251875 (secure 0 0 0) 0x7fcc88070360 con 0x7fcc640762d0 2024-05-09T04:35:29.145 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:35:29.146 INFO:teuthology.orchestra.run.smithi145.stderr:dumped all 2024-05-09T04:35:29.148 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.148+0000 7fcc8dc00640 1 -- 172.21.15.145:0/4054749161 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fcc640762d0 msgr2=0x7fcc64078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:29.148 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.148+0000 7fcc8dc00640 1 --2- 172.21.15.145:0/4054749161 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fcc640762d0 0x7fcc64078790 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fcc78006a90 tx=0x7fcc78006a20 comp rx=0 tx=0).stop 2024-05-09T04:35:29.148 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.148+0000 7fcc8dc00640 1 -- 172.21.15.145:0/4054749161 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc8815bec0 msgr2=0x7fcc88142350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:29.148 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.148+0000 7fcc8dc00640 1 --2- 172.21.15.145:0/4054749161 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc8815bec0 0x7fcc88142350 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fcc80034040 tx=0x7fcc80004a40 comp rx=0 tx=0).stop 2024-05-09T04:35:29.148 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.148+0000 7fcc8dc00640 1 -- 172.21.15.145:0/4054749161 shutdown_connections 2024-05-09T04:35:29.148 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.148+0000 7fcc8dc00640 1 --2- 172.21.15.145:0/4054749161 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fcc640762d0 0x7fcc64078790 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:29.149 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.148+0000 7fcc8dc00640 1 --2- 172.21.15.145:0/4054749161 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcc8815bec0 0x7fcc88142350 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:29.149 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.148+0000 7fcc8dc00640 1 --2- 172.21.15.145:0/4054749161 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc8815bae0 0x7fcc880b4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:29.149 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.148+0000 7fcc8dc00640 1 --2- 172.21.15.145:0/4054749161 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc880cfe30 0x7fcc88157600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:29.149 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.148+0000 7fcc8dc00640 1 -- 172.21.15.145:0/4054749161 >> 172.21.15.145:0/4054749161 conn(0x7fcc8806c8a0 msgr2=0x7fcc8815f5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:29.149 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.148+0000 7fcc8dc00640 1 -- 172.21.15.145:0/4054749161 shutdown_connections 2024-05-09T04:35:29.149 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.148+0000 7fcc8dc00640 1 -- 172.21.15.145:0/4054749161 wait complete. 2024-05-09T04:35:29.167 INFO:teuthology.orchestra.run.smithi145.stdout:{"pg_ready":true,"pg_map":{"version":94,"stamp":"2024-05-09T04:35:27.784210+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":229980,"kb_used_data":5452,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754744740,"statfs":{"total":773094113280,"available":772858613760,"internally_reserved":0,"allocated":5582848,"data_stored":2972099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":229756465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.598465"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.258715+0000","last_change":"2024-05-09T04:35:24.253736+0000","last_active":"2024-05-09T04:35:24.258715+0000","last_peered":"2024-05-09T04:35:24.258715+0000","last_clean":"2024-05-09T04:35:24.258715+0000","last_became_active":"2024-05-09T04:35:24.252217+0000","last_became_peered":"2024-05-09T04:35:24.252217+0000","last_unstale":"2024-05-09T04:35:24.258715+0000","last_undegraded":"2024-05-09T04:35:24.258715+0000","last_fullsized":"2024-05-09T04:35:24.258715+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:06:11.802530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.262929+0000","last_change":"2024-05-09T04:35:24.257063+0000","last_active":"2024-05-09T04:35:24.262929+0000","last_peered":"2024-05-09T04:35:24.262929+0000","last_clean":"2024-05-09T04:35:24.262929+0000","last_became_active":"2024-05-09T04:35:24.256800+0000","last_became_peered":"2024-05-09T04:35:24.256800+0000","last_unstale":"2024-05-09T04:35:24.262929+0000","last_undegraded":"2024-05-09T04:35:24.262929+0000","last_fullsized":"2024-05-09T04:35:24.262929+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:03:08.472325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.257620+0000","last_change":"2024-05-09T04:35:24.252124+0000","last_active":"2024-05-09T04:35:24.257620+0000","last_peered":"2024-05-09T04:35:24.257620+0000","last_clean":"2024-05-09T04:35:24.257620+0000","last_became_active":"2024-05-09T04:35:24.251901+0000","last_became_peered":"2024-05-09T04:35:24.251901+0000","last_unstale":"2024-05-09T04:35:24.257620+0000","last_undegraded":"2024-05-09T04:35:24.257620+0000","last_fullsized":"2024-05-09T04:35:24.257620+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:09:14.673363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.271430+0000","last_change":"2024-05-09T04:35:24.269122+0000","last_active":"2024-05-09T04:35:24.271430+0000","last_peered":"2024-05-09T04:35:24.271430+0000","last_clean":"2024-05-09T04:35:24.271430+0000","last_became_active":"2024-05-09T04:35:24.268794+0000","last_became_peered":"2024-05-09T04:35:24.268794+0000","last_unstale":"2024-05-09T04:35:24.271430+0000","last_undegraded":"2024-05-09T04:35:24.271430+0000","last_fullsized":"2024-05-09T04:35:24.271430+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:24:08.787200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.258302+0000","last_change":"2024-05-09T04:35:24.255247+0000","last_active":"2024-05-09T04:35:24.258302+0000","last_peered":"2024-05-09T04:35:24.258302+0000","last_clean":"2024-05-09T04:35:24.258302+0000","last_became_active":"2024-05-09T04:35:24.254540+0000","last_became_peered":"2024-05-09T04:35:24.254540+0000","last_unstale":"2024-05-09T04:35:24.258302+0000","last_undegraded":"2024-05-09T04:35:24.258302+0000","last_fullsized":"2024-05-09T04:35:24.258302+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:10:10.774894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.258656+0000","last_change":"2024-05-09T04:35:24.254553+0000","last_active":"2024-05-09T04:35:24.258656+0000","last_peered":"2024-05-09T04:35:24.258656+0000","last_clean":"2024-05-09T04:35:24.258656+0000","last_became_active":"2024-05-09T04:35:24.254351+0000","last_became_peered":"2024-05-09T04:35:24.254351+0000","last_unstale":"2024-05-09T04:35:24.258656+0000","last_undegraded":"2024-05-09T04:35:24.258656+0000","last_fullsized":"2024-05-09T04:35:24.258656+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:48:53.260192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.255475+0000","last_change":"2024-05-09T04:35:24.244839+0000","last_active":"2024-05-09T04:35:24.255475+0000","last_peered":"2024-05-09T04:35:24.255475+0000","last_clean":"2024-05-09T04:35:24.255475+0000","last_became_active":"2024-05-09T04:35:24.244566+0000","last_became_peered":"2024-05-09T04:35:24.244566+0000","last_unstale":"2024-05-09T04:35:24.255475+0000","last_undegraded":"2024-05-09T04:35:24.255475+0000","last_fullsized":"2024-05-09T04:35:24.255475+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:11:24.172582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.268337+0000","last_change":"2024-05-09T04:35:24.260670+0000","last_active":"2024-05-09T04:35:24.268337+0000","last_peered":"2024-05-09T04:35:24.268337+0000","last_clean":"2024-05-09T04:35:24.268337+0000","last_became_active":"2024-05-09T04:35:24.257218+0000","last_became_peered":"2024-05-09T04:35:24.257218+0000","last_unstale":"2024-05-09T04:35:24.268337+0000","last_undegraded":"2024-05-09T04:35:24.268337+0000","last_fullsized":"2024-05-09T04:35:24.268337+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:04:25.956768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.268664+0000","last_change":"2024-05-09T04:35:24.267325+0000","last_active":"2024-05-09T04:35:24.268664+0000","last_peered":"2024-05-09T04:35:24.268664+0000","last_clean":"2024-05-09T04:35:24.268664+0000","last_became_active":"2024-05-09T04:35:24.265960+0000","last_became_peered":"2024-05-09T04:35:24.265960+0000","last_unstale":"2024-05-09T04:35:24.268664+0000","last_undegraded":"2024-05-09T04:35:24.268664+0000","last_fullsized":"2024-05-09T04:35:24.268664+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:39:13.548453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.244809+0000","last_change":"2024-05-09T04:35:24.240356+0000","last_active":"2024-05-09T04:35:24.244809+0000","last_peered":"2024-05-09T04:35:24.244809+0000","last_clean":"2024-05-09T04:35:24.244809+0000","last_became_active":"2024-05-09T04:35:24.239958+0000","last_became_peered":"2024-05-09T04:35:24.239958+0000","last_unstale":"2024-05-09T04:35:24.244809+0000","last_undegraded":"2024-05-09T04:35:24.244809+0000","last_fullsized":"2024-05-09T04:35:24.244809+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:50:40.529895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.261964+0000","last_change":"2024-05-09T04:35:24.255043+0000","last_active":"2024-05-09T04:35:24.261964+0000","last_peered":"2024-05-09T04:35:24.261964+0000","last_clean":"2024-05-09T04:35:24.261964+0000","last_became_active":"2024-05-09T04:35:24.254124+0000","last_became_peered":"2024-05-09T04:35:24.254124+0000","last_unstale":"2024-05-09T04:35:24.261964+0000","last_undegraded":"2024-05-09T04:35:24.261964+0000","last_fullsized":"2024-05-09T04:35:24.261964+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:55:13.620010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.268670+0000","last_change":"2024-05-09T04:35:24.264561+0000","last_active":"2024-05-09T04:35:24.268670+0000","last_peered":"2024-05-09T04:35:24.268670+0000","last_clean":"2024-05-09T04:35:24.268670+0000","last_became_active":"2024-05-09T04:35:24.263961+0000","last_became_peered":"2024-05-09T04:35:24.263961+0000","last_unstale":"2024-05-09T04:35:24.268670+0000","last_undegraded":"2024-05-09T04:35:24.268670+0000","last_fullsized":"2024-05-09T04:35:24.268670+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:32:06.767185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.251060+0000","last_change":"2024-05-09T04:35:24.246799+0000","last_active":"2024-05-09T04:35:24.251060+0000","last_peered":"2024-05-09T04:35:24.251060+0000","last_clean":"2024-05-09T04:35:24.251060+0000","last_became_active":"2024-05-09T04:35:24.245105+0000","last_became_peered":"2024-05-09T04:35:24.245105+0000","last_unstale":"2024-05-09T04:35:24.251060+0000","last_undegraded":"2024-05-09T04:35:24.251060+0000","last_fullsized":"2024-05-09T04:35:24.251060+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:48:01.703927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.258281+0000","last_change":"2024-05-09T04:35:24.250234+0000","last_active":"2024-05-09T04:35:24.258281+0000","last_peered":"2024-05-09T04:35:24.258281+0000","last_clean":"2024-05-09T04:35:24.258281+0000","last_became_active":"2024-05-09T04:35:24.249914+0000","last_became_peered":"2024-05-09T04:35:24.249914+0000","last_unstale":"2024-05-09T04:35:24.258281+0000","last_undegraded":"2024-05-09T04:35:24.258281+0000","last_fullsized":"2024-05-09T04:35:24.258281+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:20:57.932878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.268335+0000","last_change":"2024-05-09T04:35:24.264230+0000","last_active":"2024-05-09T04:35:24.268335+0000","last_peered":"2024-05-09T04:35:24.268335+0000","last_clean":"2024-05-09T04:35:24.268335+0000","last_became_active":"2024-05-09T04:35:24.263356+0000","last_became_peered":"2024-05-09T04:35:24.263356+0000","last_unstale":"2024-05-09T04:35:24.268335+0000","last_undegraded":"2024-05-09T04:35:24.268335+0000","last_fullsized":"2024-05-09T04:35:24.268335+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:29:15.109166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.243385+0000","last_change":"2024-05-09T04:35:24.236902+0000","last_active":"2024-05-09T04:35:24.243385+0000","last_peered":"2024-05-09T04:35:24.243385+0000","last_clean":"2024-05-09T04:35:24.243385+0000","last_became_active":"2024-05-09T04:35:24.234449+0000","last_became_peered":"2024-05-09T04:35:24.234449+0000","last_unstale":"2024-05-09T04:35:24.243385+0000","last_undegraded":"2024-05-09T04:35:24.243385+0000","last_fullsized":"2024-05-09T04:35:24.243385+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:38:58.731420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.259963+0000","last_change":"2024-05-09T04:35:24.245669+0000","last_active":"2024-05-09T04:35:24.259963+0000","last_peered":"2024-05-09T04:35:24.259963+0000","last_clean":"2024-05-09T04:35:24.259963+0000","last_became_active":"2024-05-09T04:35:24.242139+0000","last_became_peered":"2024-05-09T04:35:24.242139+0000","last_unstale":"2024-05-09T04:35:24.259963+0000","last_undegraded":"2024-05-09T04:35:24.259963+0000","last_fullsized":"2024-05-09T04:35:24.259963+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:48:01.703927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.263690+0000","last_change":"2024-05-09T04:35:24.257226+0000","last_active":"2024-05-09T04:35:24.263690+0000","last_peered":"2024-05-09T04:35:24.263690+0000","last_clean":"2024-05-09T04:35:24.263690+0000","last_became_active":"2024-05-09T04:35:24.256968+0000","last_became_peered":"2024-05-09T04:35:24.256968+0000","last_unstale":"2024-05-09T04:35:24.263690+0000","last_undegraded":"2024-05-09T04:35:24.263690+0000","last_fullsized":"2024-05-09T04:35:24.263690+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:19:27.566627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.257347+0000","last_change":"2024-05-09T04:35:24.247805+0000","last_active":"2024-05-09T04:35:24.257347+0000","last_peered":"2024-05-09T04:35:24.257347+0000","last_clean":"2024-05-09T04:35:24.257347+0000","last_became_active":"2024-05-09T04:35:24.247459+0000","last_became_peered":"2024-05-09T04:35:24.247459+0000","last_unstale":"2024-05-09T04:35:24.257347+0000","last_undegraded":"2024-05-09T04:35:24.257347+0000","last_fullsized":"2024-05-09T04:35:24.257347+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:15:49.152245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.262799+0000","last_change":"2024-05-09T04:35:24.258292+0000","last_active":"2024-05-09T04:35:24.262799+0000","last_peered":"2024-05-09T04:35:24.262799+0000","last_clean":"2024-05-09T04:35:24.262799+0000","last_became_active":"2024-05-09T04:35:24.257967+0000","last_became_peered":"2024-05-09T04:35:24.257967+0000","last_unstale":"2024-05-09T04:35:24.262799+0000","last_undegraded":"2024-05-09T04:35:24.262799+0000","last_fullsized":"2024-05-09T04:35:24.262799+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:03:01.497601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.262244+0000","last_change":"2024-05-09T04:35:24.259180+0000","last_active":"2024-05-09T04:35:24.262244+0000","last_peered":"2024-05-09T04:35:24.262244+0000","last_clean":"2024-05-09T04:35:24.262244+0000","last_became_active":"2024-05-09T04:35:24.258884+0000","last_became_peered":"2024-05-09T04:35:24.258884+0000","last_unstale":"2024-05-09T04:35:24.262244+0000","last_undegraded":"2024-05-09T04:35:24.262244+0000","last_fullsized":"2024-05-09T04:35:24.262244+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:28:54.503580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.251797+0000","last_change":"2024-05-09T04:35:24.249166+0000","last_active":"2024-05-09T04:35:24.251797+0000","last_peered":"2024-05-09T04:35:24.251797+0000","last_clean":"2024-05-09T04:35:24.251797+0000","last_became_active":"2024-05-09T04:35:24.246642+0000","last_became_peered":"2024-05-09T04:35:24.246642+0000","last_unstale":"2024-05-09T04:35:24.251797+0000","last_undegraded":"2024-05-09T04:35:24.251797+0000","last_fullsized":"2024-05-09T04:35:24.251797+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:24:08.787200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.261943+0000","last_change":"2024-05-09T04:35:24.254719+0000","last_active":"2024-05-09T04:35:24.261943+0000","last_peered":"2024-05-09T04:35:24.261943+0000","last_clean":"2024-05-09T04:35:24.261943+0000","last_became_active":"2024-05-09T04:35:24.254495+0000","last_became_peered":"2024-05-09T04:35:24.254495+0000","last_unstale":"2024-05-09T04:35:24.261943+0000","last_undegraded":"2024-05-09T04:35:24.261943+0000","last_fullsized":"2024-05-09T04:35:24.261943+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:38:26.054002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.247016+0000","last_change":"2024-05-09T04:35:24.239761+0000","last_active":"2024-05-09T04:35:24.247016+0000","last_peered":"2024-05-09T04:35:24.247016+0000","last_clean":"2024-05-09T04:35:24.247016+0000","last_became_active":"2024-05-09T04:35:24.237890+0000","last_became_peered":"2024-05-09T04:35:24.237890+0000","last_unstale":"2024-05-09T04:35:24.247016+0000","last_undegraded":"2024-05-09T04:35:24.247016+0000","last_fullsized":"2024-05-09T04:35:24.247016+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:17:18.226996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.265125+0000","last_change":"2024-05-09T04:35:24.261402+0000","last_active":"2024-05-09T04:35:24.265125+0000","last_peered":"2024-05-09T04:35:24.265125+0000","last_clean":"2024-05-09T04:35:24.265125+0000","last_became_active":"2024-05-09T04:35:24.259786+0000","last_became_peered":"2024-05-09T04:35:24.259786+0000","last_unstale":"2024-05-09T04:35:24.265125+0000","last_undegraded":"2024-05-09T04:35:24.265125+0000","last_fullsized":"2024-05-09T04:35:24.265125+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:22:35.619585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.266247+0000","last_change":"2024-05-09T04:35:24.253325+0000","last_active":"2024-05-09T04:35:24.266247+0000","last_peered":"2024-05-09T04:35:24.266247+0000","last_clean":"2024-05-09T04:35:24.266247+0000","last_became_active":"2024-05-09T04:35:24.253073+0000","last_became_peered":"2024-05-09T04:35:24.253073+0000","last_unstale":"2024-05-09T04:35:24.266247+0000","last_undegraded":"2024-05-09T04:35:24.266247+0000","last_fullsized":"2024-05-09T04:35:24.266247+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:54:30.099491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.238834+0000","last_change":"2024-05-09T04:35:24.235550+0000","last_active":"2024-05-09T04:35:24.238834+0000","last_peered":"2024-05-09T04:35:24.238834+0000","last_clean":"2024-05-09T04:35:24.238834+0000","last_became_active":"2024-05-09T04:35:24.235259+0000","last_became_peered":"2024-05-09T04:35:24.235259+0000","last_unstale":"2024-05-09T04:35:24.238834+0000","last_undegraded":"2024-05-09T04:35:24.238834+0000","last_fullsized":"2024-05-09T04:35:24.238834+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:10:10.774894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.263024+0000","last_change":"2024-05-09T04:35:24.262004+0000","last_active":"2024-05-09T04:35:24.263024+0000","last_peered":"2024-05-09T04:35:24.263024+0000","last_clean":"2024-05-09T04:35:24.263024+0000","last_became_active":"2024-05-09T04:35:24.261701+0000","last_became_peered":"2024-05-09T04:35:24.261701+0000","last_unstale":"2024-05-09T04:35:24.263024+0000","last_undegraded":"2024-05-09T04:35:24.263024+0000","last_fullsized":"2024-05-09T04:35:24.263024+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:24:12.119694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.263475+0000","last_change":"2024-05-09T04:35:24.256667+0000","last_active":"2024-05-09T04:35:24.263475+0000","last_peered":"2024-05-09T04:35:24.263475+0000","last_clean":"2024-05-09T04:35:24.263475+0000","last_became_active":"2024-05-09T04:35:24.256403+0000","last_became_peered":"2024-05-09T04:35:24.256403+0000","last_unstale":"2024-05-09T04:35:24.263475+0000","last_undegraded":"2024-05-09T04:35:24.263475+0000","last_fullsized":"2024-05-09T04:35:24.263475+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:08:43.697014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.260132+0000","last_change":"2024-05-09T04:35:24.254911+0000","last_active":"2024-05-09T04:35:24.260132+0000","last_peered":"2024-05-09T04:35:24.260132+0000","last_clean":"2024-05-09T04:35:24.260132+0000","last_became_active":"2024-05-09T04:35:24.254599+0000","last_became_peered":"2024-05-09T04:35:24.254599+0000","last_unstale":"2024-05-09T04:35:24.260132+0000","last_undegraded":"2024-05-09T04:35:24.260132+0000","last_fullsized":"2024-05-09T04:35:24.260132+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:22:35.619585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.255360+0000","last_change":"2024-05-09T04:35:24.244616+0000","last_active":"2024-05-09T04:35:24.255360+0000","last_peered":"2024-05-09T04:35:24.255360+0000","last_clean":"2024-05-09T04:35:24.255360+0000","last_became_active":"2024-05-09T04:35:24.244333+0000","last_became_peered":"2024-05-09T04:35:24.244333+0000","last_unstale":"2024-05-09T04:35:24.255360+0000","last_undegraded":"2024-05-09T04:35:24.255360+0000","last_fullsized":"2024-05-09T04:35:24.255360+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:50:40.529895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.263548+0000","last_change":"2024-05-09T04:35:24.260896+0000","last_active":"2024-05-09T04:35:24.263548+0000","last_peered":"2024-05-09T04:35:24.263548+0000","last_clean":"2024-05-09T04:35:24.263548+0000","last_became_active":"2024-05-09T04:35:24.260576+0000","last_became_peered":"2024-05-09T04:35:24.260576+0000","last_unstale":"2024-05-09T04:35:24.263548+0000","last_undegraded":"2024-05-09T04:35:24.263548+0000","last_fullsized":"2024-05-09T04:35:24.263548+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:47:36.113715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.268328+0000","last_change":"2024-05-09T04:35:24.263898+0000","last_active":"2024-05-09T04:35:24.268328+0000","last_peered":"2024-05-09T04:35:24.268328+0000","last_clean":"2024-05-09T04:35:24.268328+0000","last_became_active":"2024-05-09T04:35:24.263595+0000","last_became_peered":"2024-05-09T04:35:24.263595+0000","last_unstale":"2024-05-09T04:35:24.268328+0000","last_undegraded":"2024-05-09T04:35:24.268328+0000","last_fullsized":"2024-05-09T04:35:24.268328+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:51:11.277081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"46'4","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.257864+0000","last_change":"2024-05-09T04:35:23.232422+0000","last_active":"2024-05-09T04:35:24.257864+0000","last_peered":"2024-05-09T04:35:24.257864+0000","last_clean":"2024-05-09T04:35:24.257864+0000","last_became_active":"2024-05-09T04:35:23.231940+0000","last_became_peered":"2024-05-09T04:35:23.231940+0000","last_unstale":"2024-05-09T04:35:24.257864+0000","last_undegraded":"2024-05-09T04:35:24.257864+0000","last_fullsized":"2024-05-09T04:35:24.257864+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:13:43.609611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.254868+0000","last_change":"2024-05-09T04:35:24.250337+0000","last_active":"2024-05-09T04:35:24.254868+0000","last_peered":"2024-05-09T04:35:24.254868+0000","last_clean":"2024-05-09T04:35:24.254868+0000","last_became_active":"2024-05-09T04:35:24.250066+0000","last_became_peered":"2024-05-09T04:35:24.250066+0000","last_unstale":"2024-05-09T04:35:24.254868+0000","last_undegraded":"2024-05-09T04:35:24.254868+0000","last_fullsized":"2024-05-09T04:35:24.254868+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:20:57.932878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"46'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.251125+0000","last_change":"2024-05-09T04:35:23.224183+0000","last_active":"2024-05-09T04:35:24.251125+0000","last_peered":"2024-05-09T04:35:24.251125+0000","last_clean":"2024-05-09T04:35:24.251125+0000","last_became_active":"2024-05-09T04:35:23.223901+0000","last_became_peered":"2024-05-09T04:35:23.223901+0000","last_unstale":"2024-05-09T04:35:24.251125+0000","last_undegraded":"2024-05-09T04:35:24.251125+0000","last_fullsized":"2024-05-09T04:35:24.251125+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:45:18.026556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.262281+0000","last_change":"2024-05-09T04:35:24.256033+0000","last_active":"2024-05-09T04:35:24.262281+0000","last_peered":"2024-05-09T04:35:24.262281+0000","last_clean":"2024-05-09T04:35:24.262281+0000","last_became_active":"2024-05-09T04:35:24.255706+0000","last_became_peered":"2024-05-09T04:35:24.255706+0000","last_unstale":"2024-05-09T04:35:24.262281+0000","last_undegraded":"2024-05-09T04:35:24.262281+0000","last_fullsized":"2024-05-09T04:35:24.262281+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:28:45.183673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.258464+0000","last_change":"2024-05-09T04:35:23.230924+0000","last_active":"2024-05-09T04:35:24.258464+0000","last_peered":"2024-05-09T04:35:24.258464+0000","last_clean":"2024-05-09T04:35:24.258464+0000","last_became_active":"2024-05-09T04:35:23.230679+0000","last_became_peered":"2024-05-09T04:35:23.230679+0000","last_unstale":"2024-05-09T04:35:24.258464+0000","last_undegraded":"2024-05-09T04:35:24.258464+0000","last_fullsized":"2024-05-09T04:35:24.258464+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:44.211487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.260485+0000","last_change":"2024-05-09T04:35:24.252740+0000","last_active":"2024-05-09T04:35:24.260485+0000","last_peered":"2024-05-09T04:35:24.260485+0000","last_clean":"2024-05-09T04:35:24.260485+0000","last_became_active":"2024-05-09T04:35:24.252258+0000","last_became_peered":"2024-05-09T04:35:24.252258+0000","last_unstale":"2024-05-09T04:35:24.260485+0000","last_undegraded":"2024-05-09T04:35:24.260485+0000","last_fullsized":"2024-05-09T04:35:24.260485+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:18:04.045651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.255620+0000","last_change":"2024-05-09T04:35:23.223959+0000","last_active":"2024-05-09T04:35:24.255620+0000","last_peered":"2024-05-09T04:35:24.255620+0000","last_clean":"2024-05-09T04:35:24.255620+0000","last_became_active":"2024-05-09T04:35:23.223672+0000","last_became_peered":"2024-05-09T04:35:23.223672+0000","last_unstale":"2024-05-09T04:35:24.255620+0000","last_undegraded":"2024-05-09T04:35:24.255620+0000","last_fullsized":"2024-05-09T04:35:24.255620+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:48:00.689997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.253334+0000","last_change":"2024-05-09T04:35:24.249572+0000","last_active":"2024-05-09T04:35:24.253334+0000","last_peered":"2024-05-09T04:35:24.253334+0000","last_clean":"2024-05-09T04:35:24.253334+0000","last_became_active":"2024-05-09T04:35:24.249301+0000","last_became_peered":"2024-05-09T04:35:24.249301+0000","last_unstale":"2024-05-09T04:35:24.253334+0000","last_undegraded":"2024-05-09T04:35:24.253334+0000","last_fullsized":"2024-05-09T04:35:24.253334+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:45.225417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"46'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.250819+0000","last_change":"2024-05-09T04:35:23.215932+0000","last_active":"2024-05-09T04:35:24.250819+0000","last_peered":"2024-05-09T04:35:24.250819+0000","last_clean":"2024-05-09T04:35:24.250819+0000","last_became_active":"2024-05-09T04:35:23.215635+0000","last_became_peered":"2024-05-09T04:35:23.215635+0000","last_unstale":"2024-05-09T04:35:24.250819+0000","last_undegraded":"2024-05-09T04:35:24.250819+0000","last_fullsized":"2024-05-09T04:35:24.250819+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:11:53.567574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.263408+0000","last_change":"2024-05-09T04:35:24.259303+0000","last_active":"2024-05-09T04:35:24.263408+0000","last_peered":"2024-05-09T04:35:24.263408+0000","last_clean":"2024-05-09T04:35:24.263408+0000","last_became_active":"2024-05-09T04:35:24.259044+0000","last_became_peered":"2024-05-09T04:35:24.259044+0000","last_unstale":"2024-05-09T04:35:24.263408+0000","last_undegraded":"2024-05-09T04:35:24.263408+0000","last_fullsized":"2024-05-09T04:35:24.263408+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:47:00.435558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"46'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.257747+0000","last_change":"2024-05-09T04:35:23.232488+0000","last_active":"2024-05-09T04:35:24.257747+0000","last_peered":"2024-05-09T04:35:24.257747+0000","last_clean":"2024-05-09T04:35:24.257747+0000","last_became_active":"2024-05-09T04:35:23.231585+0000","last_became_peered":"2024-05-09T04:35:23.231585+0000","last_unstale":"2024-05-09T04:35:24.257747+0000","last_undegraded":"2024-05-09T04:35:24.257747+0000","last_fullsized":"2024-05-09T04:35:24.257747+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:57:42.352879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.268435+0000","last_change":"2024-05-09T04:35:24.259706+0000","last_active":"2024-05-09T04:35:24.268435+0000","last_peered":"2024-05-09T04:35:24.268435+0000","last_clean":"2024-05-09T04:35:24.268435+0000","last_became_active":"2024-05-09T04:35:24.254043+0000","last_became_peered":"2024-05-09T04:35:24.254043+0000","last_unstale":"2024-05-09T04:35:24.268435+0000","last_undegraded":"2024-05-09T04:35:24.268435+0000","last_fullsized":"2024-05-09T04:35:24.268435+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:36:44.724956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.246917+0000","last_change":"2024-05-09T04:35:23.230199+0000","last_active":"2024-05-09T04:35:24.246917+0000","last_peered":"2024-05-09T04:35:24.246917+0000","last_clean":"2024-05-09T04:35:24.246917+0000","last_became_active":"2024-05-09T04:35:23.229830+0000","last_became_peered":"2024-05-09T04:35:23.229830+0000","last_unstale":"2024-05-09T04:35:24.246917+0000","last_undegraded":"2024-05-09T04:35:24.246917+0000","last_fullsized":"2024-05-09T04:35:24.246917+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:40:59.071549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.265617+0000","last_change":"2024-05-09T04:35:24.262731+0000","last_active":"2024-05-09T04:35:24.265617+0000","last_peered":"2024-05-09T04:35:24.265617+0000","last_clean":"2024-05-09T04:35:24.265617+0000","last_became_active":"2024-05-09T04:35:24.262401+0000","last_became_peered":"2024-05-09T04:35:24.262401+0000","last_unstale":"2024-05-09T04:35:24.265617+0000","last_undegraded":"2024-05-09T04:35:24.265617+0000","last_fullsized":"2024-05-09T04:35:24.265617+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:56:30.472788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.230656+0000","last_change":"2024-05-09T04:35:23.228702+0000","last_active":"2024-05-09T04:35:24.230656+0000","last_peered":"2024-05-09T04:35:24.230656+0000","last_clean":"2024-05-09T04:35:24.230656+0000","last_became_active":"2024-05-09T04:35:23.228026+0000","last_became_peered":"2024-05-09T04:35:23.228026+0000","last_unstale":"2024-05-09T04:35:24.230656+0000","last_undegraded":"2024-05-09T04:35:24.230656+0000","last_fullsized":"2024-05-09T04:35:24.230656+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:45:18.026556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.265896+0000","last_change":"2024-05-09T04:35:24.261499+0000","last_active":"2024-05-09T04:35:24.265896+0000","last_peered":"2024-05-09T04:35:24.265896+0000","last_clean":"2024-05-09T04:35:24.265896+0000","last_became_active":"2024-05-09T04:35:24.260127+0000","last_became_peered":"2024-05-09T04:35:24.260127+0000","last_unstale":"2024-05-09T04:35:24.265896+0000","last_undegraded":"2024-05-09T04:35:24.265896+0000","last_fullsized":"2024-05-09T04:35:24.265896+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:38:58.731420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.248850+0000","last_change":"2024-05-09T04:35:23.230669+0000","last_active":"2024-05-09T04:35:24.248850+0000","last_peered":"2024-05-09T04:35:24.248850+0000","last_clean":"2024-05-09T04:35:24.248850+0000","last_became_active":"2024-05-09T04:35:23.230146+0000","last_became_peered":"2024-05-09T04:35:23.230146+0000","last_unstale":"2024-05-09T04:35:24.248850+0000","last_undegraded":"2024-05-09T04:35:24.248850+0000","last_fullsized":"2024-05-09T04:35:24.248850+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:47:35.099785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.252405+0000","last_change":"2024-05-09T04:35:24.245039+0000","last_active":"2024-05-09T04:35:24.252405+0000","last_peered":"2024-05-09T04:35:24.252405+0000","last_clean":"2024-05-09T04:35:24.252405+0000","last_became_active":"2024-05-09T04:35:24.241594+0000","last_became_peered":"2024-05-09T04:35:24.241594+0000","last_unstale":"2024-05-09T04:35:24.252405+0000","last_undegraded":"2024-05-09T04:35:24.252405+0000","last_fullsized":"2024-05-09T04:35:24.252405+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:06:11.802530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.239272+0000","last_change":"2024-05-09T04:35:23.208131+0000","last_active":"2024-05-09T04:35:24.239272+0000","last_peered":"2024-05-09T04:35:24.239272+0000","last_clean":"2024-05-09T04:35:24.239272+0000","last_became_active":"2024-05-09T04:35:23.207166+0000","last_became_peered":"2024-05-09T04:35:23.207166+0000","last_unstale":"2024-05-09T04:35:24.239272+0000","last_undegraded":"2024-05-09T04:35:24.239272+0000","last_fullsized":"2024-05-09T04:35:24.239272+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:02.848544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.263494+0000","last_change":"2024-05-09T04:35:24.258823+0000","last_active":"2024-05-09T04:35:24.263494+0000","last_peered":"2024-05-09T04:35:24.263494+0000","last_clean":"2024-05-09T04:35:24.263494+0000","last_became_active":"2024-05-09T04:35:24.258458+0000","last_became_peered":"2024-05-09T04:35:24.258458+0000","last_unstale":"2024-05-09T04:35:24.263494+0000","last_undegraded":"2024-05-09T04:35:24.263494+0000","last_fullsized":"2024-05-09T04:35:24.263494+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:43:22.117586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"46'3","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.260347+0000","last_change":"2024-05-09T04:35:23.231022+0000","last_active":"2024-05-09T04:35:24.260347+0000","last_peered":"2024-05-09T04:35:24.260347+0000","last_clean":"2024-05-09T04:35:24.260347+0000","last_became_active":"2024-05-09T04:35:23.230720+0000","last_became_peered":"2024-05-09T04:35:23.230720+0000","last_unstale":"2024-05-09T04:35:24.260347+0000","last_undegraded":"2024-05-09T04:35:24.260347+0000","last_fullsized":"2024-05-09T04:35:24.260347+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:43:21.103656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.246040+0000","last_change":"2024-05-09T04:35:24.241846+0000","last_active":"2024-05-09T04:35:24.246040+0000","last_peered":"2024-05-09T04:35:24.246040+0000","last_clean":"2024-05-09T04:35:24.246040+0000","last_became_active":"2024-05-09T04:35:24.240937+0000","last_became_peered":"2024-05-09T04:35:24.240937+0000","last_unstale":"2024-05-09T04:35:24.246040+0000","last_undegraded":"2024-05-09T04:35:24.246040+0000","last_fullsized":"2024-05-09T04:35:24.246040+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:11:54.581504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"46'6","reported_seq":18,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.268716+0000","last_change":"2024-05-09T04:35:23.220710+0000","last_active":"2024-05-09T04:35:24.268716+0000","last_peered":"2024-05-09T04:35:24.268716+0000","last_clean":"2024-05-09T04:35:24.268716+0000","last_became_active":"2024-05-09T04:35:23.219368+0000","last_became_peered":"2024-05-09T04:35:23.219368+0000","last_unstale":"2024-05-09T04:35:24.268716+0000","last_undegraded":"2024-05-09T04:35:24.268716+0000","last_fullsized":"2024-05-09T04:35:24.268716+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:56:29.458858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.261263+0000","last_change":"2024-05-09T04:35:24.255664+0000","last_active":"2024-05-09T04:35:24.261263+0000","last_peered":"2024-05-09T04:35:24.261263+0000","last_clean":"2024-05-09T04:35:24.261263+0000","last_became_active":"2024-05-09T04:35:24.255068+0000","last_became_peered":"2024-05-09T04:35:24.255068+0000","last_unstale":"2024-05-09T04:35:24.261263+0000","last_undegraded":"2024-05-09T04:35:24.261263+0000","last_fullsized":"2024-05-09T04:35:24.261263+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:08:43.697014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.259256+0000","last_change":"2024-05-09T04:35:23.226565+0000","last_active":"2024-05-09T04:35:24.259256+0000","last_peered":"2024-05-09T04:35:24.259256+0000","last_clean":"2024-05-09T04:35:24.259256+0000","last_became_active":"2024-05-09T04:35:23.226303+0000","last_became_peered":"2024-05-09T04:35:23.226303+0000","last_unstale":"2024-05-09T04:35:24.259256+0000","last_undegraded":"2024-05-09T04:35:24.259256+0000","last_fullsized":"2024-05-09T04:35:24.259256+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:44.211487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.243236+0000","last_change":"2024-05-09T04:35:24.236331+0000","last_active":"2024-05-09T04:35:24.243236+0000","last_peered":"2024-05-09T04:35:24.243236+0000","last_clean":"2024-05-09T04:35:24.243236+0000","last_became_active":"2024-05-09T04:35:24.236058+0000","last_became_peered":"2024-05-09T04:35:24.236058+0000","last_unstale":"2024-05-09T04:35:24.243236+0000","last_undegraded":"2024-05-09T04:35:24.243236+0000","last_fullsized":"2024-05-09T04:35:24.243236+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:38:58.731420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.237899+0000","last_change":"2024-05-09T04:35:23.229353+0000","last_active":"2024-05-09T04:35:24.237899+0000","last_peered":"2024-05-09T04:35:24.237899+0000","last_clean":"2024-05-09T04:35:24.237899+0000","last_became_active":"2024-05-09T04:35:23.228801+0000","last_became_peered":"2024-05-09T04:35:23.228801+0000","last_unstale":"2024-05-09T04:35:24.237899+0000","last_undegraded":"2024-05-09T04:35:24.237899+0000","last_fullsized":"2024-05-09T04:35:24.237899+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:14:35.671258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.268289+0000","last_change":"2024-05-09T04:35:24.259077+0000","last_active":"2024-05-09T04:35:24.268289+0000","last_peered":"2024-05-09T04:35:24.268289+0000","last_clean":"2024-05-09T04:35:24.268289+0000","last_became_active":"2024-05-09T04:35:24.258266+0000","last_became_peered":"2024-05-09T04:35:24.258266+0000","last_unstale":"2024-05-09T04:35:24.268289+0000","last_undegraded":"2024-05-09T04:35:24.268289+0000","last_fullsized":"2024-05-09T04:35:24.268289+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:07:02.781092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.245780+0000","last_change":"2024-05-09T04:35:23.233816+0000","last_active":"2024-05-09T04:35:24.245780+0000","last_peered":"2024-05-09T04:35:24.245780+0000","last_clean":"2024-05-09T04:35:24.245780+0000","last_became_active":"2024-05-09T04:35:23.233096+0000","last_became_peered":"2024-05-09T04:35:23.233096+0000","last_unstale":"2024-05-09T04:35:24.245780+0000","last_undegraded":"2024-05-09T04:35:24.245780+0000","last_fullsized":"2024-05-09T04:35:24.245780+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:08:42.683084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.261362+0000","last_change":"2024-05-09T04:35:24.248388+0000","last_active":"2024-05-09T04:35:24.261362+0000","last_peered":"2024-05-09T04:35:24.261362+0000","last_clean":"2024-05-09T04:35:24.261362+0000","last_became_active":"2024-05-09T04:35:24.248132+0000","last_became_peered":"2024-05-09T04:35:24.248132+0000","last_unstale":"2024-05-09T04:35:24.261362+0000","last_undegraded":"2024-05-09T04:35:24.261362+0000","last_fullsized":"2024-05-09T04:35:24.261362+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:06:11.802530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"46'8","reported_seq":20,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.269539+0000","last_change":"2024-05-09T04:35:23.215879+0000","last_active":"2024-05-09T04:35:24.269539+0000","last_peered":"2024-05-09T04:35:24.269539+0000","last_clean":"2024-05-09T04:35:24.269539+0000","last_became_active":"2024-05-09T04:35:23.214959+0000","last_became_peered":"2024-05-09T04:35:23.214959+0000","last_unstale":"2024-05-09T04:35:24.269539+0000","last_undegraded":"2024-05-09T04:35:24.269539+0000","last_fullsized":"2024-05-09T04:35:24.269539+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:19:26.552697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.268522+0000","last_change":"2024-05-09T04:35:24.263955+0000","last_active":"2024-05-09T04:35:24.268522+0000","last_peered":"2024-05-09T04:35:24.268522+0000","last_clean":"2024-05-09T04:35:24.268522+0000","last_became_active":"2024-05-09T04:35:24.262715+0000","last_became_peered":"2024-05-09T04:35:24.262715+0000","last_unstale":"2024-05-09T04:35:24.268522+0000","last_undegraded":"2024-05-09T04:35:24.268522+0000","last_fullsized":"2024-05-09T04:35:24.268522+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:43:34.179388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"46'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.264475+0000","last_change":"2024-05-09T04:35:23.218612+0000","last_active":"2024-05-09T04:35:24.264475+0000","last_peered":"2024-05-09T04:35:24.264475+0000","last_clean":"2024-05-09T04:35:24.264475+0000","last_became_active":"2024-05-09T04:35:23.218258+0000","last_became_peered":"2024-05-09T04:35:23.218258+0000","last_unstale":"2024-05-09T04:35:24.264475+0000","last_undegraded":"2024-05-09T04:35:24.264475+0000","last_fullsized":"2024-05-09T04:35:24.264475+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:05:43.481549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.244673+0000","last_change":"2024-05-09T04:35:24.242891+0000","last_active":"2024-05-09T04:35:24.244673+0000","last_peered":"2024-05-09T04:35:24.244673+0000","last_clean":"2024-05-09T04:35:24.244673+0000","last_became_active":"2024-05-09T04:35:24.241067+0000","last_became_peered":"2024-05-09T04:35:24.241067+0000","last_unstale":"2024-05-09T04:35:24.244673+0000","last_undegraded":"2024-05-09T04:35:24.244673+0000","last_fullsized":"2024-05-09T04:35:24.244673+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:11:24.172582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.248889+0000","last_change":"2024-05-09T04:35:23.223741+0000","last_active":"2024-05-09T04:35:24.248889+0000","last_peered":"2024-05-09T04:35:24.248889+0000","last_clean":"2024-05-09T04:35:24.248889+0000","last_became_active":"2024-05-09T04:35:23.223426+0000","last_became_peered":"2024-05-09T04:35:23.223426+0000","last_unstale":"2024-05-09T04:35:24.248889+0000","last_undegraded":"2024-05-09T04:35:24.248889+0000","last_fullsized":"2024-05-09T04:35:24.248889+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:40:30.949891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.245708+0000","last_change":"2024-05-09T04:35:24.240282+0000","last_active":"2024-05-09T04:35:24.245708+0000","last_peered":"2024-05-09T04:35:24.245708+0000","last_clean":"2024-05-09T04:35:24.245708+0000","last_became_active":"2024-05-09T04:35:24.239867+0000","last_became_peered":"2024-05-09T04:35:24.239867+0000","last_unstale":"2024-05-09T04:35:24.245708+0000","last_undegraded":"2024-05-09T04:35:24.245708+0000","last_fullsized":"2024-05-09T04:35:24.245708+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:45:19.040487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"46'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.256441+0000","last_change":"2024-05-09T04:35:23.233478+0000","last_active":"2024-05-09T04:35:24.256441+0000","last_peered":"2024-05-09T04:35:24.256441+0000","last_clean":"2024-05-09T04:35:24.256441+0000","last_became_active":"2024-05-09T04:35:23.233123+0000","last_became_peered":"2024-05-09T04:35:23.233123+0000","last_unstale":"2024-05-09T04:35:24.256441+0000","last_undegraded":"2024-05-09T04:35:24.256441+0000","last_fullsized":"2024-05-09T04:35:24.256441+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:50:54.527868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.258590+0000","last_change":"2024-05-09T04:35:24.252157+0000","last_active":"2024-05-09T04:35:24.258590+0000","last_peered":"2024-05-09T04:35:24.258590+0000","last_clean":"2024-05-09T04:35:24.258590+0000","last_became_active":"2024-05-09T04:35:24.251826+0000","last_became_peered":"2024-05-09T04:35:24.251826+0000","last_unstale":"2024-05-09T04:35:24.258590+0000","last_undegraded":"2024-05-09T04:35:24.258590+0000","last_fullsized":"2024-05-09T04:35:24.258590+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:15:49.152245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.249106+0000","last_change":"2024-05-09T04:35:23.230027+0000","last_active":"2024-05-09T04:35:24.249106+0000","last_peered":"2024-05-09T04:35:24.249106+0000","last_clean":"2024-05-09T04:35:24.249106+0000","last_became_active":"2024-05-09T04:35:23.227428+0000","last_became_peered":"2024-05-09T04:35:23.227428+0000","last_unstale":"2024-05-09T04:35:24.249106+0000","last_undegraded":"2024-05-09T04:35:24.249106+0000","last_fullsized":"2024-05-09T04:35:24.249106+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:32:14.116321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.255534+0000","last_change":"2024-05-09T04:35:24.249839+0000","last_active":"2024-05-09T04:35:24.255534+0000","last_peered":"2024-05-09T04:35:24.255534+0000","last_clean":"2024-05-09T04:35:24.255534+0000","last_became_active":"2024-05-09T04:35:24.249526+0000","last_became_peered":"2024-05-09T04:35:24.249526+0000","last_unstale":"2024-05-09T04:35:24.255534+0000","last_undegraded":"2024-05-09T04:35:24.255534+0000","last_fullsized":"2024-05-09T04:35:24.255534+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:48:53.260192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.251685+0000","last_change":"2024-05-09T04:35:23.228101+0000","last_active":"2024-05-09T04:35:24.251685+0000","last_peered":"2024-05-09T04:35:24.251685+0000","last_clean":"2024-05-09T04:35:24.251685+0000","last_became_active":"2024-05-09T04:35:23.227279+0000","last_became_peered":"2024-05-09T04:35:23.227279+0000","last_unstale":"2024-05-09T04:35:24.251685+0000","last_undegraded":"2024-05-09T04:35:24.251685+0000","last_fullsized":"2024-05-09T04:35:24.251685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:11:53.567574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.262210+0000","last_change":"2024-05-09T04:35:24.256874+0000","last_active":"2024-05-09T04:35:24.262210+0000","last_peered":"2024-05-09T04:35:24.262210+0000","last_clean":"2024-05-09T04:35:24.262210+0000","last_became_active":"2024-05-09T04:35:24.256492+0000","last_became_peered":"2024-05-09T04:35:24.256492+0000","last_unstale":"2024-05-09T04:35:24.262210+0000","last_undegraded":"2024-05-09T04:35:24.262210+0000","last_fullsized":"2024-05-09T04:35:24.262210+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:45:10.725727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.227046+0000","last_change":"2024-05-09T04:35:23.216804+0000","last_active":"2024-05-09T04:35:24.227046+0000","last_peered":"2024-05-09T04:35:24.227046+0000","last_clean":"2024-05-09T04:35:24.227046+0000","last_became_active":"2024-05-09T04:35:23.216518+0000","last_became_peered":"2024-05-09T04:35:23.216518+0000","last_unstale":"2024-05-09T04:35:24.227046+0000","last_undegraded":"2024-05-09T04:35:24.227046+0000","last_fullsized":"2024-05-09T04:35:24.227046+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:03:07.458395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.263814+0000","last_change":"2024-05-09T04:35:24.260012+0000","last_active":"2024-05-09T04:35:24.263814+0000","last_peered":"2024-05-09T04:35:24.263814+0000","last_clean":"2024-05-09T04:35:24.263814+0000","last_became_active":"2024-05-09T04:35:24.259675+0000","last_became_peered":"2024-05-09T04:35:24.259675+0000","last_unstale":"2024-05-09T04:35:24.263814+0000","last_undegraded":"2024-05-09T04:35:24.263814+0000","last_fullsized":"2024-05-09T04:35:24.263814+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:41:23.533288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.228776+0000","last_change":"2024-05-09T04:35:23.223222+0000","last_active":"2024-05-09T04:35:24.228776+0000","last_peered":"2024-05-09T04:35:24.228776+0000","last_clean":"2024-05-09T04:35:24.228776+0000","last_became_active":"2024-05-09T04:35:23.222919+0000","last_became_peered":"2024-05-09T04:35:23.222919+0000","last_unstale":"2024-05-09T04:35:24.228776+0000","last_undegraded":"2024-05-09T04:35:24.228776+0000","last_fullsized":"2024-05-09T04:35:24.228776+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:02.848544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.260815+0000","last_change":"2024-05-09T04:35:24.255432+0000","last_active":"2024-05-09T04:35:24.260815+0000","last_peered":"2024-05-09T04:35:24.260815+0000","last_clean":"2024-05-09T04:35:24.260815+0000","last_became_active":"2024-05-09T04:35:24.255142+0000","last_became_peered":"2024-05-09T04:35:24.255142+0000","last_unstale":"2024-05-09T04:35:24.260815+0000","last_undegraded":"2024-05-09T04:35:24.260815+0000","last_fullsized":"2024-05-09T04:35:24.260815+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:20:57.932878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.219637+0000","last_change":"2024-05-09T04:35:23.228776+0000","last_active":"2024-05-09T04:35:24.219637+0000","last_peered":"2024-05-09T04:35:24.219637+0000","last_clean":"2024-05-09T04:35:24.219637+0000","last_became_active":"2024-05-09T04:35:23.227388+0000","last_became_peered":"2024-05-09T04:35:23.227388+0000","last_unstale":"2024-05-09T04:35:24.219637+0000","last_undegraded":"2024-05-09T04:35:24.219637+0000","last_fullsized":"2024-05-09T04:35:24.219637+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:36:13.725524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.263526+0000","last_change":"2024-05-09T04:35:24.259870+0000","last_active":"2024-05-09T04:35:24.263526+0000","last_peered":"2024-05-09T04:35:24.263526+0000","last_clean":"2024-05-09T04:35:24.263526+0000","last_became_active":"2024-05-09T04:35:24.259548+0000","last_became_peered":"2024-05-09T04:35:24.259548+0000","last_unstale":"2024-05-09T04:35:24.263526+0000","last_undegraded":"2024-05-09T04:35:24.263526+0000","last_fullsized":"2024-05-09T04:35:24.263526+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:43:22.117586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.261595+0000","last_change":"2024-05-09T04:35:23.235093+0000","last_active":"2024-05-09T04:35:24.261595+0000","last_peered":"2024-05-09T04:35:24.261595+0000","last_clean":"2024-05-09T04:35:24.261595+0000","last_became_active":"2024-05-09T04:35:23.234833+0000","last_became_peered":"2024-05-09T04:35:23.234833+0000","last_unstale":"2024-05-09T04:35:24.261595+0000","last_undegraded":"2024-05-09T04:35:24.261595+0000","last_fullsized":"2024-05-09T04:35:24.261595+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:08:02.005553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.243514+0000","last_change":"2024-05-09T04:35:24.238100+0000","last_active":"2024-05-09T04:35:24.243514+0000","last_peered":"2024-05-09T04:35:24.243514+0000","last_clean":"2024-05-09T04:35:24.243514+0000","last_became_active":"2024-05-09T04:35:24.237649+0000","last_became_peered":"2024-05-09T04:35:24.237649+0000","last_unstale":"2024-05-09T04:35:24.243514+0000","last_undegraded":"2024-05-09T04:35:24.243514+0000","last_fullsized":"2024-05-09T04:35:24.243514+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:17:18.226996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.249024+0000","last_change":"2024-05-09T04:35:23.229778+0000","last_active":"2024-05-09T04:35:24.249024+0000","last_peered":"2024-05-09T04:35:24.249024+0000","last_clean":"2024-05-09T04:35:24.249024+0000","last_became_active":"2024-05-09T04:35:23.229538+0000","last_became_peered":"2024-05-09T04:35:23.229538+0000","last_unstale":"2024-05-09T04:35:24.249024+0000","last_undegraded":"2024-05-09T04:35:24.249024+0000","last_fullsized":"2024-05-09T04:35:24.249024+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:41:22.519358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.260463+0000","last_change":"2024-05-09T04:35:24.252509+0000","last_active":"2024-05-09T04:35:24.260463+0000","last_peered":"2024-05-09T04:35:24.260463+0000","last_clean":"2024-05-09T04:35:24.260463+0000","last_became_active":"2024-05-09T04:35:24.251843+0000","last_became_peered":"2024-05-09T04:35:24.251843+0000","last_unstale":"2024-05-09T04:35:24.260463+0000","last_undegraded":"2024-05-09T04:35:24.260463+0000","last_fullsized":"2024-05-09T04:35:24.260463+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:44:42.597472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.228498+0000","last_change":"2024-05-09T04:35:23.219904+0000","last_active":"2024-05-09T04:35:24.228498+0000","last_peered":"2024-05-09T04:35:24.228498+0000","last_clean":"2024-05-09T04:35:24.228498+0000","last_became_active":"2024-05-09T04:35:23.219605+0000","last_became_peered":"2024-05-09T04:35:23.219605+0000","last_unstale":"2024-05-09T04:35:24.228498+0000","last_undegraded":"2024-05-09T04:35:24.228498+0000","last_fullsized":"2024-05-09T04:35:24.228498+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:57:42.352879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.263517+0000","last_change":"2024-05-09T04:35:24.259603+0000","last_active":"2024-05-09T04:35:24.263517+0000","last_peered":"2024-05-09T04:35:24.263517+0000","last_clean":"2024-05-09T04:35:24.263517+0000","last_became_active":"2024-05-09T04:35:24.259350+0000","last_became_peered":"2024-05-09T04:35:24.259350+0000","last_unstale":"2024-05-09T04:35:24.263517+0000","last_undegraded":"2024-05-09T04:35:24.263517+0000","last_fullsized":"2024-05-09T04:35:24.263517+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:47:36.113715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.233706+0000","last_change":"2024-05-09T04:35:23.214211+0000","last_active":"2024-05-09T04:35:24.233706+0000","last_peered":"2024-05-09T04:35:24.233706+0000","last_clean":"2024-05-09T04:35:24.233706+0000","last_became_active":"2024-05-09T04:35:23.213905+0000","last_became_peered":"2024-05-09T04:35:23.213905+0000","last_unstale":"2024-05-09T04:35:24.233706+0000","last_undegraded":"2024-05-09T04:35:24.233706+0000","last_fullsized":"2024-05-09T04:35:24.233706+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:51:10.263151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.258539+0000","last_change":"2024-05-09T04:35:24.252148+0000","last_active":"2024-05-09T04:35:24.258539+0000","last_peered":"2024-05-09T04:35:24.258539+0000","last_clean":"2024-05-09T04:35:24.258539+0000","last_became_active":"2024-05-09T04:35:24.251778+0000","last_became_peered":"2024-05-09T04:35:24.251778+0000","last_unstale":"2024-05-09T04:35:24.258539+0000","last_undegraded":"2024-05-09T04:35:24.258539+0000","last_fullsized":"2024-05-09T04:35:24.258539+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:09:14.673363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.227258+0000","last_change":"2024-05-09T04:35:23.221479+0000","last_active":"2024-05-09T04:35:24.227258+0000","last_peered":"2024-05-09T04:35:24.227258+0000","last_clean":"2024-05-09T04:35:24.227258+0000","last_became_active":"2024-05-09T04:35:23.220945+0000","last_became_peered":"2024-05-09T04:35:23.220945+0000","last_unstale":"2024-05-09T04:35:24.227258+0000","last_undegraded":"2024-05-09T04:35:24.227258+0000","last_fullsized":"2024-05-09T04:35:24.227258+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:46:59.421628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.255940+0000","last_change":"2024-05-09T04:35:24.249909+0000","last_active":"2024-05-09T04:35:24.255940+0000","last_peered":"2024-05-09T04:35:24.255940+0000","last_clean":"2024-05-09T04:35:24.255940+0000","last_became_active":"2024-05-09T04:35:24.249495+0000","last_became_peered":"2024-05-09T04:35:24.249495+0000","last_unstale":"2024-05-09T04:35:24.255940+0000","last_undegraded":"2024-05-09T04:35:24.255940+0000","last_fullsized":"2024-05-09T04:35:24.255940+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:48:53.260192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.233400+0000","last_change":"2024-05-09T04:35:23.218454+0000","last_active":"2024-05-09T04:35:24.233400+0000","last_peered":"2024-05-09T04:35:24.233400+0000","last_clean":"2024-05-09T04:35:24.233400+0000","last_became_active":"2024-05-09T04:35:23.218150+0000","last_became_peered":"2024-05-09T04:35:23.218150+0000","last_unstale":"2024-05-09T04:35:24.233400+0000","last_undegraded":"2024-05-09T04:35:24.233400+0000","last_fullsized":"2024-05-09T04:35:24.233400+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:05:43.481549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.252856+0000","last_change":"2024-05-09T04:35:24.248678+0000","last_active":"2024-05-09T04:35:24.252856+0000","last_peered":"2024-05-09T04:35:24.252856+0000","last_clean":"2024-05-09T04:35:24.252856+0000","last_became_active":"2024-05-09T04:35:24.246998+0000","last_became_peered":"2024-05-09T04:35:24.246998+0000","last_unstale":"2024-05-09T04:35:24.252856+0000","last_undegraded":"2024-05-09T04:35:24.252856+0000","last_fullsized":"2024-05-09T04:35:24.252856+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:29:15.109166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.225907+0000","last_change":"2024-05-09T04:35:23.227744+0000","last_active":"2024-05-09T04:35:24.225907+0000","last_peered":"2024-05-09T04:35:24.225907+0000","last_clean":"2024-05-09T04:35:24.225907+0000","last_became_active":"2024-05-09T04:35:23.227333+0000","last_became_peered":"2024-05-09T04:35:23.227333+0000","last_unstale":"2024-05-09T04:35:24.225907+0000","last_undegraded":"2024-05-09T04:35:24.225907+0000","last_fullsized":"2024-05-09T04:35:24.225907+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:11:23.158652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.262180+0000","last_change":"2024-05-09T04:35:24.256432+0000","last_active":"2024-05-09T04:35:24.262180+0000","last_peered":"2024-05-09T04:35:24.262180+0000","last_clean":"2024-05-09T04:35:24.262180+0000","last_became_active":"2024-05-09T04:35:24.256139+0000","last_became_peered":"2024-05-09T04:35:24.256139+0000","last_unstale":"2024-05-09T04:35:24.262180+0000","last_undegraded":"2024-05-09T04:35:24.262180+0000","last_fullsized":"2024-05-09T04:35:24.262180+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:23.183189+0000","last_clean_scrub_stamp":"2024-05-09T04:35:23.183189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:39:34.024712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.232838+0000","last_change":"2024-05-09T04:35:23.229344+0000","last_active":"2024-05-09T04:35:24.232838+0000","last_peered":"2024-05-09T04:35:24.232838+0000","last_clean":"2024-05-09T04:35:24.232838+0000","last_became_active":"2024-05-09T04:35:23.227086+0000","last_became_peered":"2024-05-09T04:35:23.227086+0000","last_unstale":"2024-05-09T04:35:24.232838+0000","last_undegraded":"2024-05-09T04:35:24.232838+0000","last_fullsized":"2024-05-09T04:35:24.232838+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:35:22.169259+0000","last_clean_scrub_stamp":"2024-05-09T04:35:22.169259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:10:09.760964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":158,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-05-09T04:35:24.244258+0000","last_change":"2024-05-09T04:33:41.092063+0000","last_active":"2024-05-09T04:35:24.244258+0000","last_peered":"2024-05-09T04:35:24.244258+0000","last_clean":"2024-05-09T04:35:24.244258+0000","last_became_active":"2024-05-09T04:33:41.091194+0000","last_became_peered":"2024-05-09T04:33:41.091194+0000","last_unstale":"2024-05-09T04:35:24.244258+0000","last_undegraded":"2024-05-09T04:35:24.244258+0000","last_fullsized":"2024-05-09T04:35:24.244258+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:40.045339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:40.045339+0000","last_clean_scrub_stamp":"2024-05-09T04:33:40.045339+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:04.908456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28144,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343696,"statfs":{"total":96636764160,"available":96607944704,"internally_reserved":0,"allocated":1146880,"data_stored":814861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":552960,"data_stored":225021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27560,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344280,"statfs":{"total":96636764160,"available":96608542720,"internally_reserved":0,"allocated":544768,"data_stored":224369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27564,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344276,"statfs":{"total":96636764160,"available":96608538624,"internally_reserved":0,"allocated":548864,"data_stored":224951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27544,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344296,"statfs":{"total":96636764160,"available":96608559104,"internally_reserved":0,"allocated":532480,"data_stored":217594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27572,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344268,"statfs":{"total":96636764160,"available":96608530432,"internally_reserved":0,"allocated":557056,"data_stored":225981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31724,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340116,"statfs":{"total":96636764160,"available":96604278784,"internally_reserved":0,"allocated":552960,"data_stored":224529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32304,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339536,"statfs":{"total":96636764160,"available":96603684864,"internally_reserved":0,"allocated":1146880,"data_stored":814793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T04:35:29.170 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-09T04:35:29.171 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-05-09T04:35:29.416 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.416+0000 7f0a86e00640 1 Processor -- start 2024-05-09T04:35:29.416 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.417+0000 7f0a86e00640 1 -- start start 2024-05-09T04:35:29.418 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.418+0000 7f0a86e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f0a800d5a00 0x7f0a800d3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:29.418 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.418+0000 7f0a86e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a800d3b70 0x7f0a800b53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:29.418 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.418+0000 7f0a86e00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a800d2290 con 0x7f0a8015f360 2024-05-09T04:35:29.418 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.418+0000 7f0a86e00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f0a800d3fc0 con 0x7f0a800d5a00 2024-05-09T04:35:29.418 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.418+0000 7f0a86e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0a800d2030 con 0x7f0a800d3b70 2024-05-09T04:35:29.418 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.418+0000 7f0a85a00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f0a8015f360 0x7f0a800d1920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:55210/0 (socket says 172.21.15.145:55210) 2024-05-09T04:35:29.418 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.418+0000 7f0a85a00640 1 -- 172.21.15.145:0/137487958 learned_addr learned my addr 172.21.15.145:0/137487958 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:35:29.419 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.419+0000 7f0a85000640 1 --2- 172.21.15.145:0/137487958 >> v2:172.21.15.145:3300/0 conn(0x7f0a800d5a00 0x7f0a800d3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:29.419 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.419+0000 7f0a7fe00640 1 -- 172.21.15.145:0/137487958 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1233543964 0 0) 0x7f0a800d2290 con 0x7f0a8015f360 2024-05-09T04:35:29.419 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.419+0000 7f0a7fe00640 1 -- 172.21.15.145:0/137487958 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0a68003520 con 0x7f0a8015f360 2024-05-09T04:35:29.420 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.419+0000 7f0a86400640 1 --2- 172.21.15.145:0/137487958 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a800d3b70 0x7f0a800b53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:29.420 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.419+0000 7f0a85000640 1 -- 172.21.15.145:0/137487958 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a800d3b70 msgr2=0x7f0a800b53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:29.420 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.419+0000 7f0a85000640 1 --2- 172.21.15.145:0/137487958 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a800d3b70 0x7f0a800b53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:29.420 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.419+0000 7f0a85000640 1 -- 172.21.15.145:0/137487958 >> v1:172.21.15.145:6789/0 conn(0x7f0a8015f360 legacy=0x7f0a800d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:29.420 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.419+0000 7f0a85000640 1 -- 172.21.15.145:0/137487958 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a800d2570 con 0x7f0a800d5a00 2024-05-09T04:35:29.420 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.419+0000 7f0a86400640 1 --2- 172.21.15.145:0/137487958 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a800d3b70 0x7f0a800b53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:35:29.421 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.420+0000 7f0a85000640 1 --2- 172.21.15.145:0/137487958 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0a800d5a00 0x7f0a800d3630 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f0a740097a0 tx=0x7f0a7402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af8e753025283124 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:29.422 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.420+0000 7f0a7fe00640 1 -- 172.21.15.145:0/137487958 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a74037070 con 0x7f0a800d5a00 2024-05-09T04:35:29.422 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.421+0000 7f0a7fe00640 1 -- 172.21.15.145:0/137487958 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a74032440 con 0x7f0a800d5a00 2024-05-09T04:35:29.422 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.421+0000 7f0a7fe00640 1 -- 172.21.15.145:0/137487958 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a74032810 con 0x7f0a800d5a00 2024-05-09T04:35:29.422 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.421+0000 7f0a86e00640 1 -- 172.21.15.145:0/137487958 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0a800d5a00 msgr2=0x7f0a800d3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:29.422 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.421+0000 7f0a86e00640 1 --2- 172.21.15.145:0/137487958 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0a800d5a00 0x7f0a800d3630 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f0a740097a0 tx=0x7f0a7402a180 comp rx=0 tx=0).stop 2024-05-09T04:35:29.422 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.421+0000 7f0a86e00640 1 -- 172.21.15.145:0/137487958 shutdown_connections 2024-05-09T04:35:29.422 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.421+0000 7f0a86e00640 1 --2- 172.21.15.145:0/137487958 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a800d3b70 0x7f0a800b53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:29.422 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.421+0000 7f0a86e00640 1 --2- 172.21.15.145:0/137487958 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0a800d5a00 0x7f0a800d3630 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:29.422 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.421+0000 7f0a86e00640 1 -- 172.21.15.145:0/137487958 >> 172.21.15.145:0/137487958 conn(0x7f0a8009e380 msgr2=0x7f0a800d5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:29.422 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.421+0000 7f0a86e00640 1 -- 172.21.15.145:0/137487958 shutdown_connections 2024-05-09T04:35:29.422 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.421+0000 7f0a86e00640 1 -- 172.21.15.145:0/137487958 wait complete. 2024-05-09T04:35:29.423 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.423+0000 7f0a86e00640 1 Processor -- start 2024-05-09T04:35:29.423 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.423+0000 7f0a86e00640 1 -- start start 2024-05-09T04:35:29.425 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.424+0000 7f0a86e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a800d3b70 0x7f0a800ba570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:29.426 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.424+0000 7f0a86e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a800d5a00 0x7f0a8015d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:29.426 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.424+0000 7f0a86e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0a8015f360 0x7f0a800ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:29.426 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.424+0000 7f0a86e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0a800b4e80 con 0x7f0a8015f360 2024-05-09T04:35:29.426 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.424+0000 7f0a86e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0a800b4ba0 con 0x7f0a800d5a00 2024-05-09T04:35:29.426 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.424+0000 7f0a86e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0a8015f740 con 0x7f0a800d3b70 2024-05-09T04:35:29.426 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.425+0000 7f0a86400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0a8015f360 0x7f0a800ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:29.426 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.425+0000 7f0a86400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0a8015f360 0x7f0a800ce2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:52954/0 (socket says 172.21.15.145:52954) 2024-05-09T04:35:29.426 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.425+0000 7f0a86400640 1 -- 172.21.15.145:0/3511406928 learned_addr learned my addr 172.21.15.145:0/3511406928 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:29.426 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.425+0000 7f0a85000640 1 --2- 172.21.15.145:0/3511406928 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a800d5a00 0x7f0a8015d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:29.426 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.425+0000 7f0a85a00640 1 --2- 172.21.15.145:0/3511406928 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a800d3b70 0x7f0a800ba570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:29.426 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.425+0000 7f0a86400640 1 -- 172.21.15.145:0/3511406928 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a800d3b70 msgr2=0x7f0a800ba570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:29.427 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.425+0000 7f0a86400640 1 --2- 172.21.15.145:0/3511406928 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a800d3b70 0x7f0a800ba570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:29.427 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.425+0000 7f0a86400640 1 -- 172.21.15.145:0/3511406928 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a800d5a00 msgr2=0x7f0a8015d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:29.427 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.425+0000 7f0a86400640 1 --2- 172.21.15.145:0/3511406928 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a800d5a00 0x7f0a8015d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:29.427 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.425+0000 7f0a86400640 1 -- 172.21.15.145:0/3511406928 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a801f9120 con 0x7f0a8015f360 2024-05-09T04:35:29.427 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.425+0000 7f0a85000640 1 --2- 172.21.15.145:0/3511406928 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a800d5a00 0x7f0a8015d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:35:29.427 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.425+0000 7f0a86400640 1 --2- 172.21.15.145:0/3511406928 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0a8015f360 0x7f0a800ce2f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f0a7800cd50 tx=0x7f0a7800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:29.427 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.426+0000 7f0a7d600640 1 -- 172.21.15.145:0/3511406928 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a7800f680 con 0x7f0a8015f360 2024-05-09T04:35:29.427 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.426+0000 7f0a7d600640 1 -- 172.21.15.145:0/3511406928 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a7800f840 con 0x7f0a8015f360 2024-05-09T04:35:29.427 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.426+0000 7f0a7d600640 1 -- 172.21.15.145:0/3511406928 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a7800fc30 con 0x7f0a8015f360 2024-05-09T04:35:29.427 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.426+0000 7f0a86e00640 1 -- 172.21.15.145:0/3511406928 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a800b9a90 con 0x7f0a8015f360 2024-05-09T04:35:29.427 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.426+0000 7f0a86e00640 1 -- 172.21.15.145:0/3511406928 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a8015d8f0 con 0x7f0a8015f360 2024-05-09T04:35:29.428 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.427+0000 7f0a86e00640 1 -- 172.21.15.145:0/3511406928 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a800b9880 con 0x7f0a8015f360 2024-05-09T04:35:29.428 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.428+0000 7f0a7d600640 1 -- 172.21.15.145:0/3511406928 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0a780213f0 con 0x7f0a8015f360 2024-05-09T04:35:29.429 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.429+0000 7f0a7d600640 1 --2- 172.21.15.145:0/3511406928 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f0a64076320 0x7f0a640787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:29.429 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.430+0000 7f0a7d600640 1 -- 172.21.15.145:0/3511406928 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f0a78098400 con 0x7f0a8015f360 2024-05-09T04:35:29.429 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.430+0000 7f0a85a00640 1 --2- 172.21.15.145:0/3511406928 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f0a64076320 0x7f0a640787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:29.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.431+0000 7f0a85a00640 1 --2- 172.21.15.145:0/3511406928 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f0a64076320 0x7f0a640787e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0a70001220 tx=0x7f0a70009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:29.433 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.434+0000 7f0a7d600640 1 -- 172.21.15.145:0/3511406928 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0a78014030 con 0x7f0a8015f360 2024-05-09T04:35:29.715 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.716+0000 7f0a86e00640 1 -- 172.21.15.145:0/3511406928 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f0a80076890 con 0x7f0a8015f360 2024-05-09T04:35:29.720 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.721+0000 7f0a7d600640 1 -- 172.21.15.145:0/3511406928 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) ==== 190+0+147 (secure 0 0 0) 0x7f0a78061d40 con 0x7f0a8015f360 2024-05-09T04:35:29.721 INFO:teuthology.orchestra.run.smithi145.stdout:[client.0] 2024-05-09T04:35:29.721 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQDmUTxmfB7QDRAAs0TGpMOXJSerMXVrobw+hA== 2024-05-09T04:35:29.721 INFO:teuthology.orchestra.run.smithi145.stdout: caps mds = "allow" 2024-05-09T04:35:29.721 INFO:teuthology.orchestra.run.smithi145.stdout: caps mon = "allow r" 2024-05-09T04:35:29.721 INFO:teuthology.orchestra.run.smithi145.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-05-09T04:35:29.721 INFO:teuthology.orchestra.run.smithi145.stderr:updated caps for client.0 2024-05-09T04:35:29.724 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.724+0000 7f0a86e00640 1 -- 172.21.15.145:0/3511406928 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f0a64076320 msgr2=0x7f0a640787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:29.724 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.724+0000 7f0a86e00640 1 --2- 172.21.15.145:0/3511406928 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f0a64076320 0x7f0a640787e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0a70001220 tx=0x7f0a70009040 comp rx=0 tx=0).stop 2024-05-09T04:35:29.724 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.724+0000 7f0a86e00640 1 -- 172.21.15.145:0/3511406928 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0a8015f360 msgr2=0x7f0a800ce2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:29.724 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.724+0000 7f0a86e00640 1 --2- 172.21.15.145:0/3511406928 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0a8015f360 0x7f0a800ce2f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f0a7800cd50 tx=0x7f0a7800a650 comp rx=0 tx=0).stop 2024-05-09T04:35:29.724 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.724+0000 7f0a86e00640 1 -- 172.21.15.145:0/3511406928 shutdown_connections 2024-05-09T04:35:29.724 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.724+0000 7f0a86e00640 1 --2- 172.21.15.145:0/3511406928 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f0a64076320 0x7f0a640787e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:29.724 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.724+0000 7f0a86e00640 1 --2- 172.21.15.145:0/3511406928 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0a8015f360 0x7f0a800ce2f0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:29.724 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.724+0000 7f0a86e00640 1 --2- 172.21.15.145:0/3511406928 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a800d5a00 0x7f0a8015d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:29.724 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.724+0000 7f0a86e00640 1 --2- 172.21.15.145:0/3511406928 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a800d3b70 0x7f0a800ba570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:29.725 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.724+0000 7f0a86e00640 1 -- 172.21.15.145:0/3511406928 >> 172.21.15.145:0/3511406928 conn(0x7f0a8009e380 msgr2=0x7f0a800b3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:29.725 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.724+0000 7f0a86e00640 1 -- 172.21.15.145:0/3511406928 shutdown_connections 2024-05-09T04:35:29.725 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.724+0000 7f0a86e00640 1 -- 172.21.15.145:0/3511406928 wait complete. 2024-05-09T04:35:29.743 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T04:35:29.936 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.936+0000 7f0d2b000640 1 Processor -- start 2024-05-09T04:35:29.936 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.936+0000 7f0d2b000640 1 -- start start 2024-05-09T04:35:29.937 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.937+0000 7f0d2b000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f0d2415bec0 0x7f0d2415f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:29.937 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.937+0000 7f0d2b000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d2415fde0 0x7f0d240cfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:29.938 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.937+0000 7f0d2b000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d24160280 con 0x7f0d2415bae0 2024-05-09T04:35:29.938 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.937+0000 7f0d2b000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f0d24160840 con 0x7f0d2415bec0 2024-05-09T04:35:29.938 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.937+0000 7f0d2b000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0d240b4190 con 0x7f0d2415fde0 2024-05-09T04:35:29.938 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.938+0000 7f0d29c00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f0d2415bae0 0x7f0d240721b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:55214/0 (socket says 172.21.15.145:55214) 2024-05-09T04:35:29.938 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.938+0000 7f0d29c00640 1 -- 172.21.15.145:0/2516133763 learned_addr learned my addr 172.21.15.145:0/2516133763 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:35:29.938 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.938+0000 7f0d29200640 1 --2- 172.21.15.145:0/2516133763 >> v2:172.21.15.145:3300/0 conn(0x7f0d2415bec0 0x7f0d2415f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:29.939 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.938+0000 7f0d23e00640 1 -- 172.21.15.145:0/2516133763 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1563365200 0 0) 0x7f0d24160280 con 0x7f0d2415bae0 2024-05-09T04:35:29.939 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.938+0000 7f0d23e00640 1 -- 172.21.15.145:0/2516133763 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0d0c005730 con 0x7f0d2415bae0 2024-05-09T04:35:29.939 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.938+0000 7f0d2a600640 1 --2- 172.21.15.145:0/2516133763 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d2415fde0 0x7f0d240cfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:29.939 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.938+0000 7f0d29200640 1 -- 172.21.15.145:0/2516133763 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d2415fde0 msgr2=0x7f0d240cfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:29.939 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.938+0000 7f0d29200640 1 --2- 172.21.15.145:0/2516133763 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d2415fde0 0x7f0d240cfe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:29.939 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.939+0000 7f0d29200640 1 -- 172.21.15.145:0/2516133763 >> v1:172.21.15.145:6789/0 conn(0x7f0d2415bae0 legacy=0x7f0d240721b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:29.939 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.939+0000 7f0d29200640 1 -- 172.21.15.145:0/2516133763 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d24160560 con 0x7f0d2415bec0 2024-05-09T04:35:29.940 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.940+0000 7f0d29200640 1 --2- 172.21.15.145:0/2516133763 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d2415bec0 0x7f0d2415f8a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f0d100097a0 tx=0x7f0d1002a0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b60c3b6161eed69c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:29.940 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.940+0000 7f0d23e00640 1 -- 172.21.15.145:0/2516133763 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d10037070 con 0x7f0d2415bec0 2024-05-09T04:35:29.940 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.940+0000 7f0d23e00640 1 -- 172.21.15.145:0/2516133763 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d1002ae40 con 0x7f0d2415bec0 2024-05-09T04:35:29.940 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.940+0000 7f0d23e00640 1 -- 172.21.15.145:0/2516133763 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d10032640 con 0x7f0d2415bec0 2024-05-09T04:35:29.941 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.940+0000 7f0d2b000640 1 -- 172.21.15.145:0/2516133763 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d2415bec0 msgr2=0x7f0d2415f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:29.941 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.940+0000 7f0d2b000640 1 --2- 172.21.15.145:0/2516133763 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d2415bec0 0x7f0d2415f8a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f0d100097a0 tx=0x7f0d1002a0f0 comp rx=0 tx=0).stop 2024-05-09T04:35:29.941 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.941+0000 7f0d2b000640 1 -- 172.21.15.145:0/2516133763 shutdown_connections 2024-05-09T04:35:29.941 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.941+0000 7f0d2b000640 1 --2- 172.21.15.145:0/2516133763 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d2415fde0 0x7f0d240cfe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:29.941 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.941+0000 7f0d2b000640 1 --2- 172.21.15.145:0/2516133763 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d2415bec0 0x7f0d2415f8a0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:29.941 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.941+0000 7f0d2b000640 1 -- 172.21.15.145:0/2516133763 >> 172.21.15.145:0/2516133763 conn(0x7f0d2406c8a0 msgr2=0x7f0d2406ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:29.941 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.941+0000 7f0d2b000640 1 -- 172.21.15.145:0/2516133763 shutdown_connections 2024-05-09T04:35:29.941 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.941+0000 7f0d2b000640 1 -- 172.21.15.145:0/2516133763 wait complete. 2024-05-09T04:35:29.942 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.942+0000 7f0d2b000640 1 Processor -- start 2024-05-09T04:35:29.942 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.942+0000 7f0d2b000640 1 -- start start 2024-05-09T04:35:29.943 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.943+0000 7f0d2b000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d2415bae0 0x7f0d2415dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:29.943 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.943+0000 7f0d2b000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d2415bec0 0x7f0d24157360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:29.943 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.943+0000 7f0d2b000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d2415fde0 0x7f0d241578a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:29.943 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.943+0000 7f0d2b000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0d240ae050 con 0x7f0d2415bae0 2024-05-09T04:35:29.944 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.943+0000 7f0d2b000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0d240aded0 con 0x7f0d2415fde0 2024-05-09T04:35:29.944 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.943+0000 7f0d2b000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0d240d0370 con 0x7f0d2415bec0 2024-05-09T04:35:29.944 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.944+0000 7f0d29c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d2415bae0 0x7f0d2415dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:29.944 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.944+0000 7f0d29c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d2415bae0 0x7f0d2415dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:52970/0 (socket says 172.21.15.145:52970) 2024-05-09T04:35:29.944 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.944+0000 7f0d29c00640 1 -- 172.21.15.145:0/1734009833 learned_addr learned my addr 172.21.15.145:0/1734009833 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:29.945 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.944+0000 7f0d2a600640 1 --2- 172.21.15.145:0/1734009833 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d2415fde0 0x7f0d241578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:29.945 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.944+0000 7f0d29200640 1 --2- 172.21.15.145:0/1734009833 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d2415bec0 0x7f0d24157360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:29.945 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.944+0000 7f0d29c00640 1 -- 172.21.15.145:0/1734009833 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d2415bec0 msgr2=0x7f0d24157360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:29.945 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.944+0000 7f0d29c00640 1 --2- 172.21.15.145:0/1734009833 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d2415bec0 0x7f0d24157360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:29.945 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.944+0000 7f0d29c00640 1 -- 172.21.15.145:0/1734009833 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d2415fde0 msgr2=0x7f0d241578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:29.945 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.944+0000 7f0d29c00640 1 --2- 172.21.15.145:0/1734009833 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d2415fde0 0x7f0d241578a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:29.945 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.944+0000 7f0d29c00640 1 -- 172.21.15.145:0/1734009833 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d241f5c20 con 0x7f0d2415bae0 2024-05-09T04:35:29.945 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.944+0000 7f0d29c00640 1 --2- 172.21.15.145:0/1734009833 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d2415bae0 0x7f0d2415dbd0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f0d1800c530 tx=0x7f0d1800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:29.945 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.945+0000 7f0d21600640 1 -- 172.21.15.145:0/1734009833 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d18017070 con 0x7f0d2415bae0 2024-05-09T04:35:29.946 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.945+0000 7f0d21600640 1 -- 172.21.15.145:0/1734009833 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d18005ce0 con 0x7f0d2415bae0 2024-05-09T04:35:29.946 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.945+0000 7f0d21600640 1 -- 172.21.15.145:0/1734009833 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d18007270 con 0x7f0d2415bae0 2024-05-09T04:35:29.946 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.945+0000 7f0d2b000640 1 -- 172.21.15.145:0/1734009833 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d2415c370 con 0x7f0d2415bae0 2024-05-09T04:35:29.946 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.945+0000 7f0d2b000640 1 -- 172.21.15.145:0/1734009833 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d2415c930 con 0x7f0d2415bae0 2024-05-09T04:35:29.946 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.946+0000 7f0d2b000640 1 -- 172.21.15.145:0/1734009833 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d2406a310 con 0x7f0d2415bae0 2024-05-09T04:35:29.947 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.947+0000 7f0d21600640 1 -- 172.21.15.145:0/1734009833 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0d1801a070 con 0x7f0d2415bae0 2024-05-09T04:35:29.948 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.948+0000 7f0d21600640 1 --2- 172.21.15.145:0/1734009833 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f0d040762d0 0x7f0d04078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:29.948 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.949+0000 7f0d21600640 1 -- 172.21.15.145:0/1734009833 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f0d18096850 con 0x7f0d2415bae0 2024-05-09T04:35:29.948 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.949+0000 7f0d29200640 1 --2- 172.21.15.145:0/1734009833 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f0d040762d0 0x7f0d04078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:29.949 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.949+0000 7f0d29200640 1 --2- 172.21.15.145:0/1734009833 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f0d040762d0 0x7f0d04078790 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0d10034000 tx=0x7f0d1002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:29.952 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:29.953+0000 7f0d21600640 1 -- 172.21.15.145:0/1734009833 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0d1805fac0 con 0x7f0d2415bae0 2024-05-09T04:35:30.200 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.200+0000 7f0d2b000640 1 -- 172.21.15.145:0/1734009833 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0d24070360 con 0x7f0d2415bae0 2024-05-09T04:35:30.201 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.201+0000 7f0d21600640 1 -- 172.21.15.145:0/1734009833 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5187 (secure 0 0 0) 0x7f0d18063dc0 con 0x7f0d2415bae0 2024-05-09T04:35:30.201 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:35:30.201 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4661,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6834/1033874179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":1033874179},{"type":"v1","addr":"172.21.15.192:6834","nonce":1033874179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4664,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1787962921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6833","nonce":1787962921},{"type":"v1","addr":"172.21.15.192:6835","nonce":1787962921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5313,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6836/65063948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":65063948},{"type":"v1","addr":"172.21.15.145:6836","nonce":65063948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T04:35:23.667303+0000","modified":"2024-05-09T04:35:26.750569+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5316},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5316":{"gid":5316,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.145:6837/2593908438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6835","nonce":2593908438},{"type":"v1","addr":"172.21.15.145:6837","nonce":2593908438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5316,"qdb_cluster":[5316]},"id":3}]} 2024-05-09T04:35:30.202 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 27 2024-05-09T04:35:30.204 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.204+0000 7f0d2b000640 1 -- 172.21.15.145:0/1734009833 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f0d040762d0 msgr2=0x7f0d04078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:30.204 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.204+0000 7f0d2b000640 1 --2- 172.21.15.145:0/1734009833 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f0d040762d0 0x7f0d04078790 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0d10034000 tx=0x7f0d1002e000 comp rx=0 tx=0).stop 2024-05-09T04:35:30.204 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.204+0000 7f0d2b000640 1 -- 172.21.15.145:0/1734009833 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d2415bae0 msgr2=0x7f0d2415dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:30.204 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.204+0000 7f0d2b000640 1 --2- 172.21.15.145:0/1734009833 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d2415bae0 0x7f0d2415dbd0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f0d1800c530 tx=0x7f0d1800c9f0 comp rx=0 tx=0).stop 2024-05-09T04:35:30.204 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.204+0000 7f0d2b000640 1 -- 172.21.15.145:0/1734009833 shutdown_connections 2024-05-09T04:35:30.205 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.204+0000 7f0d2b000640 1 --2- 172.21.15.145:0/1734009833 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f0d040762d0 0x7f0d04078790 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:30.205 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.204+0000 7f0d2b000640 1 --2- 172.21.15.145:0/1734009833 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d2415fde0 0x7f0d241578a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:30.205 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.204+0000 7f0d2b000640 1 --2- 172.21.15.145:0/1734009833 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d2415bec0 0x7f0d24157360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:30.205 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.204+0000 7f0d2b000640 1 --2- 172.21.15.145:0/1734009833 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0d2415bae0 0x7f0d2415dbd0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:30.205 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.204+0000 7f0d2b000640 1 -- 172.21.15.145:0/1734009833 >> 172.21.15.145:0/1734009833 conn(0x7f0d2406c8a0 msgr2=0x7f0d240b0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:30.205 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.205+0000 7f0d2b000640 1 -- 172.21.15.145:0/1734009833 shutdown_connections 2024-05-09T04:35:30.205 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.205+0000 7f0d2b000640 1 -- 172.21.15.145:0/1734009833 wait complete. 2024-05-09T04:35:30.626 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-09T04:35:23.667303+0000', 'modified': '2024-05-09T04:35:26.750569+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5316}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5316': {'gid': 5316, 'name': 'c', 'rank': 0, 'incarnation': 23, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.145:6837/2593908438', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.145:6835', 'nonce': 2593908438}, {'type': 'v1', 'addr': '172.21.15.145:6837', 'nonce': 2593908438}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5316, 'qdb_cluster': [5316]} 2024-05-09T04:35:30.626 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-09T04:35:30.626 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-05-09T04:35:30.837 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.837+0000 7f11ee600640 1 Processor -- start 2024-05-09T04:35:30.837 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.837+0000 7f11ee600640 1 -- start start 2024-05-09T04:35:30.839 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.839+0000 7f11ee600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f11e814d1a0 0x7f11e80cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:30.839 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.839+0000 7f11ee600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11e806edb0 con 0x7f11e8164af0 2024-05-09T04:35:30.839 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.839+0000 7f11ee600640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11e806f020 con 0x7f11e814cdc0 2024-05-09T04:35:30.839 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.839+0000 7f11ee600640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f11e8165640 con 0x7f11e814d1a0 2024-05-09T04:35:30.839 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.839+0000 7f11e7e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f11e814d1a0 0x7f11e80cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:30.839 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.839+0000 7f11e7e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f11e814d1a0 0x7f11e80cddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:56252/0 (socket says 172.21.15.145:56252) 2024-05-09T04:35:30.839 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.839+0000 7f11e7e00640 1 -- 172.21.15.145:0/28152412 learned_addr learned my addr 172.21.15.145:0/28152412 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:30.840 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.840+0000 7f11e7e00640 1 -- 172.21.15.145:0/28152412 >> v1:172.21.15.192:6789/0 conn(0x7f11e814cdc0 legacy=0x7f11e8164f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:30.840 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.840+0000 7f11e7e00640 1 -- 172.21.15.145:0/28152412 >> v1:172.21.15.145:6789/0 conn(0x7f11e8164af0 legacy=0x7f11e80b47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:30.840 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.840+0000 7f11e7e00640 1 -- 172.21.15.145:0/28152412 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11e806f300 con 0x7f11e814d1a0 2024-05-09T04:35:30.840 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.840+0000 7f11e6a00640 1 -- 172.21.15.145:0/28152412 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 96674754 0 0) 0x7f11e806edb0 con 0x7f11e8164af0 2024-05-09T04:35:30.841 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.841+0000 7f11e7e00640 1 --2- 172.21.15.145:0/28152412 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f11e814d1a0 0x7f11e80cddf0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f11e0009e50 tx=0x7f11e002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=898046db63592689 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:30.841 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.841+0000 7f11e6a00640 1 -- 172.21.15.145:0/28152412 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e0036070 con 0x7f11e814d1a0 2024-05-09T04:35:30.841 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.841+0000 7f11e6a00640 1 -- 172.21.15.145:0/28152412 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11e0031600 con 0x7f11e814d1a0 2024-05-09T04:35:30.841 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.841+0000 7f11e6a00640 1 -- 172.21.15.145:0/28152412 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e00319d0 con 0x7f11e814d1a0 2024-05-09T04:35:30.842 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.842+0000 7f11ee600640 1 -- 172.21.15.145:0/28152412 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f11e814d1a0 msgr2=0x7f11e80cddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:30.842 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.842+0000 7f11ee600640 1 --2- 172.21.15.145:0/28152412 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f11e814d1a0 0x7f11e80cddf0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f11e0009e50 tx=0x7f11e002a680 comp rx=0 tx=0).stop 2024-05-09T04:35:30.842 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.842+0000 7f11ee600640 1 -- 172.21.15.145:0/28152412 shutdown_connections 2024-05-09T04:35:30.842 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.842+0000 7f11ee600640 1 --2- 172.21.15.145:0/28152412 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f11e814d1a0 0x7f11e80cddf0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:30.842 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.842+0000 7f11ee600640 1 -- 172.21.15.145:0/28152412 >> 172.21.15.145:0/28152412 conn(0x7f11e806c8d0 msgr2=0x7f11e806cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:30.842 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.842+0000 7f11ee600640 1 -- 172.21.15.145:0/28152412 shutdown_connections 2024-05-09T04:35:30.842 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.842+0000 7f11ee600640 1 -- 172.21.15.145:0/28152412 wait complete. 2024-05-09T04:35:30.843 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.844+0000 7f11ee600640 1 Processor -- start 2024-05-09T04:35:30.843 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.844+0000 7f11ee600640 1 -- start start 2024-05-09T04:35:30.845 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.845+0000 7f11ee600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f11e814cdc0 0x7f11e813cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:30.845 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.845+0000 7f11ee600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11e814d1a0 0x7f11e8162c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:30.845 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.845+0000 7f11ee600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f11e8164af0 0x7f11e8073500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:30.845 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.845+0000 7f11ee600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f11e80b52c0 con 0x7f11e8164af0 2024-05-09T04:35:30.845 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.845+0000 7f11ee600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f11e80b5140 con 0x7f11e814d1a0 2024-05-09T04:35:30.845 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.845+0000 7f11ee600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f11e80ce330 con 0x7f11e814cdc0 2024-05-09T04:35:30.845 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.845+0000 7f11e7e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f11e8164af0 0x7f11e8073500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:30.845 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.845+0000 7f11e7e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f11e8164af0 0x7f11e8073500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:56266/0 (socket says 172.21.15.145:56266) 2024-05-09T04:35:30.845 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.845+0000 7f11e7e00640 1 -- 172.21.15.145:0/1254607197 learned_addr learned my addr 172.21.15.145:0/1254607197 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:30.846 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.845+0000 7f11dfe00640 1 --2- 172.21.15.145:0/1254607197 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11e814d1a0 0x7f11e8162c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:30.846 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.846+0000 7f11e7e00640 1 -- 172.21.15.145:0/1254607197 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f11e814cdc0 msgr2=0x7f11e813cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:30.847 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.846+0000 7f11e7e00640 1 --2- 172.21.15.145:0/1254607197 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f11e814cdc0 0x7f11e813cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:30.847 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.846+0000 7f11e7e00640 1 -- 172.21.15.145:0/1254607197 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11e814d1a0 msgr2=0x7f11e8162c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:30.847 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.846+0000 7f11e7e00640 1 --2- 172.21.15.145:0/1254607197 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11e814d1a0 0x7f11e8162c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:30.847 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.846+0000 7f11e7e00640 1 -- 172.21.15.145:0/1254607197 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11e80acb40 con 0x7f11e8164af0 2024-05-09T04:35:30.847 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.846+0000 7f11dfe00640 1 --2- 172.21.15.145:0/1254607197 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11e814d1a0 0x7f11e8162c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:35:30.847 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.846+0000 7f11e7e00640 1 --2- 172.21.15.145:0/1254607197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f11e8164af0 0x7f11e8073500 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f11e0009e50 tx=0x7f11e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:30.847 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.846+0000 7f11df400640 1 -- 172.21.15.145:0/1254607197 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e0041020 con 0x7f11e8164af0 2024-05-09T04:35:30.847 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.846+0000 7f11df400640 1 -- 172.21.15.145:0/1254607197 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11e0007780 con 0x7f11e8164af0 2024-05-09T04:35:30.847 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.846+0000 7f11ee600640 1 -- 172.21.15.145:0/1254607197 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11e80ace20 con 0x7f11e8164af0 2024-05-09T04:35:30.847 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.846+0000 7f11df400640 1 -- 172.21.15.145:0/1254607197 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e0036040 con 0x7f11e8164af0 2024-05-09T04:35:30.847 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.846+0000 7f11ee600640 1 -- 172.21.15.145:0/1254607197 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11e80ad6c0 con 0x7f11e8164af0 2024-05-09T04:35:30.849 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.850+0000 7f11df400640 1 -- 172.21.15.145:0/1254607197 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f11e0044020 con 0x7f11e8164af0 2024-05-09T04:35:30.850 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.851+0000 7f11df400640 1 --2- 172.21.15.145:0/1254607197 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f11c40762d0 0x7f11c4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:30.851 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.851+0000 7f11df400640 1 -- 172.21.15.145:0/1254607197 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f11e00b8400 con 0x7f11e8164af0 2024-05-09T04:35:30.851 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.851+0000 7f11f1580740 1 Processor -- start 2024-05-09T04:35:30.851 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.851+0000 7f11f1580740 1 -- start start 2024-05-09T04:35:30.851 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.851+0000 7f11e7400640 1 --2- 172.21.15.145:0/1254607197 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f11c40762d0 0x7f11c4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:30.853 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.852+0000 7f11e7400640 1 --2- 172.21.15.145:0/1254607197 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f11c40762d0 0x7f11c4078790 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f11d8003cc0 tx=0x7f11d80079b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:30.853 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.852+0000 7f11f1580740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b0f801840 0x563b0f801c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:30.853 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.852+0000 7f11f1580740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563b0f7fc6d0 0x563b0f800440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:30.853 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.852+0000 7f11f1580740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x563b0f800980 0x563b0f800dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:30.853 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.852+0000 7f11f1580740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x563b0f765790 con 0x563b0f7fc6d0 2024-05-09T04:35:30.853 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.852+0000 7f11f1580740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x563b0f70af20 con 0x563b0f801840 2024-05-09T04:35:30.853 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.852+0000 7f11f1580740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x563b0f761750 con 0x563b0f800980 2024-05-09T04:35:30.853 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.852+0000 7f11dfe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563b0f7fc6d0 0x563b0f800440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:30.853 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.852+0000 7f11dfe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563b0f7fc6d0 0x563b0f800440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:56276/0 (socket says 172.21.15.145:56276) 2024-05-09T04:35:30.853 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.852+0000 7f11dfe00640 1 -- 172.21.15.145:0/3622454824 learned_addr learned my addr 172.21.15.145:0/3622454824 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:30.853 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.852+0000 7f11e7400640 1 --2- 172.21.15.145:0/3622454824 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b0f801840 0x563b0f801c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:30.853 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.853+0000 7f11e7e00640 1 --2- 172.21.15.145:0/3622454824 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x563b0f800980 0x563b0f800dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:30.854 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.853+0000 7f11dfe00640 1 -- 172.21.15.145:0/3622454824 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x563b0f800980 msgr2=0x563b0f800dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:30.854 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.853+0000 7f11dfe00640 1 --2- 172.21.15.145:0/3622454824 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x563b0f800980 0x563b0f800dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:30.854 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.853+0000 7f11dfe00640 1 -- 172.21.15.145:0/3622454824 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b0f801840 msgr2=0x563b0f801c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:30.854 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.853+0000 7f11dfe00640 1 --2- 172.21.15.145:0/3622454824 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b0f801840 0x563b0f801c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:30.854 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.853+0000 7f11dfe00640 1 -- 172.21.15.145:0/3622454824 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b0f4772d0 con 0x563b0f7fc6d0 2024-05-09T04:35:30.854 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.853+0000 7f11dfe00640 1 --2- 172.21.15.145:0/3622454824 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563b0f7fc6d0 0x563b0f800440 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f11d00016a0 tx=0x7f11d0001b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b51da62d869e189f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:30.854 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.853+0000 7f11d7e00640 1 -- 172.21.15.145:0/3622454824 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d0018070 con 0x563b0f7fc6d0 2024-05-09T04:35:30.854 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.853+0000 7f11d7e00640 1 -- 172.21.15.145:0/3622454824 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11d000d730 con 0x563b0f7fc6d0 2024-05-09T04:35:30.854 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.853+0000 7f11d7e00640 1 -- 172.21.15.145:0/3622454824 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d000db20 con 0x563b0f7fc6d0 2024-05-09T04:35:30.858 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.854+0000 7f11f1580740 1 -- 172.21.15.145:0/3622454824 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563b0f7fc6d0 msgr2=0x563b0f800440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:30.858 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.854+0000 7f11f1580740 1 --2- 172.21.15.145:0/3622454824 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563b0f7fc6d0 0x563b0f800440 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f11d00016a0 tx=0x7f11d0001b60 comp rx=0 tx=0).stop 2024-05-09T04:35:30.858 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.854+0000 7f11f1580740 1 -- 172.21.15.145:0/3622454824 shutdown_connections 2024-05-09T04:35:30.858 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.854+0000 7f11f1580740 1 --2- 172.21.15.145:0/3622454824 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x563b0f800980 0x563b0f800dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:30.858 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.854+0000 7f11f1580740 1 --2- 172.21.15.145:0/3622454824 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563b0f7fc6d0 0x563b0f800440 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:30.858 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.854+0000 7f11f1580740 1 --2- 172.21.15.145:0/3622454824 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b0f801840 0x563b0f801c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:30.858 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.854+0000 7f11f1580740 1 -- 172.21.15.145:0/3622454824 >> 172.21.15.145:0/3622454824 conn(0x563b0f7fe940 msgr2=0x563b0f847400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:30.858 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.854+0000 7f11f1580740 1 -- 172.21.15.145:0/3622454824 shutdown_connections 2024-05-09T04:35:30.858 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.854+0000 7f11f1580740 1 -- 172.21.15.145:0/3622454824 wait complete. 2024-05-09T04:35:30.858 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.855+0000 7f11f1580740 1 -- start start 2024-05-09T04:35:30.858 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.855+0000 7f11f1580740 1 Processor -- start 2024-05-09T04:35:30.858 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.856+0000 7f11f1580740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x563b0f800980 0x563b0f84c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:30.858 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.856+0000 7f11f1580740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b0f801840 0x563b0f84cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:30.859 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.856+0000 7f11f1580740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563b0f84eb40 0x563b0f85aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:30.859 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.856+0000 7f11f1580740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x563b0f82d3d0 con 0x563b0f84eb40 2024-05-09T04:35:30.859 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.856+0000 7f11f1580740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x563b0f71e9c0 con 0x563b0f801840 2024-05-09T04:35:30.859 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.856+0000 7f11f1580740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x563b0f82c850 con 0x563b0f800980 2024-05-09T04:35:30.859 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.856+0000 7f11f1580740 10 client.0 _ll_register_callbacks cb 0x563b0f6eb120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:35:30.859 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.856+0000 7f11e7e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563b0f84eb40 0x563b0f85aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:30.859 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.856+0000 7f11e7e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563b0f84eb40 0x563b0f85aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:56288/0 (socket says 172.21.15.145:56288) 2024-05-09T04:35:30.859 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.856+0000 7f11e7e00640 1 -- 172.21.15.145:0/2343747668 learned_addr learned my addr 172.21.15.145:0/2343747668 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:30.859 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.857+0000 7f11dfe00640 1 --2- 172.21.15.145:0/2343747668 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b0f801840 0x563b0f84cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:30.859 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.857+0000 7f11e7e00640 1 -- 172.21.15.145:0/2343747668 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x563b0f800980 msgr2=0x563b0f84c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:30.859 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.857+0000 7f11e7e00640 1 --2- 172.21.15.145:0/2343747668 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x563b0f800980 0x563b0f84c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:30.859 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.857+0000 7f11e7e00640 1 -- 172.21.15.145:0/2343747668 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b0f801840 msgr2=0x563b0f84cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:30.859 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.857+0000 7f11e7e00640 1 --2- 172.21.15.145:0/2343747668 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b0f801840 0x563b0f84cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:30.859 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.857+0000 7f11e7e00640 1 -- 172.21.15.145:0/2343747668 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b0f5416b0 con 0x563b0f84eb40 2024-05-09T04:35:30.860 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.857+0000 7f11dfe00640 1 --2- 172.21.15.145:0/2343747668 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b0f801840 0x563b0f84cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:35:30.860 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.857+0000 7f11e7e00640 1 --2- 172.21.15.145:0/2343747668 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563b0f84eb40 0x563b0f85aa50 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f11e0081550 tx=0x7f11e0080df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96a9e9427f7cdacf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:30.860 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.857+0000 7f11d4c00640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-05-09T04:35:30.860 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.857+0000 7f11d4c00640 1 -- 172.21.15.145:0/2343747668 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e008a100 con 0x563b0f84eb40 2024-05-09T04:35:30.860 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.857+0000 7f11d4c00640 1 -- 172.21.15.145:0/2343747668 <== mon.0 v2:172.21.15.145:3300/0 2 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f11e0007b70 con 0x563b0f84eb40 2024-05-09T04:35:30.860 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.857+0000 7f11f1580740 1 -- 172.21.15.145:0/2343747668 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11d000d730 con 0x563b0f84eb40 2024-05-09T04:35:30.860 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.857+0000 7f11d4c00640 1 -- 172.21.15.145:0/2343747668 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f11e00b6050 con 0x563b0f84eb40 2024-05-09T04:35:30.860 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.857+0000 7f11f1580740 10 client.5466 fetch_fsmap learned FSMap version 27 2024-05-09T04:35:30.860 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.857+0000 7f11f1580740 1 -- 172.21.15.145:0/2343747668 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) -- 0x563b0f84d530 con 0x563b0f84eb40 2024-05-09T04:35:30.860 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.858+0000 7f11d4c00640 1 -- 172.21.15.145:0/2343747668 <== mon.0 v2:172.21.15.145:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11d000d730 con 0x563b0f84eb40 2024-05-09T04:35:30.860 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.858+0000 7f11d4c00640 1 -- 172.21.15.145:0/2343747668 <== mon.0 v2:172.21.15.145:3300/0 5 ==== fsmap(e 27) ==== 3494+0+0 (secure 0 0 0) 0x7f11e0080820 con 0x563b0f84eb40 2024-05-09T04:35:30.860 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.858+0000 7f11f1580740 10 client.5466 fetch_fsmap finished waiting for FSMap version 27 2024-05-09T04:35:30.861 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.858+0000 7f11f1580740 10 client.5466 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-09T04:35:30.861 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.858+0000 7f11f1580740 20 client.5466 populate_metadata read hostname 'smithi145' 2024-05-09T04:35:30.861 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.858+0000 7f11f1580740 1 --2- 172.21.15.145:0/2343747668 >> [v2:172.21.15.145:6835/2593908438,v1:172.21.15.145:6837/2593908438] conn(0x563b0f823c20 0x563b0f8573e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:30.861 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.858+0000 7f11f1580740 4 client.5466 mds_command: new command op to 5316 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:35:30.861 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.858+0000 7f11f1580740 1 -- 172.21.15.145:0/2343747668 --> [v2:172.21.15.145:6835/2593908438,v1:172.21.15.145:6837/2593908438] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b0f7733a0 con 0x563b0f823c20 2024-05-09T04:35:30.861 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.858+0000 7f11e7400640 1 --2- 172.21.15.145:0/2343747668 >> [v2:172.21.15.145:6835/2593908438,v1:172.21.15.145:6837/2593908438] conn(0x563b0f823c20 0x563b0f8573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:30.862 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.859+0000 7f11e7400640 1 --2- 172.21.15.145:0/2343747668 >> [v2:172.21.15.145:6835/2593908438,v1:172.21.15.145:6837/2593908438] conn(0x563b0f823c20 0x563b0f8573e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2275ce99bec6eb84 server_cookie=f0f11e410831cb12 in_seq=0 out_seq=0 2024-05-09T04:35:30.862 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.859+0000 7f11d4c00640 10 client.5466 ms_handle_connect on v2:172.21.15.145:6835/2593908438 2024-05-09T04:35:30.862 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.860+0000 7f11d4c00640 1 -- 172.21.15.145:0/2343747668 <== mds.0 v2:172.21.15.145:6835/2593908438 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563b0f7733a0 con 0x563b0f823c20 2024-05-09T04:35:30.863 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.860+0000 7f11d4c00640 10 client.5466 handle_command_reply: tid=0 2024-05-09T04:35:30.863 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.860+0000 7f11f1580740 1 client.5466 shutdown 2024-05-09T04:35:30.863 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.860+0000 7f11f1580740 1 -- 172.21.15.145:0/2343747668 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563b0f84eb40 msgr2=0x563b0f85aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:30.863 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.860+0000 7f11f1580740 1 --2- 172.21.15.145:0/2343747668 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563b0f84eb40 0x563b0f85aa50 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f11e0081550 tx=0x7f11e0080df0 comp rx=0 tx=0).stop 2024-05-09T04:35:30.863 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.860+0000 7f11f1580740 1 -- 172.21.15.145:0/2343747668 shutdown_connections 2024-05-09T04:35:30.863 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.860+0000 7f11f1580740 1 --2- 172.21.15.145:0/2343747668 >> [v2:172.21.15.145:6835/2593908438,v1:172.21.15.145:6837/2593908438] conn(0x563b0f823c20 0x563b0f8573e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:30.863 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.860+0000 7f11f1580740 1 --2- 172.21.15.145:0/2343747668 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563b0f84eb40 0x563b0f85aa50 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:30.863 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.860+0000 7f11f1580740 1 --2- 172.21.15.145:0/2343747668 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b0f801840 0x563b0f84cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:30.863 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.860+0000 7f11f1580740 1 --2- 172.21.15.145:0/2343747668 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x563b0f800980 0x563b0f84c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:30.863 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.860+0000 7f11d4c00640 0 client.5466 ms_handle_reset on v2:172.21.15.145:6835/2593908438 2024-05-09T04:35:30.863 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.860+0000 7f11f1580740 1 -- 172.21.15.145:0/2343747668 >> 172.21.15.145:0/2343747668 conn(0x563b0f7fc6d0 msgr2=0x563b0f802330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:30.863 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.860+0000 7f11f1580740 1 -- 172.21.15.145:0/2343747668 shutdown_connections 2024-05-09T04:35:30.863 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.860+0000 7f11f1580740 1 -- 172.21.15.145:0/2343747668 wait complete. 2024-05-09T04:35:30.863 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.860+0000 7f11f1580740 20 client.5466 trim_cache size 0 max 16384 2024-05-09T04:35:30.880 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.880+0000 7f11f1580740 1 Processor -- start 2024-05-09T04:35:30.880 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.881+0000 7f11f1580740 1 -- start start 2024-05-09T04:35:30.882 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.881+0000 7f11f1580740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563b0f800980 0x563b0f801c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:30.882 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.881+0000 7f11f1580740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b0f801840 0x563b0f802160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:30.882 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.881+0000 7f11f1580740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x563b0f84eb40 0x563b0f824340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:30.882 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.881+0000 7f11f1580740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f11e8204d90 con 0x563b0f800980 2024-05-09T04:35:30.882 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.881+0000 7f11f1580740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f11e80aa8d0 con 0x563b0f801840 2024-05-09T04:35:30.882 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.881+0000 7f11f1580740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f11e80aaf20 con 0x563b0f84eb40 2024-05-09T04:35:30.882 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.882+0000 7f11e7400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563b0f800980 0x563b0f801c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:30.882 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.882+0000 7f11e7400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563b0f800980 0x563b0f801c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:56294/0 (socket says 172.21.15.145:56294) 2024-05-09T04:35:30.882 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.882+0000 7f11e7400640 1 -- 172.21.15.145:0/1049943283 learned_addr learned my addr 172.21.15.145:0/1049943283 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:30.882 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.882+0000 7f11e7400640 1 -- 172.21.15.145:0/1049943283 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x563b0f84eb40 msgr2=0x563b0f824340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:30.882 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.882+0000 7f11e7400640 1 --2- 172.21.15.145:0/1049943283 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x563b0f84eb40 0x563b0f824340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:30.883 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.882+0000 7f11e7400640 1 -- 172.21.15.145:0/1049943283 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b0f801840 msgr2=0x563b0f802160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:30.883 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.882+0000 7f11e7400640 1 --2- 172.21.15.145:0/1049943283 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b0f801840 0x563b0f802160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:30.883 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.882+0000 7f11e7400640 1 -- 172.21.15.145:0/1049943283 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b0f85aca0 con 0x563b0f800980 2024-05-09T04:35:30.883 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.882+0000 7f11e7400640 1 --2- 172.21.15.145:0/1049943283 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563b0f800980 0x563b0f801c20 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f11d8014980 tx=0x7f11d800f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8323974d9eb8b6f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:30.884 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.883+0000 7f11d7400640 1 -- 172.21.15.145:0/1049943283 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d802a070 con 0x563b0f800980 2024-05-09T04:35:30.884 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.883+0000 7f11d7400640 1 -- 172.21.15.145:0/1049943283 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11d80139b0 con 0x563b0f800980 2024-05-09T04:35:30.884 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.883+0000 7f11d7400640 1 -- 172.21.15.145:0/1049943283 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d8013d60 con 0x563b0f800980 2024-05-09T04:35:30.884 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.883+0000 7f11f1580740 1 -- 172.21.15.145:0/1049943283 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563b0f800980 msgr2=0x563b0f801c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:30.884 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.883+0000 7f11f1580740 1 --2- 172.21.15.145:0/1049943283 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563b0f800980 0x563b0f801c20 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f11d8014980 tx=0x7f11d800f000 comp rx=0 tx=0).stop 2024-05-09T04:35:30.884 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.883+0000 7f11f1580740 1 -- 172.21.15.145:0/1049943283 shutdown_connections 2024-05-09T04:35:30.885 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.883+0000 7f11f1580740 1 --2- 172.21.15.145:0/1049943283 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x563b0f84eb40 0x563b0f824340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:30.885 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.883+0000 7f11f1580740 1 --2- 172.21.15.145:0/1049943283 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b0f801840 0x563b0f802160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:30.885 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.883+0000 7f11f1580740 1 --2- 172.21.15.145:0/1049943283 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563b0f800980 0x563b0f801c20 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:30.885 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.883+0000 7f11f1580740 1 -- 172.21.15.145:0/1049943283 >> 172.21.15.145:0/1049943283 conn(0x563b0f7fc6d0 msgr2=0x563b0f84c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:30.885 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.883+0000 7f11f1580740 1 -- 172.21.15.145:0/1049943283 shutdown_connections 2024-05-09T04:35:30.885 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.883+0000 7f11f1580740 1 -- 172.21.15.145:0/1049943283 wait complete. 2024-05-09T04:35:30.885 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.884+0000 7f11f1580740 1 -- start start 2024-05-09T04:35:30.885 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.884+0000 7f11f1580740 1 Processor -- start 2024-05-09T04:35:30.888 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.885+0000 7f11f1580740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563b0f800980 0x563b0f801c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:30.889 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.885+0000 7f11f1580740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x563b0f801840 0x563b0f802160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:30.889 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.885+0000 7f11f1580740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b0f84eb40 0x563b0f859760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:30.889 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.885+0000 7f11f1580740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x563b0f849070 con 0x563b0f800980 2024-05-09T04:35:30.889 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.885+0000 7f11f1580740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x563b0f848b80 con 0x563b0f84eb40 2024-05-09T04:35:30.889 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.885+0000 7f11f1580740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x563b0f8496c0 con 0x563b0f801840 2024-05-09T04:35:30.889 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.885+0000 7f11f1580740 10 client.0 _ll_register_callbacks cb 0x7f11e0009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:35:30.889 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.886+0000 7f11e7400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563b0f800980 0x563b0f801c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:30.889 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.886+0000 7f11e7400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563b0f800980 0x563b0f801c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:56302/0 (socket says 172.21.15.145:56302) 2024-05-09T04:35:30.889 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.886+0000 7f11e7400640 1 -- 172.21.15.145:0/2274725152 learned_addr learned my addr 172.21.15.145:0/2274725152 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:30.889 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.886+0000 7f11e7400640 1 -- 172.21.15.145:0/2274725152 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x563b0f801840 msgr2=0x563b0f802160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:30.889 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.886+0000 7f11e7400640 1 --2- 172.21.15.145:0/2274725152 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x563b0f801840 0x563b0f802160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:30.889 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.886+0000 7f11e7400640 1 -- 172.21.15.145:0/2274725152 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b0f84eb40 msgr2=0x563b0f859760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:30.889 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.886+0000 7f11e7400640 1 --2- 172.21.15.145:0/2274725152 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b0f84eb40 0x563b0f859760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:30.890 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.886+0000 7f11e7400640 1 -- 172.21.15.145:0/2274725152 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b0f86b0f0 con 0x563b0f800980 2024-05-09T04:35:30.890 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.886+0000 7f11e7400640 1 --2- 172.21.15.145:0/2274725152 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563b0f800980 0x563b0f801c20 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f11d8028040 tx=0x7f11e0081460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7460031ec55ed91 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:30.890 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.886+0000 7f11d4c00640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-05-09T04:35:30.890 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.887+0000 7f11d4c00640 1 -- 172.21.15.145:0/2274725152 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d8034060 con 0x563b0f800980 2024-05-09T04:35:30.890 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.887+0000 7f11d4c00640 1 -- 172.21.15.145:0/2274725152 <== mon.0 v2:172.21.15.145:3300/0 2 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f11d8021620 con 0x563b0f800980 2024-05-09T04:35:30.890 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.887+0000 7f11f1580740 1 -- 172.21.15.145:0/2274725152 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11d80139b0 con 0x563b0f800980 2024-05-09T04:35:30.890 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.887+0000 7f11d4c00640 1 -- 172.21.15.145:0/2274725152 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f11d802a070 con 0x563b0f800980 2024-05-09T04:35:30.890 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.887+0000 7f11f1580740 10 client.5472 fetch_fsmap learned FSMap version 27 2024-05-09T04:35:30.890 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.887+0000 7f11f1580740 1 -- 172.21.15.145:0/2274725152 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) -- 0x563b0f824dc0 con 0x563b0f800980 2024-05-09T04:35:30.890 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.887+0000 7f11d4c00640 1 -- 172.21.15.145:0/2274725152 <== mon.0 v2:172.21.15.145:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11d80139b0 con 0x563b0f800980 2024-05-09T04:35:30.890 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.887+0000 7f11d4c00640 1 -- 172.21.15.145:0/2274725152 <== mon.0 v2:172.21.15.145:3300/0 5 ==== fsmap(e 27) ==== 3494+0+0 (secure 0 0 0) 0x7f11d803d080 con 0x563b0f800980 2024-05-09T04:35:30.890 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.887+0000 7f11f1580740 10 client.5472 fetch_fsmap finished waiting for FSMap version 27 2024-05-09T04:35:30.890 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.887+0000 7f11f1580740 10 client.5472 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-09T04:35:30.890 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.887+0000 7f11f1580740 20 client.5472 populate_metadata read hostname 'smithi145' 2024-05-09T04:35:30.890 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.887+0000 7f11f1580740 1 --2- 172.21.15.145:0/2274725152 >> [v2:172.21.15.145:6835/2593908438,v1:172.21.15.145:6837/2593908438] conn(0x563b0f82b660 0x563b0f82ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:30.891 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.887+0000 7f11f1580740 4 client.5472 mds_command: new command op to 5316 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T04:35:30.891 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.887+0000 7f11f1580740 1 -- 172.21.15.145:0/2274725152 --> [v2:172.21.15.145:6835/2593908438,v1:172.21.15.145:6837/2593908438] -- command(tid 0: {"prefix": "status"}) -- 0x563b0f76d210 con 0x563b0f82b660 2024-05-09T04:35:30.891 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.888+0000 7f11dfe00640 1 --2- 172.21.15.145:0/2274725152 >> [v2:172.21.15.145:6835/2593908438,v1:172.21.15.145:6837/2593908438] conn(0x563b0f82b660 0x563b0f82ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:30.891 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.888+0000 7f11dfe00640 1 --2- 172.21.15.145:0/2274725152 >> [v2:172.21.15.145:6835/2593908438,v1:172.21.15.145:6837/2593908438] conn(0x563b0f82b660 0x563b0f82ba40 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5807c38f6b5256e0 server_cookie=72e1c8e2bf13410 in_seq=0 out_seq=0 2024-05-09T04:35:30.891 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.888+0000 7f11d4c00640 10 client.5472 ms_handle_connect on v2:172.21.15.145:6835/2593908438 2024-05-09T04:35:30.891 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.888+0000 7f11d4c00640 1 -- 172.21.15.145:0/2274725152 <== mds.0 v2:172.21.15.145:6835/2593908438 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x563b0f76d210 con 0x563b0f82b660 2024-05-09T04:35:30.891 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.888+0000 7f11d4c00640 10 client.5472 handle_command_reply: tid=0 2024-05-09T04:35:30.891 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.888+0000 7f11f1580740 1 client.5472 shutdown 2024-05-09T04:35:30.891 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.888+0000 7f11f1580740 1 -- 172.21.15.145:0/2274725152 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563b0f800980 msgr2=0x563b0f801c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:30.891 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.888+0000 7f11f1580740 1 --2- 172.21.15.145:0/2274725152 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563b0f800980 0x563b0f801c20 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f11d8028040 tx=0x7f11e0081460 comp rx=0 tx=0).stop 2024-05-09T04:35:30.891 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.889+0000 7f11f1580740 1 -- 172.21.15.145:0/2274725152 shutdown_connections 2024-05-09T04:35:30.891 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.889+0000 7f11f1580740 1 --2- 172.21.15.145:0/2274725152 >> [v2:172.21.15.145:6835/2593908438,v1:172.21.15.145:6837/2593908438] conn(0x563b0f82b660 0x563b0f82ba40 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:30.891 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.889+0000 7f11f1580740 1 --2- 172.21.15.145:0/2274725152 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x563b0f84eb40 0x563b0f859760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:30.891 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.889+0000 7f11f1580740 1 --2- 172.21.15.145:0/2274725152 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x563b0f801840 0x563b0f802160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:30.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.889+0000 7f11f1580740 1 --2- 172.21.15.145:0/2274725152 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x563b0f800980 0x563b0f801c20 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:30.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.889+0000 7f11d4c00640 0 client.5472 ms_handle_reset on v2:172.21.15.145:6835/2593908438 2024-05-09T04:35:30.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.889+0000 7f11f1580740 1 -- 172.21.15.145:0/2274725152 >> 172.21.15.145:0/2274725152 conn(0x563b0f7fc6d0 msgr2=0x563b0f861a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:30.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.889+0000 7f11f1580740 1 -- 172.21.15.145:0/2274725152 shutdown_connections 2024-05-09T04:35:30.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.889+0000 7f11f1580740 1 -- 172.21.15.145:0/2274725152 wait complete. 2024-05-09T04:35:30.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.889+0000 7f11f1580740 20 client.5472 trim_cache size 0 max 16384 2024-05-09T04:35:30.892 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-05-09T04:35:30.892 INFO:teuthology.orchestra.run.smithi145.stdout: "cluster_fsid": "957ee6c1-073b-44cf-9b93-9491b3fdfc1d", 2024-05-09T04:35:30.892 INFO:teuthology.orchestra.run.smithi145.stdout: "whoami": 0, 2024-05-09T04:35:30.892 INFO:teuthology.orchestra.run.smithi145.stdout: "id": 5316, 2024-05-09T04:35:30.892 INFO:teuthology.orchestra.run.smithi145.stdout: "want_state": "up:active", 2024-05-09T04:35:30.893 INFO:teuthology.orchestra.run.smithi145.stdout: "state": "up:active", 2024-05-09T04:35:30.893 INFO:teuthology.orchestra.run.smithi145.stdout: "fs_name": "cephfs", 2024-05-09T04:35:30.893 INFO:teuthology.orchestra.run.smithi145.stdout: "rank_uptime": 6.6740098889999997, 2024-05-09T04:35:30.893 INFO:teuthology.orchestra.run.smithi145.stdout: "mdsmap_epoch": 27, 2024-05-09T04:35:30.893 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_epoch": 47, 2024-05-09T04:35:30.893 INFO:teuthology.orchestra.run.smithi145.stdout: "osdmap_epoch_barrier": 47, 2024-05-09T04:35:30.893 INFO:teuthology.orchestra.run.smithi145.stdout: "uptime": 11.097380889 2024-05-09T04:35:30.893 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-05-09T04:35:30.894 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.889+0000 7f11ee600640 1 -- 172.21.15.145:0/1254607197 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f11c40762d0 msgr2=0x7f11c4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:30.894 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.889+0000 7f11ee600640 1 --2- 172.21.15.145:0/1254607197 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f11c40762d0 0x7f11c4078790 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f11d8003cc0 tx=0x7f11d80079b0 comp rx=0 tx=0).stop 2024-05-09T04:35:30.894 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.890+0000 7f11ee600640 1 -- 172.21.15.145:0/1254607197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f11e8164af0 msgr2=0x7f11e8073500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:30.894 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.890+0000 7f11ee600640 1 --2- 172.21.15.145:0/1254607197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f11e8164af0 0x7f11e8073500 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f11e0009e50 tx=0x7f11e0004a40 comp rx=0 tx=0).stop 2024-05-09T04:35:30.894 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.890+0000 7f11ee600640 1 -- 172.21.15.145:0/1254607197 shutdown_connections 2024-05-09T04:35:30.894 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.890+0000 7f11ee600640 1 --2- 172.21.15.145:0/1254607197 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f11c40762d0 0x7f11c4078790 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:30.894 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.890+0000 7f11ee600640 1 --2- 172.21.15.145:0/1254607197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f11e8164af0 0x7f11e8073500 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:30.894 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.890+0000 7f11ee600640 1 --2- 172.21.15.145:0/1254607197 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11e814d1a0 0x7f11e8162c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:30.894 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.890+0000 7f11ee600640 1 --2- 172.21.15.145:0/1254607197 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f11e814cdc0 0x7f11e813cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:30.894 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.890+0000 7f11ee600640 1 -- 172.21.15.145:0/1254607197 >> 172.21.15.145:0/1254607197 conn(0x7f11e806c8d0 msgr2=0x7f11e8165e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:30.895 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.890+0000 7f11ee600640 1 -- 172.21.15.145:0/1254607197 shutdown_connections 2024-05-09T04:35:30.895 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:30.890+0000 7f11ee600640 1 -- 172.21.15.145:0/1254607197 wait complete. 2024-05-09T04:35:30.906 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-09T04:35:31.100 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.100+0000 7f88eb800640 1 Processor -- start 2024-05-09T04:35:31.100 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.100+0000 7f88eb800640 1 -- start start 2024-05-09T04:35:31.101 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.102+0000 7f88eb800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f88e4142200 0x7f88e40cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:31.102 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.102+0000 7f88eb800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f88e40cfe10 0x7f88e40726b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:31.102 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.102+0000 7f88eb800640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88e413f340 con 0x7f88e415fbc0 2024-05-09T04:35:31.102 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.102+0000 7f88eb800640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f88e4142660 con 0x7f88e40cfe10 2024-05-09T04:35:31.102 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.102+0000 7f88eb800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f88e4072220 con 0x7f88e4142200 2024-05-09T04:35:31.103 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.102+0000 7f88e9800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f88e40cfe10 0x7f88e40726b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:31.103 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.102+0000 7f88e9800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f88e40cfe10 0x7f88e40726b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:56308/0 (socket says 172.21.15.145:56308) 2024-05-09T04:35:31.103 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.102+0000 7f88e9800640 1 -- 172.21.15.145:0/2093749800 learned_addr learned my addr 172.21.15.145:0/2093749800 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:31.103 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.103+0000 7f88e3e00640 1 --2- 172.21.15.145:0/2093749800 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f88e4142200 0x7f88e40cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:31.103 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.103+0000 7f88e9800640 1 -- 172.21.15.145:0/2093749800 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f88e4142200 msgr2=0x7f88e40cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:31.103 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.103+0000 7f88e9800640 1 --2- 172.21.15.145:0/2093749800 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f88e4142200 0x7f88e40cf8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:31.103 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.103+0000 7f88e9800640 1 -- 172.21.15.145:0/2093749800 >> v1:172.21.15.192:6789/0 conn(0x7f88e415fbc0 legacy=0x7f88e415ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:31.103 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.103+0000 7f88e9800640 1 -- 172.21.15.145:0/2093749800 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88e40d02b0 con 0x7f88e40cfe10 2024-05-09T04:35:31.103 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.103+0000 7f88e3e00640 1 --2- 172.21.15.145:0/2093749800 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f88e4142200 0x7f88e40cf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:35:31.104 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.104+0000 7f88e9800640 1 --2- 172.21.15.145:0/2093749800 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88e40cfe10 0x7f88e40726b0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f88dc009e50 tx=0x7f88dc02a5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76ee77e8684c64f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:31.105 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.104+0000 7f88e3400640 1 -- 172.21.15.145:0/2093749800 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88dc036070 con 0x7f88e40cfe10 2024-05-09T04:35:31.105 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.104+0000 7f88e3400640 1 -- 172.21.15.145:0/2093749800 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88dc031400 con 0x7f88e40cfe10 2024-05-09T04:35:31.105 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.104+0000 7f88e3400640 1 -- 172.21.15.145:0/2093749800 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88dc0317d0 con 0x7f88e40cfe10 2024-05-09T04:35:31.105 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.105+0000 7f88eb800640 1 -- 172.21.15.145:0/2093749800 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88e40cfe10 msgr2=0x7f88e40726b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:31.105 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.105+0000 7f88eb800640 1 --2- 172.21.15.145:0/2093749800 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88e40cfe10 0x7f88e40726b0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f88dc009e50 tx=0x7f88dc02a5e0 comp rx=0 tx=0).stop 2024-05-09T04:35:31.105 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.105+0000 7f88eb800640 1 -- 172.21.15.145:0/2093749800 shutdown_connections 2024-05-09T04:35:31.105 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.105+0000 7f88eb800640 1 --2- 172.21.15.145:0/2093749800 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88e40cfe10 0x7f88e40726b0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:31.105 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.105+0000 7f88eb800640 1 --2- 172.21.15.145:0/2093749800 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f88e4142200 0x7f88e40cf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:31.105 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.105+0000 7f88eb800640 1 -- 172.21.15.145:0/2093749800 >> 172.21.15.145:0/2093749800 conn(0x7f88e406c870 msgr2=0x7f88e406cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:31.105 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.105+0000 7f88eb800640 1 -- 172.21.15.145:0/2093749800 shutdown_connections 2024-05-09T04:35:31.105 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.105+0000 7f88eb800640 1 -- 172.21.15.145:0/2093749800 wait complete. 2024-05-09T04:35:31.106 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.107+0000 7f88eb800640 1 Processor -- start 2024-05-09T04:35:31.106 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.107+0000 7f88eb800640 1 -- start start 2024-05-09T04:35:31.110 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.108+0000 7f88eb800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88e40cfe10 0x7f88e4151eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:31.110 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.108+0000 7f88eb800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f88e4142200 0x7f88e41344d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:31.110 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.108+0000 7f88eb800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88e415fbc0 0x7f88e4140220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:31.110 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.108+0000 7f88eb800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f88e40d0b50 con 0x7f88e40cfe10 2024-05-09T04:35:31.110 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.108+0000 7f88eb800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f88e40d0870 con 0x7f88e415fbc0 2024-05-09T04:35:31.110 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.108+0000 7f88eb800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f88e4072bf0 con 0x7f88e4142200 2024-05-09T04:35:31.110 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.108+0000 7f88e8e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88e40cfe10 0x7f88e4151eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:31.110 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.108+0000 7f88e8e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88e40cfe10 0x7f88e4151eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:56312/0 (socket says 172.21.15.145:56312) 2024-05-09T04:35:31.111 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.108+0000 7f88e8e00640 1 -- 172.21.15.145:0/4060261396 learned_addr learned my addr 172.21.15.145:0/4060261396 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:31.111 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.108+0000 7f88e8e00640 1 -- 172.21.15.145:0/4060261396 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f88e4142200 msgr2=0x7f88e41344d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:31.111 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.108+0000 7f88e8e00640 1 --2- 172.21.15.145:0/4060261396 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f88e4142200 0x7f88e41344d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:31.111 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.108+0000 7f88e9800640 1 --2- 172.21.15.145:0/4060261396 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88e415fbc0 0x7f88e4140220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:31.111 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.108+0000 7f88e8e00640 1 -- 172.21.15.145:0/4060261396 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88e415fbc0 msgr2=0x7f88e4140220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:31.111 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.108+0000 7f88e8e00640 1 --2- 172.21.15.145:0/4060261396 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88e415fbc0 0x7f88e4140220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:31.111 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.108+0000 7f88e8e00640 1 -- 172.21.15.145:0/4060261396 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88e4134a10 con 0x7f88e40cfe10 2024-05-09T04:35:31.111 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.109+0000 7f88e8e00640 1 --2- 172.21.15.145:0/4060261396 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88e40cfe10 0x7f88e4151eb0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f88d800b3f0 tx=0x7f88d800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:31.111 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.109+0000 7f88e0c00640 1 -- 172.21.15.145:0/4060261396 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d8015020 con 0x7f88e40cfe10 2024-05-09T04:35:31.111 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.109+0000 7f88e0c00640 1 -- 172.21.15.145:0/4060261396 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88d8011830 con 0x7f88e40cfe10 2024-05-09T04:35:31.111 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.109+0000 7f88eb800640 1 -- 172.21.15.145:0/4060261396 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88e40b0090 con 0x7f88e40cfe10 2024-05-09T04:35:31.111 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.109+0000 7f88e0c00640 1 -- 172.21.15.145:0/4060261396 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d8011c20 con 0x7f88e40cfe10 2024-05-09T04:35:31.111 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.109+0000 7f88eb800640 1 -- 172.21.15.145:0/4060261396 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88e41683c0 con 0x7f88e40cfe10 2024-05-09T04:35:31.111 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.110+0000 7f88eb800640 1 -- 172.21.15.145:0/4060261396 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88e406a310 con 0x7f88e40cfe10 2024-05-09T04:35:31.112 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.112+0000 7f88e0c00640 1 -- 172.21.15.145:0/4060261396 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f88d8018020 con 0x7f88e40cfe10 2024-05-09T04:35:31.112 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.112+0000 7f88e0c00640 1 --2- 172.21.15.145:0/4060261396 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f88c40762d0 0x7f88c4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:31.112 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.113+0000 7f88e0c00640 1 -- 172.21.15.145:0/4060261396 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f88d80971b0 con 0x7f88e40cfe10 2024-05-09T04:35:31.112 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.113+0000 7f88e3e00640 1 --2- 172.21.15.145:0/4060261396 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f88c40762d0 0x7f88c4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:31.113 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.113+0000 7f88e3e00640 1 --2- 172.21.15.145:0/4060261396 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f88c40762d0 0x7f88c4078790 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f88d0006fd0 tx=0x7f88d0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:31.116 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.117+0000 7f88e0c00640 1 -- 172.21.15.145:0/4060261396 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f88d801f030 con 0x7f88e40cfe10 2024-05-09T04:35:31.351 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.351+0000 7f88eb800640 1 -- 172.21.15.145:0/4060261396 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f88e40b4070 con 0x7f88e40cfe10 2024-05-09T04:35:31.352 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.352+0000 7f88e0c00640 1 -- 172.21.15.145:0/4060261396 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+73 (secure 0 0 0) 0x7f88d8060b10 con 0x7f88e40cfe10 2024-05-09T04:35:31.352 INFO:teuthology.orchestra.run.smithi145.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-09T04:35:31.355 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.355+0000 7f88eb800640 1 -- 172.21.15.145:0/4060261396 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f88c40762d0 msgr2=0x7f88c4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:31.355 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.355+0000 7f88eb800640 1 --2- 172.21.15.145:0/4060261396 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f88c40762d0 0x7f88c4078790 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f88d0006fd0 tx=0x7f88d0008040 comp rx=0 tx=0).stop 2024-05-09T04:35:31.355 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.355+0000 7f88eb800640 1 -- 172.21.15.145:0/4060261396 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88e40cfe10 msgr2=0x7f88e4151eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:31.355 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.355+0000 7f88eb800640 1 --2- 172.21.15.145:0/4060261396 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88e40cfe10 0x7f88e4151eb0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f88d800b3f0 tx=0x7f88d800b8b0 comp rx=0 tx=0).stop 2024-05-09T04:35:31.355 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.355+0000 7f88eb800640 1 -- 172.21.15.145:0/4060261396 shutdown_connections 2024-05-09T04:35:31.355 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.355+0000 7f88eb800640 1 --2- 172.21.15.145:0/4060261396 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f88c40762d0 0x7f88c4078790 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:31.355 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.355+0000 7f88eb800640 1 --2- 172.21.15.145:0/4060261396 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88e415fbc0 0x7f88e4140220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:31.355 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.355+0000 7f88eb800640 1 --2- 172.21.15.145:0/4060261396 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f88e4142200 0x7f88e41344d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:31.356 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.355+0000 7f88eb800640 1 --2- 172.21.15.145:0/4060261396 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f88e40cfe10 0x7f88e4151eb0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:31.356 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.355+0000 7f88eb800640 1 -- 172.21.15.145:0/4060261396 >> 172.21.15.145:0/4060261396 conn(0x7f88e406c870 msgr2=0x7f88e406cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:31.356 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.355+0000 7f88eb800640 1 -- 172.21.15.145:0/4060261396 shutdown_connections 2024-05-09T04:35:31.356 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31.355+0000 7f88eb800640 1 -- 172.21.15.145:0/4060261396 wait complete. 2024-05-09T04:35:31.372 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-09T04:35:31.372 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-09T04:35:31.373 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi145.front.sepia.ceph.com 2024-05-09T04:35:31.373 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-09T04:35:31.373 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-09T04:35:31.373 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-09T04:35:31.373 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-09T04:35:31.373 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-09T04:35:31.373 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-09T04:35:31.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:35:31.373 DEBUG:teuthology.orchestra.run.smithi145:> ip addr 2024-05-09T04:35:31.421 INFO:teuthology.orchestra.run.smithi145.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T04:35:31.422 INFO:teuthology.orchestra.run.smithi145.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T04:35:31.422 INFO:teuthology.orchestra.run.smithi145.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T04:35:31.422 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-05-09T04:35:31.422 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 ::1/128 scope host 2024-05-09T04:35:31.422 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-05-09T04:35:31.422 INFO:teuthology.orchestra.run.smithi145.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T04:35:31.422 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:d9:38:66 brd ff:ff:ff:ff:ff:ff 2024-05-09T04:35:31.422 INFO:teuthology.orchestra.run.smithi145.stdout: altname enp7s0f0 2024-05-09T04:35:31.422 INFO:teuthology.orchestra.run.smithi145.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T04:35:31.422 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:d9:38:67 brd ff:ff:ff:ff:ff:ff 2024-05-09T04:35:31.422 INFO:teuthology.orchestra.run.smithi145.stdout: altname enp7s0f1 2024-05-09T04:35:31.422 INFO:teuthology.orchestra.run.smithi145.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T04:35:31.422 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:88:77:90 brd ff:ff:ff:ff:ff:ff 2024-05-09T04:35:31.422 INFO:teuthology.orchestra.run.smithi145.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-09T04:35:31.422 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0c:c4:7a:88:77:91 brd ff:ff:ff:ff:ff:ff 2024-05-09T04:35:31.423 INFO:teuthology.orchestra.run.smithi145.stdout: inet 172.21.15.145/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-05-09T04:35:31.423 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft 42530sec preferred_lft 42530sec 2024-05-09T04:35:31.423 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::ec4:7aff:fe88:7791/64 scope link 2024-05-09T04:35:31.423 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-05-09T04:35:31.423 INFO:teuthology.orchestra.run.smithi145.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-05-09T04:35:31.423 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0e:0f:1c:f7:96:a9 brd ff:ff:ff:ff:ff:ff 2024-05-09T04:35:31.423 INFO:teuthology.orchestra.run.smithi145.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-05-09T04:35:31.423 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-05-09T04:35:31.423 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::e0a8:1bff:fe9d:6975/64 scope link 2024-05-09T04:35:31.423 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-05-09T04:35:31.423 INFO:teuthology.orchestra.run.smithi145.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-09T04:35:31.423 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether 0e:0f:1c:f7:96:a9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T04:35:31.423 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::c0f:1cff:fef7:96a9/64 scope link 2024-05-09T04:35:31.423 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-05-09T04:35:31.423 INFO:teuthology.orchestra.run.smithi145.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-05-09T04:35:31.423 INFO:teuthology.orchestra.run.smithi145.stdout: link/ether b2:86:d9:a3:05:22 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T04:35:31.423 INFO:teuthology.orchestra.run.smithi145.stdout: inet6 fe80::b086:d9ff:fea3:522/64 scope link 2024-05-09T04:35:31.424 INFO:teuthology.orchestra.run.smithi145.stdout: valid_lft forever preferred_lft forever 2024-05-09T04:35:31.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:35:31.425 DEBUG:teuthology.orchestra.run.smithi145:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-09T04:35:31.543 INFO:teuthology.orchestra.run.smithi145.stdout:1 2024-05-09T04:35:31.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:35:31.547 DEBUG:teuthology.orchestra.run.smithi145:> route 2024-05-09T04:35:31.599 INFO:teuthology.orchestra.run.smithi145.stdout:Kernel IP routing table 2024-05-09T04:35:31.599 INFO:teuthology.orchestra.run.smithi145.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-09T04:35:31.599 INFO:teuthology.orchestra.run.smithi145.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-09T04:35:31.599 INFO:teuthology.orchestra.run.smithi145.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-09T04:35:31.599 INFO:teuthology.orchestra.run.smithi145.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-09T04:35:31.600 DEBUG:teuthology.orchestra.run.smithi145:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T04:35:31.600 DEBUG:teuthology.orchestra.run.smithi145:> set -e 2024-05-09T04:35:31.600 DEBUG:teuthology.orchestra.run.smithi145:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-09T04:35:31.600 DEBUG:teuthology.orchestra.run.smithi145:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-09T04:35:31.600 DEBUG:teuthology.orchestra.run.smithi145:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-09T04:35:31.601 DEBUG:teuthology.orchestra.run.smithi145:> ') 2024-05-09T04:35:31.705 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T04:35:31.801 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T04:35:31.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:35:31.810 DEBUG:teuthology.orchestra.run.smithi145:> ip netns list 2024-05-09T04:35:31.853 INFO:teuthology.orchestra.run.smithi145.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-09T04:35:31.854 INFO:teuthology.orchestra.run.smithi145.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-09T04:35:31.854 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-09T04:35:31.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:35:31.855 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip link set brx.0 up 2024-05-09T04:35:31.977 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-09T04:35:31.977 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T04:35:31.977 DEBUG:teuthology.orchestra.run.smithi145:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-09T04:35:32.026 INFO:teuthology.orchestra.run.smithi145.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-09T04:35:32.026 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T04:35:32.026 DEBUG:teuthology.orchestra.run.smithi145:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-09T04:35:32.074 DEBUG:teuthology.orchestra.run.smithi145:> sudo modprobe fuse 2024-05-09T04:35:32.154 DEBUG:teuthology.orchestra.run.smithi145:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T04:35:32.207 INFO:teuthology.orchestra.run.smithi145.stdout:/proc 2024-05-09T04:35:32.207 INFO:teuthology.orchestra.run.smithi145.stdout:/sys 2024-05-09T04:35:32.207 INFO:teuthology.orchestra.run.smithi145.stdout:/dev 2024-05-09T04:35:32.207 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/kernel/security 2024-05-09T04:35:32.207 INFO:teuthology.orchestra.run.smithi145.stdout:/dev/shm 2024-05-09T04:35:32.207 INFO:teuthology.orchestra.run.smithi145.stdout:/dev/pts 2024-05-09T04:35:32.207 INFO:teuthology.orchestra.run.smithi145.stdout:/run 2024-05-09T04:35:32.207 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/fs/cgroup 2024-05-09T04:35:32.207 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/fs/pstore 2024-05-09T04:35:32.208 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/fs/bpf 2024-05-09T04:35:32.208 INFO:teuthology.orchestra.run.smithi145.stdout:/ 2024-05-09T04:35:32.208 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/fs/selinux 2024-05-09T04:35:32.208 INFO:teuthology.orchestra.run.smithi145.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T04:35:32.208 INFO:teuthology.orchestra.run.smithi145.stdout:/dev/mqueue 2024-05-09T04:35:32.208 INFO:teuthology.orchestra.run.smithi145.stdout:/dev/hugepages 2024-05-09T04:35:32.208 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/kernel/debug 2024-05-09T04:35:32.208 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/kernel/tracing 2024-05-09T04:35:32.208 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/fs/fuse/connections 2024-05-09T04:35:32.208 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/kernel/config 2024-05-09T04:35:32.208 INFO:teuthology.orchestra.run.smithi145.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T04:35:32.208 INFO:teuthology.orchestra.run.smithi145.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T04:35:32.208 INFO:teuthology.orchestra.run.smithi145.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T04:35:32.208 INFO:teuthology.orchestra.run.smithi145.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T04:35:32.209 INFO:teuthology.orchestra.run.smithi145.stdout:/run/user/1000 2024-05-09T04:35:32.209 INFO:teuthology.orchestra.run.smithi145.stdout:/var/lib/ceph 2024-05-09T04:35:32.209 INFO:teuthology.orchestra.run.smithi145.stdout:/var/lib/ceph/osd/ceph-0 2024-05-09T04:35:32.209 INFO:teuthology.orchestra.run.smithi145.stdout:/var/lib/ceph/osd/ceph-1 2024-05-09T04:35:32.209 INFO:teuthology.orchestra.run.smithi145.stdout:/var/lib/ceph/osd/ceph-2 2024-05-09T04:35:32.209 INFO:teuthology.orchestra.run.smithi145.stdout:/var/lib/ceph/osd/ceph-3 2024-05-09T04:35:32.209 INFO:teuthology.orchestra.run.smithi145.stdout:/run/netns 2024-05-09T04:35:32.209 INFO:teuthology.orchestra.run.smithi145.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T04:35:32.209 INFO:teuthology.orchestra.run.smithi145.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T04:35:32.209 INFO:teuthology.orchestra.run.smithi145.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T04:35:32.209 INFO:teuthology.orchestra.run.smithi145.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T04:35:32.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:35:32.210 DEBUG:teuthology.orchestra.run.smithi145:> ls /sys/fs/fuse/connections 2024-05-09T04:35:32.263 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-09T04:35:32.264 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-09T04:35:32.268 DEBUG:teuthology.orchestra.run.smithi145:> sudo modprobe fuse 2024-05-09T04:35:32.334 DEBUG:teuthology.orchestra.run.smithi145:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T04:35:32.379 INFO:teuthology.orchestra.run.smithi145.stdout:/proc 2024-05-09T04:35:32.380 INFO:teuthology.orchestra.run.smithi145.stdout:/sys 2024-05-09T04:35:32.380 INFO:teuthology.orchestra.run.smithi145.stdout:/dev 2024-05-09T04:35:32.380 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/kernel/security 2024-05-09T04:35:32.380 INFO:teuthology.orchestra.run.smithi145.stdout:/dev/shm 2024-05-09T04:35:32.380 INFO:teuthology.orchestra.run.smithi145.stdout:/dev/pts 2024-05-09T04:35:32.380 INFO:teuthology.orchestra.run.smithi145.stdout:/run 2024-05-09T04:35:32.380 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/fs/cgroup 2024-05-09T04:35:32.380 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/fs/pstore 2024-05-09T04:35:32.380 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/fs/bpf 2024-05-09T04:35:32.380 INFO:teuthology.orchestra.run.smithi145.stdout:/ 2024-05-09T04:35:32.380 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/fs/selinux 2024-05-09T04:35:32.380 INFO:teuthology.orchestra.run.smithi145.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T04:35:32.380 INFO:teuthology.orchestra.run.smithi145.stdout:/dev/mqueue 2024-05-09T04:35:32.381 INFO:teuthology.orchestra.run.smithi145.stdout:/dev/hugepages 2024-05-09T04:35:32.381 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/kernel/debug 2024-05-09T04:35:32.381 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/kernel/tracing 2024-05-09T04:35:32.381 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/fs/fuse/connections 2024-05-09T04:35:32.381 INFO:teuthology.orchestra.run.smithi145.stdout:/sys/kernel/config 2024-05-09T04:35:32.381 INFO:teuthology.orchestra.run.smithi145.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T04:35:32.381 INFO:teuthology.orchestra.run.smithi145.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T04:35:32.381 INFO:teuthology.orchestra.run.smithi145.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T04:35:32.381 INFO:teuthology.orchestra.run.smithi145.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T04:35:32.381 INFO:teuthology.orchestra.run.smithi145.stdout:/run/user/1000 2024-05-09T04:35:32.381 INFO:teuthology.orchestra.run.smithi145.stdout:/var/lib/ceph 2024-05-09T04:35:32.381 INFO:teuthology.orchestra.run.smithi145.stdout:/var/lib/ceph/osd/ceph-0 2024-05-09T04:35:32.381 INFO:teuthology.orchestra.run.smithi145.stdout:/var/lib/ceph/osd/ceph-1 2024-05-09T04:35:32.381 INFO:teuthology.orchestra.run.smithi145.stdout:/var/lib/ceph/osd/ceph-2 2024-05-09T04:35:32.381 INFO:teuthology.orchestra.run.smithi145.stdout:/var/lib/ceph/osd/ceph-3 2024-05-09T04:35:32.382 INFO:teuthology.orchestra.run.smithi145.stdout:/run/netns 2024-05-09T04:35:32.382 INFO:teuthology.orchestra.run.smithi145.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T04:35:32.382 INFO:teuthology.orchestra.run.smithi145.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T04:35:32.382 INFO:teuthology.orchestra.run.smithi145.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T04:35:32.382 INFO:teuthology.orchestra.run.smithi145.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T04:35:32.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:35:32.383 DEBUG:teuthology.orchestra.run.smithi145:> ls /sys/fs/fuse/connections 2024-05-09T04:35:32.437 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi145.stderr:2024-05-09T04:35:32.437+0000 7f34461e3480 -1 init, newargv = 0x55e4109223a0 newargc=15 2024-05-09T04:35:32.437 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi145.stderr:ceph-fuse[50203]: starting ceph client 2024-05-09T04:35:32.445 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi145.stderr:ceph-fuse[50203]: starting fuse 2024-05-09T04:35:32.467 INFO:teuthology.orchestra.run.smithi145.stdout:40 2024-05-09T04:35:32.468 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2024-05-09T04:35:32.468 DEBUG:teuthology.orchestra.run.smithi145:> sudo stdin-killer -- python3 -c ' 2024-05-09T04:35:32.468 DEBUG:teuthology.orchestra.run.smithi145:> import glob 2024-05-09T04:35:32.468 DEBUG:teuthology.orchestra.run.smithi145:> import re 2024-05-09T04:35:32.468 DEBUG:teuthology.orchestra.run.smithi145:> import os 2024-05-09T04:35:32.468 DEBUG:teuthology.orchestra.run.smithi145:> import subprocess 2024-05-09T04:35:32.468 DEBUG:teuthology.orchestra.run.smithi145:> 2024-05-09T04:35:32.468 DEBUG:teuthology.orchestra.run.smithi145:> def _find_admin_socket(client_name): 2024-05-09T04:35:32.468 DEBUG:teuthology.orchestra.run.smithi145:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-09T04:35:32.469 DEBUG:teuthology.orchestra.run.smithi145:> files = glob.glob(asok_path) 2024-05-09T04:35:32.469 DEBUG:teuthology.orchestra.run.smithi145:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-09T04:35:32.469 DEBUG:teuthology.orchestra.run.smithi145:> 2024-05-09T04:35:32.469 DEBUG:teuthology.orchestra.run.smithi145:> # Given a non-glob path, it better be there 2024-05-09T04:35:32.469 DEBUG:teuthology.orchestra.run.smithi145:> if "*" not in asok_path: 2024-05-09T04:35:32.469 DEBUG:teuthology.orchestra.run.smithi145:> assert(len(files) == 1) 2024-05-09T04:35:32.469 DEBUG:teuthology.orchestra.run.smithi145:> return files[0] 2024-05-09T04:35:32.469 DEBUG:teuthology.orchestra.run.smithi145:> 2024-05-09T04:35:32.469 DEBUG:teuthology.orchestra.run.smithi145:> for f in files: 2024-05-09T04:35:32.469 DEBUG:teuthology.orchestra.run.smithi145:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-09T04:35:32.469 DEBUG:teuthology.orchestra.run.smithi145:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-09T04:35:32.469 DEBUG:teuthology.orchestra.run.smithi145:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-09T04:35:32.469 DEBUG:teuthology.orchestra.run.smithi145:> contents = proc_f.read() 2024-05-09T04:35:32.469 DEBUG:teuthology.orchestra.run.smithi145:> if mountpoint in contents: 2024-05-09T04:35:32.469 DEBUG:teuthology.orchestra.run.smithi145:> return f 2024-05-09T04:35:32.469 DEBUG:teuthology.orchestra.run.smithi145:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-09T04:35:32.470 DEBUG:teuthology.orchestra.run.smithi145:> 2024-05-09T04:35:32.470 DEBUG:teuthology.orchestra.run.smithi145:> print(_find_admin_socket("client.0")) 2024-05-09T04:35:32.470 DEBUG:teuthology.orchestra.run.smithi145:> ' 2024-05-09T04:35:32.657 INFO:teuthology.orchestra.run.smithi145.stdout:/var/run/ceph/ceph-client.0.50203.asok 2024-05-09T04:35:32.661 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T04:35:32.674 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.50203.asok 2024-05-09T04:35:32.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:35:32.675 DEBUG:teuthology.orchestra.run.smithi145:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.50203.asok status 2024-05-09T04:35:32.899 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-05-09T04:35:32.899 INFO:teuthology.orchestra.run.smithi145.stdout: "metadata": { 2024-05-09T04:35:32.899 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_sha1": "165a15e51d08073b5317c443415b0d8a7141b1ab", 2024-05-09T04:35:32.899 INFO:teuthology.orchestra.run.smithi145.stdout: "ceph_version": "ceph version 19.0.0-3568-g165a15e5 (165a15e51d08073b5317c443415b0d8a7141b1ab) squid (dev)", 2024-05-09T04:35:32.900 INFO:teuthology.orchestra.run.smithi145.stdout: "entity_id": "0", 2024-05-09T04:35:32.900 INFO:teuthology.orchestra.run.smithi145.stdout: "hostname": "smithi145", 2024-05-09T04:35:32.900 INFO:teuthology.orchestra.run.smithi145.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-09T04:35:32.900 INFO:teuthology.orchestra.run.smithi145.stdout: "pid": "50203", 2024-05-09T04:35:32.900 INFO:teuthology.orchestra.run.smithi145.stdout: "root": "/" 2024-05-09T04:35:32.900 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:35:32.900 INFO:teuthology.orchestra.run.smithi145.stdout: "dentry_count": 0, 2024-05-09T04:35:32.900 INFO:teuthology.orchestra.run.smithi145.stdout: "dentry_pinned_count": 0, 2024-05-09T04:35:32.900 INFO:teuthology.orchestra.run.smithi145.stdout: "id": 5484, 2024-05-09T04:35:32.900 INFO:teuthology.orchestra.run.smithi145.stdout: "inst": { 2024-05-09T04:35:32.900 INFO:teuthology.orchestra.run.smithi145.stdout: "name": { 2024-05-09T04:35:32.901 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "client", 2024-05-09T04:35:32.901 INFO:teuthology.orchestra.run.smithi145.stdout: "num": 5484 2024-05-09T04:35:32.901 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:35:32.901 INFO:teuthology.orchestra.run.smithi145.stdout: "addr": { 2024-05-09T04:35:32.901 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "v1", 2024-05-09T04:35:32.901 INFO:teuthology.orchestra.run.smithi145.stdout: "addr": "192.168.0.1:0", 2024-05-09T04:35:32.901 INFO:teuthology.orchestra.run.smithi145.stdout: "nonce": 3166185683 2024-05-09T04:35:32.901 INFO:teuthology.orchestra.run.smithi145.stdout: } 2024-05-09T04:35:32.901 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:35:32.901 INFO:teuthology.orchestra.run.smithi145.stdout: "addr": { 2024-05-09T04:35:32.901 INFO:teuthology.orchestra.run.smithi145.stdout: "type": "v1", 2024-05-09T04:35:32.901 INFO:teuthology.orchestra.run.smithi145.stdout: "addr": "192.168.0.1:0", 2024-05-09T04:35:32.901 INFO:teuthology.orchestra.run.smithi145.stdout: "nonce": 3166185683 2024-05-09T04:35:32.901 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2024-05-09T04:35:32.901 INFO:teuthology.orchestra.run.smithi145.stdout: "inst_str": "client.5484 192.168.0.1:0/3166185683", 2024-05-09T04:35:32.902 INFO:teuthology.orchestra.run.smithi145.stdout: "addr_str": "192.168.0.1:0/3166185683", 2024-05-09T04:35:32.902 INFO:teuthology.orchestra.run.smithi145.stdout: "inode_count": 1, 2024-05-09T04:35:32.902 INFO:teuthology.orchestra.run.smithi145.stdout: "mds_epoch": 27, 2024-05-09T04:35:32.902 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_epoch": 47, 2024-05-09T04:35:32.902 INFO:teuthology.orchestra.run.smithi145.stdout: "osd_epoch_barrier": 0, 2024-05-09T04:35:32.902 INFO:teuthology.orchestra.run.smithi145.stdout: "blocklisted": false, 2024-05-09T04:35:32.902 INFO:teuthology.orchestra.run.smithi145.stdout: "fs_name": "cephfs" 2024-05-09T04:35:32.902 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-05-09T04:35:32.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:35:32.915 DEBUG:teuthology.orchestra.run.smithi145:> stat --file-system '--printf=%T 2024-05-09T04:35:32.915 DEBUG:teuthology.orchestra.run.smithi145:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-09T04:35:32.968 INFO:teuthology.orchestra.run.smithi145.stdout:fuseblk 2024-05-09T04:35:32.968 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-09T04:35:32.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:35:32.968 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-09T04:35:33.051 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_extra1 --pg_num_min 64 2024-05-09T04:35:33.259 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:33.259+0000 7f036b200640 1 Processor -- start 2024-05-09T04:35:33.259 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:33.259+0000 7f036b200640 1 -- start start 2024-05-09T04:35:33.262 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:33.261+0000 7f036b200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f036c0b4af0 0x7f036c0cf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:33.262 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:33.261+0000 7f036b200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f036c0cfcc0 0x7f036c06e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:33.262 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:33.261+0000 7f036b200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f036c0b6030 con 0x7f036c143590 2024-05-09T04:35:33.262 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:33.261+0000 7f036b200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f036c0b5ba0 con 0x7f036c0cfcc0 2024-05-09T04:35:33.262 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:33.261+0000 7f036b200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f036c0b4f90 con 0x7f036c0b4af0 2024-05-09T04:35:33.262 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:33.261+0000 7f036a800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f036c0cfcc0 0x7f036c06e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:33.262 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:33.261+0000 7f036a800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f036c0cfcc0 0x7f036c06e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:56322/0 (socket says 172.21.15.145:56322) 2024-05-09T04:35:33.262 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:33.261+0000 7f036a800640 1 -- 172.21.15.145:0/168999135 learned_addr learned my addr 172.21.15.145:0/168999135 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:33.263 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:33.261+0000 7f0369e00640 1 --1- 172.21.15.145:0/168999135 >> v1:172.21.15.145:6789/0 conn(0x7f036c143590 0x7f036c15e0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:52728/0 (socket says 172.21.15.145:52728) 2024-05-09T04:35:33.263 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:33.262+0000 7f0363e00640 1 --2- 172.21.15.145:0/168999135 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f036c0b4af0 0x7f036c0cf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:33.263 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:33.262+0000 7f036a800640 1 -- 172.21.15.145:0/168999135 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f036c0b4af0 msgr2=0x7f036c0cf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:33.263 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:33.262+0000 7f036a800640 1 --2- 172.21.15.145:0/168999135 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f036c0b4af0 0x7f036c0cf780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:33.263 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:33.262+0000 7f036a800640 1 -- 172.21.15.145:0/168999135 >> v1:172.21.15.145:6789/0 conn(0x7f036c143590 legacy=0x7f036c15e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:33.263 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:33.262+0000 7f036a800640 1 -- 172.21.15.145:0/168999135 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f036c0b5270 con 0x7f036c0cfcc0 2024-05-09T04:35:33.263 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:33.262+0000 7f0363e00640 1 --2- 172.21.15.145:0/168999135 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f036c0b4af0 0x7f036c0cf780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:35:33.264 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:33.263+0000 7f036a800640 1 --2- 172.21.15.145:0/168999135 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f036c0cfcc0 0x7f036c06e0c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f0364009e50 tx=0x7f036402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78226b0b56bb974c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:33.265 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:33.263+0000 7f0369400640 1 -- 172.21.15.145:0/168999135 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0364036070 con 0x7f036c0cfcc0 2024-05-09T04:35:33.265 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:33.263+0000 7f0369400640 1 -- 172.21.15.145:0/168999135 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0364031600 con 0x7f036c0cfcc0 2024-05-09T04:35:33.265 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:33.263+0000 7f0369400640 1 -- 172.21.15.145:0/168999135 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03640319d0 con 0x7f036c0cfcc0 2024-05-09T04:35:33.265 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:33.264+0000 7f036b200640 1 -- 172.21.15.145:0/168999135 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f036c0cfcc0 msgr2=0x7f036c06e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:33.265 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:33.264+0000 7f036b200640 1 --2- 172.21.15.145:0/168999135 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f036c0cfcc0 0x7f036c06e0c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f0364009e50 tx=0x7f036402a710 comp rx=0 tx=0).stop 2024-05-09T04:35:33.265 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:33.264+0000 7f036b200640 1 -- 172.21.15.145:0/168999135 shutdown_connections 2024-05-09T04:35:33.265 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:33.264+0000 7f036b200640 1 --2- 172.21.15.145:0/168999135 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f036c0cfcc0 0x7f036c06e0c0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:33.265 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:33.264+0000 7f036b200640 1 --2- 172.21.15.145:0/168999135 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f036c0b4af0 0x7f036c0cf780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:33.265 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:33.264+0000 7f036b200640 1 -- 172.21.15.145:0/168999135 >> 172.21.15.145:0/168999135 conn(0x7f036c0b2850 msgr2=0x7f036c0b2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:33.265 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:33.264+0000 7f036b200640 1 -- 172.21.15.145:0/168999135 shutdown_connections 2024-05-09T04:35:33.266 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:33.264+0000 7f036b200640 1 -- 172.21.15.145:0/168999135 wait complete. 2024-05-09T04:35:33.267 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:33.266+0000 7f036b200640 1 Processor -- start 2024-05-09T04:35:33.267 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:33.266+0000 7f036b200640 1 -- start start 2024-05-09T04:35:33.267 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:33.267+0000 7f036b200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f036c0b4af0 0x7f036c15ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:33.268 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:33.267+0000 7f036b200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f036c0cfcc0 0x7f036c0c9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:33.268 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:33.267+0000 7f036b200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f036c143590 0x7f036c0c9a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:33.268 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:33.267+0000 7f036b200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f036c0cec40 con 0x7f036c143590 2024-05-09T04:35:33.268 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:33.267+0000 7f036b200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f036c0b5830 con 0x7f036c0cfcc0 2024-05-09T04:35:33.268 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:33.267+0000 7f036b200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f036c143970 con 0x7f036c0b4af0 2024-05-09T04:35:33.268 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:33.267+0000 7f036a800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f036c143590 0x7f036c0c9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:33.268 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:33.267+0000 7f036a800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f036c143590 0x7f036c0c9a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:56328/0 (socket says 172.21.15.145:56328) 2024-05-09T04:35:33.268 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:33.267+0000 7f036a800640 1 -- 172.21.15.145:0/2783791001 learned_addr learned my addr 172.21.15.145:0/2783791001 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:33.268 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:33.267+0000 7f036a800640 1 -- 172.21.15.145:0/2783791001 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f036c0b4af0 msgr2=0x7f036c15ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:33.268 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:33.267+0000 7f036a800640 1 --2- 172.21.15.145:0/2783791001 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f036c0b4af0 0x7f036c15ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:33.268 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:33.267+0000 7f036a800640 1 -- 172.21.15.145:0/2783791001 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f036c0cfcc0 msgr2=0x7f036c0c9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:33.268 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:33.267+0000 7f036a800640 1 --2- 172.21.15.145:0/2783791001 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f036c0cfcc0 0x7f036c0c9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:33.269 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:33.267+0000 7f036a800640 1 -- 172.21.15.145:0/2783791001 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f036c132610 con 0x7f036c143590 2024-05-09T04:35:33.269 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:33.268+0000 7f036a800640 1 --2- 172.21.15.145:0/2783791001 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f036c143590 0x7f036c0c9a50 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f0364009f70 tx=0x7f0364004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:33.269 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:33.268+0000 7f0362000640 1 -- 172.21.15.145:0/2783791001 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f036403f020 con 0x7f036c143590 2024-05-09T04:35:33.270 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:33.268+0000 7f0362000640 1 -- 172.21.15.145:0/2783791001 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0364037040 con 0x7f036c143590 2024-05-09T04:35:33.270 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:33.268+0000 7f0362000640 1 -- 172.21.15.145:0/2783791001 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0364036070 con 0x7f036c143590 2024-05-09T04:35:33.270 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:33.268+0000 7f036b200640 1 -- 172.21.15.145:0/2783791001 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f036c0c90e0 con 0x7f036c143590 2024-05-09T04:35:33.270 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:33.268+0000 7f036b200640 1 -- 172.21.15.145:0/2783791001 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f036c131b30 con 0x7f036c143590 2024-05-09T04:35:33.270 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:33.269+0000 7f036b200640 1 -- 172.21.15.145:0/2783791001 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f036c1430f0 con 0x7f036c143590 2024-05-09T04:35:33.270 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:33.270+0000 7f0362000640 1 -- 172.21.15.145:0/2783791001 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0364042020 con 0x7f036c143590 2024-05-09T04:35:33.271 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:33.271+0000 7f0362000640 1 --2- 172.21.15.145:0/2783791001 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f03480762d0 0x7f0348078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:33.271 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:33.272+0000 7f0362000640 1 -- 172.21.15.145:0/2783791001 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f03640b7c60 con 0x7f036c143590 2024-05-09T04:35:33.271 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:33.272+0000 7f0369e00640 1 --2- 172.21.15.145:0/2783791001 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f03480762d0 0x7f0348078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:33.272 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:33.272+0000 7f0369e00640 1 --2- 172.21.15.145:0/2783791001 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f03480762d0 0x7f0348078790 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f035c001530 tx=0x7f035c0079c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:33.276 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:33.276+0000 7f0362000640 1 -- 172.21.15.145:0/2783791001 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f036c1430f0 con 0x7f036c143590 2024-05-09T04:35:33.493 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:33.493+0000 7f036b200640 1 -- 172.21.15.145:0/2783791001 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_extra1", "pg_num_min": 64} v 0) -- 0x7f036c076930 con 0x7f036c143590 2024-05-09T04:35:34.045 DEBUG:teuthology.orchestra.run.smithi145:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T04:35:34.053 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T04:35:34.314 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.314+0000 7f0362000640 1 -- 172.21.15.145:0/2783791001 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_extra1", "pg_num_min": 64}]=0 pool 'cephfs_data_extra1' created v49) ==== 144+0+0 (secure 0 0 0) 0x7f0364049030 con 0x7f036c143590 2024-05-09T04:35:34.315 INFO:teuthology.orchestra.run.smithi145.stderr:pool 'cephfs_data_extra1' created 2024-05-09T04:35:34.318 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.319+0000 7f036b200640 1 -- 172.21.15.145:0/2783791001 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f03480762d0 msgr2=0x7f0348078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:34.319 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.319+0000 7f036b200640 1 --2- 172.21.15.145:0/2783791001 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f03480762d0 0x7f0348078790 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f035c001530 tx=0x7f035c0079c0 comp rx=0 tx=0).stop 2024-05-09T04:35:34.319 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.319+0000 7f036b200640 1 -- 172.21.15.145:0/2783791001 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f036c143590 msgr2=0x7f036c0c9a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:34.319 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.319+0000 7f036b200640 1 --2- 172.21.15.145:0/2783791001 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f036c143590 0x7f036c0c9a50 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f0364009f70 tx=0x7f0364004a40 comp rx=0 tx=0).stop 2024-05-09T04:35:34.319 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.319+0000 7f036b200640 1 -- 172.21.15.145:0/2783791001 shutdown_connections 2024-05-09T04:35:34.320 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.319+0000 7f036b200640 1 --2- 172.21.15.145:0/2783791001 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f03480762d0 0x7f0348078790 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:34.320 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.319+0000 7f036b200640 1 --2- 172.21.15.145:0/2783791001 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f036c143590 0x7f036c0c9a50 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:34.320 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.319+0000 7f036b200640 1 --2- 172.21.15.145:0/2783791001 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f036c0cfcc0 0x7f036c0c9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:34.320 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.319+0000 7f036b200640 1 --2- 172.21.15.145:0/2783791001 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f036c0b4af0 0x7f036c15ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:34.320 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.319+0000 7f036b200640 1 -- 172.21.15.145:0/2783791001 >> 172.21.15.145:0/2783791001 conn(0x7f036c0b2850 msgr2=0x7f036c15b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:34.320 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.319+0000 7f036b200640 1 -- 172.21.15.145:0/2783791001 shutdown_connections 2024-05-09T04:35:34.320 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.319+0000 7f036b200640 1 -- 172.21.15.145:0/2783791001 wait complete. 2024-05-09T04:35:34.350 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_extra1 2024-05-09T04:35:34.534 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.534+0000 7fd67d200640 1 Processor -- start 2024-05-09T04:35:34.534 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.534+0000 7fd67d200640 1 -- start start 2024-05-09T04:35:34.536 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.536+0000 7fd67d200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fd678070040 0x7fd678073870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:34.536 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.536+0000 7fd67d200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd678162c70 0x7fd678163100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:34.536 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.536+0000 7fd67d200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd678160530 con 0x7fd67815f140 2024-05-09T04:35:34.536 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.536+0000 7fd67d200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fd678072350 con 0x7fd678070040 2024-05-09T04:35:34.536 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.536+0000 7fd67d200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fd6780734b0 con 0x7fd678162c70 2024-05-09T04:35:34.537 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.536+0000 7fd676a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fd678070040 0x7fd678073870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:34.537 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.536+0000 7fd676a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fd678070040 0x7fd678073870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:56336/0 (socket says 172.21.15.145:56336) 2024-05-09T04:35:34.537 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.536+0000 7fd676a00640 1 -- 172.21.15.145:0/1790989386 learned_addr learned my addr 172.21.15.145:0/1790989386 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:34.537 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.537+0000 7fd677e00640 1 --2- 172.21.15.145:0/1790989386 >> v2:172.21.15.192:3300/0 conn(0x7fd678162c70 0x7fd678163100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:34.537 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.537+0000 7fd676a00640 1 -- 172.21.15.145:0/1790989386 >> v2:172.21.15.192:3300/0 conn(0x7fd678162c70 msgr2=0x7fd678163100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:34.537 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.537+0000 7fd676a00640 1 --2- 172.21.15.145:0/1790989386 >> v2:172.21.15.192:3300/0 conn(0x7fd678162c70 0x7fd678163100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:34.537 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.537+0000 7fd676a00640 1 -- 172.21.15.145:0/1790989386 >> v1:172.21.15.192:6789/0 conn(0x7fd67815f140 legacy=0x7fd678072da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:34.538 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.537+0000 7fd676a00640 1 -- 172.21.15.145:0/1790989386 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd678073fa0 con 0x7fd678070040 2024-05-09T04:35:34.538 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.538+0000 7fd676a00640 1 --2- 172.21.15.145:0/1790989386 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd678070040 0x7fd678073870 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd6780b3870 tx=0x7fd66402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a1377a5ffdae462 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:34.538 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.538+0000 7fd676000640 1 -- 172.21.15.145:0/1790989386 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd664037070 con 0x7fd678070040 2024-05-09T04:35:34.538 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.538+0000 7fd676000640 1 -- 172.21.15.145:0/1790989386 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd664032440 con 0x7fd678070040 2024-05-09T04:35:34.538 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.538+0000 7fd676000640 1 -- 172.21.15.145:0/1790989386 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd664032810 con 0x7fd678070040 2024-05-09T04:35:34.539 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.539+0000 7fd67d200640 1 -- 172.21.15.145:0/1790989386 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd678070040 msgr2=0x7fd678073870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:34.539 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.539+0000 7fd67d200640 1 --2- 172.21.15.145:0/1790989386 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd678070040 0x7fd678073870 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd6780b3870 tx=0x7fd66402a140 comp rx=0 tx=0).stop 2024-05-09T04:35:34.539 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.539+0000 7fd67d200640 1 -- 172.21.15.145:0/1790989386 shutdown_connections 2024-05-09T04:35:34.540 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.539+0000 7fd67d200640 1 --2- 172.21.15.145:0/1790989386 >> v2:172.21.15.192:3300/0 conn(0x7fd678162c70 0x7fd678163100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:34.540 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.539+0000 7fd67d200640 1 --2- 172.21.15.145:0/1790989386 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd678070040 0x7fd678073870 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:34.540 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.539+0000 7fd67d200640 1 -- 172.21.15.145:0/1790989386 >> 172.21.15.145:0/1790989386 conn(0x7fd6780b28f0 msgr2=0x7fd6780b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:34.540 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.539+0000 7fd67d200640 1 -- 172.21.15.145:0/1790989386 shutdown_connections 2024-05-09T04:35:34.540 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.539+0000 7fd67d200640 1 -- 172.21.15.145:0/1790989386 wait complete. 2024-05-09T04:35:34.541 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.541+0000 7fd67d200640 1 Processor -- start 2024-05-09T04:35:34.541 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.541+0000 7fd67d200640 1 -- start start 2024-05-09T04:35:34.543 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.542+0000 7fd67d200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd678070040 0x7fd67814f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:34.543 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.542+0000 7fd67d200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd67815f140 0x7fd67815e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:34.544 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.542+0000 7fd67d200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd678162c70 0x7fd67815a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:34.544 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.542+0000 7fd67d200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fd6780725a0 con 0x7fd67815f140 2024-05-09T04:35:34.544 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.542+0000 7fd67d200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fd6780704d0 con 0x7fd678162c70 2024-05-09T04:35:34.544 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.542+0000 7fd67d200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fd678072880 con 0x7fd678070040 2024-05-09T04:35:34.544 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.543+0000 7fd676a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd67815f140 0x7fd67815e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:34.544 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.543+0000 7fd676a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd67815f140 0x7fd67815e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:56346/0 (socket says 172.21.15.145:56346) 2024-05-09T04:35:34.544 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.543+0000 7fd676a00640 1 -- 172.21.15.145:0/1599351736 learned_addr learned my addr 172.21.15.145:0/1599351736 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:34.544 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.543+0000 7fd677e00640 1 --2- 172.21.15.145:0/1599351736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd678162c70 0x7fd67815a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:34.544 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.543+0000 7fd677400640 1 --2- 172.21.15.145:0/1599351736 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd678070040 0x7fd67814f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:34.545 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.543+0000 7fd676a00640 1 -- 172.21.15.145:0/1599351736 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd678070040 msgr2=0x7fd67814f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:34.545 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.543+0000 7fd676a00640 1 --2- 172.21.15.145:0/1599351736 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd678070040 0x7fd67814f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:34.545 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.543+0000 7fd676a00640 1 -- 172.21.15.145:0/1599351736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd678162c70 msgr2=0x7fd67815a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:34.545 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.543+0000 7fd676a00640 1 --2- 172.21.15.145:0/1599351736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd678162c70 0x7fd67815a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:34.545 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.543+0000 7fd676a00640 1 -- 172.21.15.145:0/1599351736 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd678157210 con 0x7fd67815f140 2024-05-09T04:35:34.545 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.544+0000 7fd676a00640 1 --2- 172.21.15.145:0/1599351736 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd67815f140 0x7fd67815e1f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd66402a180 tx=0x7fd6640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:34.546 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.544+0000 7fd66f400640 1 -- 172.21.15.145:0/1599351736 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd664041020 con 0x7fd67815f140 2024-05-09T04:35:34.546 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.544+0000 7fd66f400640 1 -- 172.21.15.145:0/1599351736 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6640306c0 con 0x7fd67815f140 2024-05-09T04:35:34.547 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.544+0000 7fd66f400640 1 -- 172.21.15.145:0/1599351736 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd664037040 con 0x7fd67815f140 2024-05-09T04:35:34.547 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.544+0000 7fd67d200640 1 -- 172.21.15.145:0/1599351736 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6781574f0 con 0x7fd67815f140 2024-05-09T04:35:34.547 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.544+0000 7fd67d200640 1 -- 172.21.15.145:0/1599351736 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd678157ab0 con 0x7fd67815f140 2024-05-09T04:35:34.547 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.545+0000 7fd67d200640 1 -- 172.21.15.145:0/1599351736 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6780af710 con 0x7fd67815f140 2024-05-09T04:35:34.547 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.546+0000 7fd66f400640 1 -- 172.21.15.145:0/1599351736 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fd664044020 con 0x7fd67815f140 2024-05-09T04:35:34.548 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.547+0000 7fd66f400640 1 --2- 172.21.15.145:0/1599351736 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fd6580762d0 0x7fd658078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:34.548 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.547+0000 7fd66f400640 1 -- 172.21.15.145:0/1599351736 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6140+0+0 (secure 0 0 0) 0x7fd6640b7ff0 con 0x7fd67815f140 2024-05-09T04:35:34.548 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.548+0000 7fd677400640 1 --2- 172.21.15.145:0/1599351736 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fd6580762d0 0x7fd658078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:34.548 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.548+0000 7fd677400640 1 --2- 172.21.15.145:0/1599351736 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fd6580762d0 0x7fd658078790 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd668004ea0 tx=0x7fd668009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:34.553 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.553+0000 7fd66f400640 1 -- 172.21.15.145:0/1599351736 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd6780af710 con 0x7fd67815f140 2024-05-09T04:35:34.802 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:34.802+0000 7fd67d200640 1 -- 172.21.15.145:0/1599351736 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_extra1"} v 0) -- 0x7fd67809eab0 con 0x7fd67815f140 2024-05-09T04:35:35.319 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.319+0000 7fd66f400640 1 -- 172.21.15.145:0/1599351736 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_extra1"}]=0 added data pool 9 to fsmap v28) ==== 141+0+0 (secure 0 0 0) 0x7fd67809eab0 con 0x7fd67815f140 2024-05-09T04:35:35.319 INFO:teuthology.orchestra.run.smithi145.stderr:added data pool 9 to fsmap 2024-05-09T04:35:35.323 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.323+0000 7fd67d200640 1 -- 172.21.15.145:0/1599351736 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fd6580762d0 msgr2=0x7fd658078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:35.323 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.323+0000 7fd67d200640 1 --2- 172.21.15.145:0/1599351736 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fd6580762d0 0x7fd658078790 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd668004ea0 tx=0x7fd668009040 comp rx=0 tx=0).stop 2024-05-09T04:35:35.323 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.323+0000 7fd67d200640 1 -- 172.21.15.145:0/1599351736 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd67815f140 msgr2=0x7fd67815e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:35.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.323+0000 7fd67d200640 1 --2- 172.21.15.145:0/1599351736 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd67815f140 0x7fd67815e1f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd66402a180 tx=0x7fd6640046c0 comp rx=0 tx=0).stop 2024-05-09T04:35:35.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.323+0000 7fd67d200640 1 -- 172.21.15.145:0/1599351736 shutdown_connections 2024-05-09T04:35:35.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.323+0000 7fd67d200640 1 --2- 172.21.15.145:0/1599351736 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fd6580762d0 0x7fd658078790 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:35.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.323+0000 7fd67d200640 1 --2- 172.21.15.145:0/1599351736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd678162c70 0x7fd67815a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:35.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.323+0000 7fd67d200640 1 --2- 172.21.15.145:0/1599351736 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd67815f140 0x7fd67815e1f0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:35.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.323+0000 7fd67d200640 1 --2- 172.21.15.145:0/1599351736 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd678070040 0x7fd67814f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:35.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.323+0000 7fd67d200640 1 -- 172.21.15.145:0/1599351736 >> 172.21.15.145:0/1599351736 conn(0x7fd6780b28f0 msgr2=0x7fd6780b39e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:35.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.324+0000 7fd67d200640 1 -- 172.21.15.145:0/1599351736 shutdown_connections 2024-05-09T04:35:35.324 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.324+0000 7fd67d200640 1 -- 172.21.15.145:0/1599351736 wait complete. 2024-05-09T04:35:35.352 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T04:35:35.527 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.527+0000 7fa3d7c00640 1 Processor -- start 2024-05-09T04:35:35.527 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.527+0000 7fa3d7c00640 1 -- start start 2024-05-09T04:35:35.530 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.529+0000 7fa3d7c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa3d015fbc0 0x7fa3d0142200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:35.530 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.529+0000 7fa3d7c00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fa3d015ffa0 0x7fa3d00cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:35.530 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.529+0000 7fa3d7c00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3d013f340 con 0x7fa3d00cfe10 2024-05-09T04:35:35.530 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.529+0000 7fa3d7c00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fa3d0072220 con 0x7fa3d015ffa0 2024-05-09T04:35:35.530 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.529+0000 7fa3d7c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa3d00d02b0 con 0x7fa3d015fbc0 2024-05-09T04:35:35.530 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.529+0000 7fa3cfe00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fa3d015ffa0 0x7fa3d00cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:35.530 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.529+0000 7fa3cfe00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fa3d015ffa0 0x7fa3d00cf8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:56348/0 (socket says 172.21.15.145:56348) 2024-05-09T04:35:35.530 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.529+0000 7fa3cfe00640 1 -- 172.21.15.145:0/1545318577 learned_addr learned my addr 172.21.15.145:0/1545318577 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:35.530 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.530+0000 7fa3cfe00640 1 -- 172.21.15.145:0/1545318577 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa3d015fbc0 msgr2=0x7fa3d0142200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:35.530 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.530+0000 7fa3cfe00640 1 --2- 172.21.15.145:0/1545318577 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa3d015fbc0 0x7fa3d0142200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:35.530 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.530+0000 7fa3cfe00640 1 -- 172.21.15.145:0/1545318577 >> v1:172.21.15.192:6789/0 conn(0x7fa3d00cfe10 legacy=0x7fa3d015c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:35.530 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.530+0000 7fa3cfe00640 1 -- 172.21.15.145:0/1545318577 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3d00d0590 con 0x7fa3d015ffa0 2024-05-09T04:35:35.531 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.531+0000 7fa3cfe00640 1 --2- 172.21.15.145:0/1545318577 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa3d015ffa0 0x7fa3d00cf8d0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc009870 tx=0x7fa3bc02a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=305884ccc58bff26 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:35.532 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.531+0000 7fa3cf400640 1 -- 172.21.15.145:0/1545318577 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc037070 con 0x7fa3d015ffa0 2024-05-09T04:35:35.532 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.531+0000 7fa3cf400640 1 -- 172.21.15.145:0/1545318577 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3bc032440 con 0x7fa3d015ffa0 2024-05-09T04:35:35.532 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.531+0000 7fa3cf400640 1 -- 172.21.15.145:0/1545318577 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc032810 con 0x7fa3d015ffa0 2024-05-09T04:35:35.532 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.531+0000 7fa3d7c00640 1 -- 172.21.15.145:0/1545318577 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa3d015ffa0 msgr2=0x7fa3d00cf8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:35.532 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.531+0000 7fa3d7c00640 1 --2- 172.21.15.145:0/1545318577 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa3d015ffa0 0x7fa3d00cf8d0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc009870 tx=0x7fa3bc02a130 comp rx=0 tx=0).stop 2024-05-09T04:35:35.532 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.532+0000 7fa3d7c00640 1 -- 172.21.15.145:0/1545318577 shutdown_connections 2024-05-09T04:35:35.532 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.532+0000 7fa3d7c00640 1 --2- 172.21.15.145:0/1545318577 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa3d015ffa0 0x7fa3d00cf8d0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:35.532 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.532+0000 7fa3d7c00640 1 --2- 172.21.15.145:0/1545318577 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa3d015fbc0 0x7fa3d0142200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:35.532 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.532+0000 7fa3d7c00640 1 -- 172.21.15.145:0/1545318577 >> 172.21.15.145:0/1545318577 conn(0x7fa3d006c870 msgr2=0x7fa3d006cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:35.532 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.532+0000 7fa3d7c00640 1 -- 172.21.15.145:0/1545318577 shutdown_connections 2024-05-09T04:35:35.532 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.532+0000 7fa3d7c00640 1 -- 172.21.15.145:0/1545318577 wait complete. 2024-05-09T04:35:35.533 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.533+0000 7fa3d7c00640 1 Processor -- start 2024-05-09T04:35:35.533 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.534+0000 7fa3d7c00640 1 -- start start 2024-05-09T04:35:35.535 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.534+0000 7fa3d7c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3d00cfe10 0x7fa3d0153350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:35.535 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.534+0000 7fa3d7c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa3d015fbc0 0x7fa3d015d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:35.535 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.535+0000 7fa3d7c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa3d015ffa0 0x7fa3d0133380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:35.535 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.535+0000 7fa3d7c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d0160400 con 0x7fa3d015fbc0 2024-05-09T04:35:35.535 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.535+0000 7fa3d7c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d00d0b50 con 0x7fa3d00cfe10 2024-05-09T04:35:35.535 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.535+0000 7fa3d7c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa3d00736c0 con 0x7fa3d015ffa0 2024-05-09T04:35:35.535 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.535+0000 7fa3cfe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa3d015fbc0 0x7fa3d015d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:35.535 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.535+0000 7fa3cfe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa3d015fbc0 0x7fa3d015d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:56362/0 (socket says 172.21.15.145:56362) 2024-05-09T04:35:35.535 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.535+0000 7fa3cfe00640 1 -- 172.21.15.145:0/4153180622 learned_addr learned my addr 172.21.15.145:0/4153180622 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:35.535 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.535+0000 7fa3cfe00640 1 -- 172.21.15.145:0/4153180622 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa3d015ffa0 msgr2=0x7fa3d0133380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:35.535 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.535+0000 7fa3cfe00640 1 --2- 172.21.15.145:0/4153180622 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa3d015ffa0 0x7fa3d0133380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:35.536 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.535+0000 7fa3cfe00640 1 -- 172.21.15.145:0/4153180622 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3d00cfe10 msgr2=0x7fa3d0153350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:35.536 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.535+0000 7fa3cfe00640 1 --2- 172.21.15.145:0/4153180622 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3d00cfe10 0x7fa3d0153350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:35.536 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.535+0000 7fa3cfe00640 1 -- 172.21.15.145:0/4153180622 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3d01539c0 con 0x7fa3d015fbc0 2024-05-09T04:35:35.536 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.536+0000 7fa3cfe00640 1 --2- 172.21.15.145:0/4153180622 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa3d015fbc0 0x7fa3d015d700 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc009870 tx=0x7fa3bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:35.536 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.536+0000 7fa3ccc00640 1 -- 172.21.15.145:0/4153180622 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc041070 con 0x7fa3d015fbc0 2024-05-09T04:35:35.537 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.536+0000 7fa3ccc00640 1 -- 172.21.15.145:0/4153180622 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3bc03c870 con 0x7fa3d015fbc0 2024-05-09T04:35:35.537 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.536+0000 7fa3ccc00640 1 -- 172.21.15.145:0/4153180622 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc037040 con 0x7fa3d015fbc0 2024-05-09T04:35:35.537 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.536+0000 7fa3d7c00640 1 -- 172.21.15.145:0/4153180622 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3d00b4430 con 0x7fa3d015fbc0 2024-05-09T04:35:35.537 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.536+0000 7fa3d7c00640 1 -- 172.21.15.145:0/4153180622 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3d013eb70 con 0x7fa3d015fbc0 2024-05-09T04:35:35.537 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.537+0000 7fa3d7c00640 1 -- 172.21.15.145:0/4153180622 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3d006a310 con 0x7fa3d015fbc0 2024-05-09T04:35:35.538 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.538+0000 7fa3ccc00640 1 -- 172.21.15.145:0/4153180622 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa3bc044070 con 0x7fa3d015fbc0 2024-05-09T04:35:35.539 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.539+0000 7fa3ccc00640 1 --2- 172.21.15.145:0/4153180622 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fa3b00762d0 0x7fa3b0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:35.539 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.540+0000 7fa3ccc00640 1 -- 172.21.15.145:0/4153180622 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6172+0+0 (secure 0 0 0) 0x7fa3bc0b8270 con 0x7fa3d015fbc0 2024-05-09T04:35:35.539 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.540+0000 7fa3d5200640 1 --2- 172.21.15.145:0/4153180622 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fa3b00762d0 0x7fa3b0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:35.540 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.541+0000 7fa3d5200640 1 --2- 172.21.15.145:0/4153180622 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fa3b00762d0 0x7fa3b0078790 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4009670 tx=0x7fa3c4006bf0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:35.544 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.544+0000 7fa3ccc00640 1 -- 172.21.15.145:0/4153180622 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa3d013f340 con 0x7fa3d015fbc0 2024-05-09T04:35:35.792 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.793+0000 7fa3d7c00640 1 -- 172.21.15.145:0/4153180622 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3d00742c0 con 0x7fa3d015fbc0 2024-05-09T04:35:35.793 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.794+0000 7fa3ccc00640 1 -- 172.21.15.145:0/4153180622 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5189 (secure 0 0 0) 0x7fa3bc081a90 con 0x7fa3d015fbc0 2024-05-09T04:35:35.793 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:35:35.794 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4661,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6834/1033874179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":1033874179},{"type":"v1","addr":"172.21.15.192:6834","nonce":1033874179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4664,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1787962921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6833","nonce":1787962921},{"type":"v1","addr":"172.21.15.192:6835","nonce":1787962921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5313,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6836/65063948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":65063948},{"type":"v1","addr":"172.21.15.145:6836","nonce":65063948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T04:35:23.667303+0000","modified":"2024-05-09T04:35:34.804314+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5316},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5316":{"gid":5316,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.145:6837/2593908438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6835","nonce":2593908438},{"type":"v1","addr":"172.21.15.145:6837","nonce":2593908438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8,9],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5316,"qdb_cluster":[5316]},"id":3}]} 2024-05-09T04:35:35.794 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 28 2024-05-09T04:35:35.796 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.796+0000 7fa3d7c00640 1 -- 172.21.15.145:0/4153180622 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fa3b00762d0 msgr2=0x7fa3b0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:35.796 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.796+0000 7fa3d7c00640 1 --2- 172.21.15.145:0/4153180622 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fa3b00762d0 0x7fa3b0078790 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4009670 tx=0x7fa3c4006bf0 comp rx=0 tx=0).stop 2024-05-09T04:35:35.796 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.796+0000 7fa3d7c00640 1 -- 172.21.15.145:0/4153180622 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa3d015fbc0 msgr2=0x7fa3d015d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:35.797 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.796+0000 7fa3d7c00640 1 --2- 172.21.15.145:0/4153180622 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa3d015fbc0 0x7fa3d015d700 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc009870 tx=0x7fa3bc0046c0 comp rx=0 tx=0).stop 2024-05-09T04:35:35.797 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.797+0000 7fa3d7c00640 1 -- 172.21.15.145:0/4153180622 shutdown_connections 2024-05-09T04:35:35.797 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.797+0000 7fa3d7c00640 1 --2- 172.21.15.145:0/4153180622 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fa3b00762d0 0x7fa3b0078790 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:35.797 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.797+0000 7fa3d7c00640 1 --2- 172.21.15.145:0/4153180622 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa3d015ffa0 0x7fa3d0133380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:35.797 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.797+0000 7fa3d7c00640 1 --2- 172.21.15.145:0/4153180622 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa3d015fbc0 0x7fa3d015d700 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:35.797 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.797+0000 7fa3d7c00640 1 --2- 172.21.15.145:0/4153180622 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3d00cfe10 0x7fa3d0153350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:35.797 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.797+0000 7fa3d7c00640 1 -- 172.21.15.145:0/4153180622 >> 172.21.15.145:0/4153180622 conn(0x7fa3d006c870 msgr2=0x7fa3d00ce610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:35.797 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.797+0000 7fa3d7c00640 1 -- 172.21.15.145:0/4153180622 shutdown_connections 2024-05-09T04:35:35.797 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:35.797+0000 7fa3d7c00640 1 -- 172.21.15.145:0/4153180622 wait complete. 2024-05-09T04:35:35.815 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T04:35:36.017 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.017+0000 7eff05e00640 1 Processor -- start 2024-05-09T04:35:36.017 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.017+0000 7eff05e00640 1 -- start start 2024-05-09T04:35:36.019 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.019+0000 7eff05e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7eff0015bae0 0x7eff000721b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:36.019 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.019+0000 7eff05e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7eff0015bec0 0x7eff0015f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:36.019 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.019+0000 7eff05e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff00160280 con 0x7eff0015fde0 2024-05-09T04:35:36.019 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.019+0000 7eff05e00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7eff00160840 con 0x7eff0015bec0 2024-05-09T04:35:36.020 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.019+0000 7eff05e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7eff000b4190 con 0x7eff0015bae0 2024-05-09T04:35:36.020 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.019+0000 7efeffe00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7eff0015bec0 0x7eff0015f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:36.020 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.019+0000 7efeffe00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7eff0015bec0 0x7eff0015f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:56370/0 (socket says 172.21.15.145:56370) 2024-05-09T04:35:36.020 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.019+0000 7efeffe00640 1 -- 172.21.15.145:0/1639228571 learned_addr learned my addr 172.21.15.145:0/1639228571 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:36.020 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.020+0000 7eff04a00640 1 --2- 172.21.15.145:0/1639228571 >> v2:172.21.15.192:3300/0 conn(0x7eff0015bae0 0x7eff000721b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:36.020 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.020+0000 7efeffe00640 1 -- 172.21.15.145:0/1639228571 >> v2:172.21.15.192:3300/0 conn(0x7eff0015bae0 msgr2=0x7eff000721b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:36.020 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.020+0000 7efeffe00640 1 --2- 172.21.15.145:0/1639228571 >> v2:172.21.15.192:3300/0 conn(0x7eff0015bae0 0x7eff000721b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:36.020 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.020+0000 7efeffe00640 1 -- 172.21.15.145:0/1639228571 >> v1:172.21.15.192:6789/0 conn(0x7eff0015fde0 legacy=0x7eff000cfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:36.020 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.020+0000 7efeffe00640 1 -- 172.21.15.145:0/1639228571 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff00160560 con 0x7eff0015bec0 2024-05-09T04:35:36.021 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.021+0000 7efeffe00640 1 --2- 172.21.15.145:0/1639228571 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff0015bec0 0x7eff0015f8a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7eff0006eb80 tx=0x7efeec02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8c383e8f5072710 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:36.021 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.021+0000 7efeff400640 1 -- 172.21.15.145:0/1639228571 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeec037070 con 0x7eff0015bec0 2024-05-09T04:35:36.021 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.021+0000 7efeff400640 1 -- 172.21.15.145:0/1639228571 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efeec032440 con 0x7eff0015bec0 2024-05-09T04:35:36.021 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.021+0000 7efeff400640 1 -- 172.21.15.145:0/1639228571 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeec032810 con 0x7eff0015bec0 2024-05-09T04:35:36.022 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.022+0000 7eff05e00640 1 -- 172.21.15.145:0/1639228571 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff0015bec0 msgr2=0x7eff0015f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:36.022 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.022+0000 7eff05e00640 1 --2- 172.21.15.145:0/1639228571 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff0015bec0 0x7eff0015f8a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7eff0006eb80 tx=0x7efeec02a140 comp rx=0 tx=0).stop 2024-05-09T04:35:36.022 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.022+0000 7eff05e00640 1 -- 172.21.15.145:0/1639228571 shutdown_connections 2024-05-09T04:35:36.022 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.022+0000 7eff05e00640 1 --2- 172.21.15.145:0/1639228571 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff0015bec0 0x7eff0015f8a0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:36.022 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.022+0000 7eff05e00640 1 --2- 172.21.15.145:0/1639228571 >> v2:172.21.15.192:3300/0 conn(0x7eff0015bae0 0x7eff000721b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:36.022 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.022+0000 7eff05e00640 1 -- 172.21.15.145:0/1639228571 >> 172.21.15.145:0/1639228571 conn(0x7eff0006c8a0 msgr2=0x7eff0006ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:36.022 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.022+0000 7eff05e00640 1 -- 172.21.15.145:0/1639228571 shutdown_connections 2024-05-09T04:35:36.022 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.022+0000 7eff05e00640 1 -- 172.21.15.145:0/1639228571 wait complete. 2024-05-09T04:35:36.024 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.024+0000 7eff05e00640 1 Processor -- start 2024-05-09T04:35:36.024 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.024+0000 7eff05e00640 1 -- start start 2024-05-09T04:35:36.025 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.025+0000 7eff05e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff0015bae0 0x7eff0014c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:36.025 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.025+0000 7eff05e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff0015bec0 0x7eff0009da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:36.025 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.025+0000 7eff05e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7eff0015fde0 0x7eff0009df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:36.026 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.025+0000 7eff05e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7eff000aded0 con 0x7eff0015bec0 2024-05-09T04:35:36.026 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.025+0000 7eff05e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7eff000726f0 con 0x7eff0015bae0 2024-05-09T04:35:36.026 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.025+0000 7eff05e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7eff000ae050 con 0x7eff0015fde0 2024-05-09T04:35:36.026 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.026+0000 7efeffe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff0015bec0 0x7eff0009da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:36.026 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.026+0000 7efeffe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff0015bec0 0x7eff0009da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:56384/0 (socket says 172.21.15.145:56384) 2024-05-09T04:35:36.026 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.026+0000 7efeffe00640 1 -- 172.21.15.145:0/1159790124 learned_addr learned my addr 172.21.15.145:0/1159790124 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:36.027 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.026+0000 7eff04a00640 1 --2- 172.21.15.145:0/1159790124 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff0015bae0 0x7eff0014c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:36.027 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.026+0000 7eff05400640 1 --2- 172.21.15.145:0/1159790124 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7eff0015fde0 0x7eff0009df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:36.027 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.026+0000 7efeffe00640 1 -- 172.21.15.145:0/1159790124 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7eff0015fde0 msgr2=0x7eff0009df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:36.027 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.026+0000 7efeffe00640 1 --2- 172.21.15.145:0/1159790124 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7eff0015fde0 0x7eff0009df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:36.027 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.026+0000 7efeffe00640 1 -- 172.21.15.145:0/1159790124 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff0015bae0 msgr2=0x7eff0014c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:36.027 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.026+0000 7efeffe00640 1 --2- 172.21.15.145:0/1159790124 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff0015bae0 0x7eff0014c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:36.027 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.026+0000 7efeffe00640 1 -- 172.21.15.145:0/1159790124 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff00146d50 con 0x7eff0015bec0 2024-05-09T04:35:36.028 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.027+0000 7efeffe00640 1 --2- 172.21.15.145:0/1159790124 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff0015bec0 0x7eff0009da20 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7efeec02a180 tx=0x7efeec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:36.028 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.027+0000 7efefcc00640 1 -- 172.21.15.145:0/1159790124 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeec037040 con 0x7eff0015bec0 2024-05-09T04:35:36.028 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.027+0000 7efefcc00640 1 -- 172.21.15.145:0/1159790124 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efeec042070 con 0x7eff0015bec0 2024-05-09T04:35:36.028 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.027+0000 7efefcc00640 1 -- 172.21.15.145:0/1159790124 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeec030bb0 con 0x7eff0015bec0 2024-05-09T04:35:36.028 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.027+0000 7eff05e00640 1 -- 172.21.15.145:0/1159790124 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff00147030 con 0x7eff0015bec0 2024-05-09T04:35:36.028 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.027+0000 7eff05e00640 1 -- 172.21.15.145:0/1159790124 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff001475f0 con 0x7eff0015bec0 2024-05-09T04:35:36.029 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.028+0000 7eff05e00640 1 -- 172.21.15.145:0/1159790124 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff0006a310 con 0x7eff0015bec0 2024-05-09T04:35:36.030 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.030+0000 7efefcc00640 1 -- 172.21.15.145:0/1159790124 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7efeec047460 con 0x7eff0015bec0 2024-05-09T04:35:36.031 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.030+0000 7efefcc00640 1 --2- 172.21.15.145:0/1159790124 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7efee00762d0 0x7efee0078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:36.031 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.031+0000 7efefcc00640 1 -- 172.21.15.145:0/1159790124 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6172+0+0 (secure 0 0 0) 0x7efeec0b84d0 con 0x7eff0015bec0 2024-05-09T04:35:36.031 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.031+0000 7eff04a00640 1 --2- 172.21.15.145:0/1159790124 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7efee00762d0 0x7efee0078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:36.031 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.032+0000 7eff04a00640 1 --2- 172.21.15.145:0/1159790124 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7efee00762d0 0x7efee0078790 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7efef0006fd0 tx=0x7efef0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:36.034 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.035+0000 7efefcc00640 1 -- 172.21.15.145:0/1159790124 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7efeec081bc0 con 0x7eff0015bec0 2024-05-09T04:35:36.237 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.237+0000 7eff05e00640 1 -- 172.21.15.145:0/1159790124 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7eff00070360 con 0x7eff0015bec0 2024-05-09T04:35:36.241 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.241+0000 7efefcc00640 1 -- 172.21.15.145:0/1159790124 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) ==== 74+0+18646 (secure 0 0 0) 0x7efeec0814c0 con 0x7eff0015bec0 2024-05-09T04:35:36.241 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:35:36.241 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":50,"fsid":"957ee6c1-073b-44cf-9b93-9491b3fdfc1d","created":"2024-05-09T04:33:26.544367+0000","modified":"2024-05-09T04:35:34.804055+0000","last_up_change":"2024-05-09T04:33:39.039318+0000","last_in_change":"2024-05-09T04:33:29.902361+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T04:33:39.795344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-09T04:35:21.570048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-09T04:35:22.639737+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data_extra1","create_time":"2024-05-09T04:35:33.506234+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22e550ad-847b-4952-80e6-63d859ed1a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6801","nonce":3182572912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6803","nonce":3182572912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6807","nonce":3182572912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6805","nonce":3182572912}]},"public_addr":"172.21.15.145:6801/3182572912","cluster_addr":"172.21.15.145:6803/3182572912","heartbeat_back_addr":"172.21.15.145:6807/3182572912","heartbeat_front_addr":"172.21.15.145:6805/3182572912","state":["exists","up"]},{"osd":1,"uuid":"0f6b9fca-0c41-4145-8d74-598ddb42f2a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6825","nonce":1845225555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6827","nonce":1845225555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6831","nonce":1845225555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6829","nonce":1845225555}]},"public_addr":"172.21.15.145:6825/1845225555","cluster_addr":"172.21.15.145:6827/1845225555","heartbeat_back_addr":"172.21.15.145:6831/1845225555","heartbeat_front_addr":"172.21.15.145:6829/1845225555","state":["exists","up"]},{"osd":2,"uuid":"b3b1f67e-5790-483f-8610-e05a5bce4c30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6817","nonce":2168958993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6819","nonce":2168958993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6823","nonce":2168958993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6821","nonce":2168958993}]},"public_addr":"172.21.15.145:6817/2168958993","cluster_addr":"172.21.15.145:6819/2168958993","heartbeat_back_addr":"172.21.15.145:6823/2168958993","heartbeat_front_addr":"172.21.15.145:6821/2168958993","state":["exists","up"]},{"osd":3,"uuid":"884787a5-e037-4d31-9aae-743030eba4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6809","nonce":337009826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6811","nonce":337009826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6815","nonce":337009826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6813","nonce":337009826}]},"public_addr":"172.21.15.145:6809/337009826","cluster_addr":"172.21.15.145:6811/337009826","heartbeat_back_addr":"172.21.15.145:6815/337009826","heartbeat_front_addr":"172.21.15.145:6813/337009826","state":["exists","up"]},{"osd":4,"uuid":"0759fc1e-c5fc-4a2f-86bb-e8eccd83f833","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6818","nonce":3117700995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6821","nonce":3117700995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6827","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6829","nonce":3117700995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6823","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6825","nonce":3117700995}]},"public_addr":"172.21.15.192:6818/3117700995","cluster_addr":"172.21.15.192:6821/3117700995","heartbeat_back_addr":"172.21.15.192:6829/3117700995","heartbeat_front_addr":"172.21.15.192:6825/3117700995","state":["exists","up"]},{"osd":5,"uuid":"9443cc22-9379-46e3-9173-98b592bf2c16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6801","nonce":1328373268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6803","nonce":1328373268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6807","nonce":1328373268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6805","nonce":1328373268}]},"public_addr":"172.21.15.192:6801/1328373268","cluster_addr":"172.21.15.192:6803/1328373268","heartbeat_back_addr":"172.21.15.192:6807/1328373268","heartbeat_front_addr":"172.21.15.192:6805/1328373268","state":["exists","up"]},{"osd":6,"uuid":"d05a3311-505a-45b5-85b9-aa3113e96987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6817","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6820","nonce":1868011057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6824","nonce":1868011057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6831","nonce":1868011057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6828","nonce":1868011057}]},"public_addr":"172.21.15.192:6820/1868011057","cluster_addr":"172.21.15.192:6824/1868011057","heartbeat_back_addr":"172.21.15.192:6831/1868011057","heartbeat_front_addr":"172.21.15.192:6828/1868011057","state":["exists","up"]},{"osd":7,"uuid":"c7c23dbc-2029-43ec-9f53-25f3d58019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6809","nonce":3368512157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6811","nonce":3368512157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6815","nonce":3368512157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6813","nonce":3368512157}]},"public_addr":"172.21.15.192:6809/3368512157","cluster_addr":"172.21.15.192:6811/3368512157","heartbeat_back_addr":"172.21.15.192:6815/3368512157","heartbeat_front_addr":"172.21.15.192:6813/3368512157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.091188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.032567+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:34.797468+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.228839+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.622400+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.473103+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.566092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.349395+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.c","mappings":[{"from":0,"to":6}]},{"pgid":"8.e","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T04:35:36.244 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.243+0000 7eff05e00640 1 -- 172.21.15.145:0/1159790124 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7efee00762d0 msgr2=0x7efee0078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:36.244 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.243+0000 7eff05e00640 1 --2- 172.21.15.145:0/1159790124 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7efee00762d0 0x7efee0078790 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7efef0006fd0 tx=0x7efef0008040 comp rx=0 tx=0).stop 2024-05-09T04:35:36.244 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.243+0000 7eff05e00640 1 -- 172.21.15.145:0/1159790124 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff0015bec0 msgr2=0x7eff0009da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:36.244 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.243+0000 7eff05e00640 1 --2- 172.21.15.145:0/1159790124 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff0015bec0 0x7eff0009da20 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7efeec02a180 tx=0x7efeec0046c0 comp rx=0 tx=0).stop 2024-05-09T04:35:36.244 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.244+0000 7eff05e00640 1 -- 172.21.15.145:0/1159790124 shutdown_connections 2024-05-09T04:35:36.244 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.244+0000 7eff05e00640 1 --2- 172.21.15.145:0/1159790124 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7efee00762d0 0x7efee0078790 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:36.244 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.244+0000 7eff05e00640 1 --2- 172.21.15.145:0/1159790124 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7eff0015fde0 0x7eff0009df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:36.244 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.244+0000 7eff05e00640 1 --2- 172.21.15.145:0/1159790124 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7eff0015bec0 0x7eff0009da20 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:36.244 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.244+0000 7eff05e00640 1 --2- 172.21.15.145:0/1159790124 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff0015bae0 0x7eff0014c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:36.244 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.244+0000 7eff05e00640 1 -- 172.21.15.145:0/1159790124 >> 172.21.15.145:0/1159790124 conn(0x7eff0006c8a0 msgr2=0x7eff000b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:36.244 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.244+0000 7eff05e00640 1 -- 172.21.15.145:0/1159790124 shutdown_connections 2024-05-09T04:35:36.245 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.244+0000 7eff05e00640 1 -- 172.21.15.145:0/1159790124 wait complete. 2024-05-09T04:35:36.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:35:36.261 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir_x) 2024-05-09T04:35:36.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:35:36.311 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.layout.pool -v cephfs_data_extra1 dir_x) 2024-05-09T04:35:36.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:35:36.359 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir_x/file_x) 2024-05-09T04:35:36.408 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 flush journal 2024-05-09T04:35:36.597 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.597+0000 7f9ab6800640 1 Processor -- start 2024-05-09T04:35:36.598 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.598+0000 7f9ab6800640 1 -- start start 2024-05-09T04:35:36.599 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.599+0000 7f9ab6800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f9ab015f4c0 0x7f9ab00cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:36.599 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.599+0000 7f9ab6800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9ab00712f0 0x7f9ab0070650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:36.599 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.599+0000 7f9ab6800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ab006d7a0 con 0x7f9ab00701f0 2024-05-09T04:35:36.599 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.599+0000 7f9ab6800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f9ab00d2570 con 0x7f9ab015f4c0 2024-05-09T04:35:36.599 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.599+0000 7f9ab6800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9ab015f8a0 con 0x7f9ab00712f0 2024-05-09T04:35:36.599 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.600+0000 7f9aaea00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f9ab00701f0 0x7f9ab0070be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:52740/0 (socket says 172.21.15.145:52740) 2024-05-09T04:35:36.599 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.600+0000 7f9aaea00640 1 -- 172.21.15.145:0/4154700730 learned_addr learned my addr 172.21.15.145:0/4154700730 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:35:36.601 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.600+0000 7f9aafe00640 1 --2- 172.21.15.145:0/4154700730 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9ab00712f0 0x7f9ab0070650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:36.601 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.600+0000 7f9aae000640 1 -- 172.21.15.145:0/4154700730 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1434858841 0 0) 0x7f9ab006d7a0 con 0x7f9ab00701f0 2024-05-09T04:35:36.601 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.600+0000 7f9aaf400640 1 --2- 172.21.15.145:0/4154700730 >> v2:172.21.15.192:3300/0 conn(0x7f9ab015f4c0 0x7f9ab00cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:36.601 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.600+0000 7f9aae000640 1 -- 172.21.15.145:0/4154700730 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a94003520 con 0x7f9ab00701f0 2024-05-09T04:35:36.602 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.600+0000 7f9aae000640 1 -- 172.21.15.145:0/4154700730 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3605892240 0 0) 0x7f9a94003520 con 0x7f9ab00701f0 2024-05-09T04:35:36.602 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.600+0000 7f9aae000640 1 -- 172.21.15.145:0/4154700730 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9ab00712f0 msgr2=0x7f9ab0070650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:36.602 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.600+0000 7f9aae000640 1 --2- 172.21.15.145:0/4154700730 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9ab00712f0 0x7f9ab0070650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:36.602 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.600+0000 7f9aae000640 1 -- 172.21.15.145:0/4154700730 >> v2:172.21.15.192:3300/0 conn(0x7f9ab015f4c0 msgr2=0x7f9ab00cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:36.602 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.600+0000 7f9aae000640 1 --2- 172.21.15.145:0/4154700730 >> v2:172.21.15.192:3300/0 conn(0x7f9ab015f4c0 0x7f9ab00cd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:36.602 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.600+0000 7f9aae000640 1 -- 172.21.15.145:0/4154700730 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ab006da80 con 0x7f9ab00701f0 2024-05-09T04:35:36.602 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.600+0000 7f9aae000640 1 -- 172.21.15.145:0/4154700730 <== mon.0 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3474413261 0 0) 0x7f9a980016f0 con 0x7f9ab00701f0 2024-05-09T04:35:36.602 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.600+0000 7f9aafe00640 1 --2- 172.21.15.145:0/4154700730 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9ab00712f0 0x7f9ab0070650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:35:36.602 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.601+0000 7f9aae000640 1 -- 172.21.15.145:0/4154700730 >> v1:172.21.15.145:6789/0 conn(0x7f9ab00701f0 legacy=0x7f9ab0070be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:36.602 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.601+0000 7f9aae000640 1 --2- 172.21.15.145:0/4154700730 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9a94003a00 0x7f9a94003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:36.602 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.601+0000 7f9aae000640 1 --2- 172.21.15.145:0/4154700730 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a94004800 0x7f9a94004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:36.602 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.601+0000 7f9aae000640 1 --2- 172.21.15.145:0/4154700730 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9ab00712f0 0x7f9a94005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:36.602 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.601+0000 7f9aae000640 1 -- 172.21.15.145:0/4154700730 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab015f8a0 con 0x7f9a94003a00 2024-05-09T04:35:36.602 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.601+0000 7f9aae000640 1 -- 172.21.15.145:0/4154700730 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9a9400bb50 con 0x7f9a94004800 2024-05-09T04:35:36.603 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.601+0000 7f9aae000640 1 -- 172.21.15.145:0/4154700730 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9a9400bd50 con 0x7f9ab00712f0 2024-05-09T04:35:36.603 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.601+0000 7f9aaf400640 1 --2- 172.21.15.145:0/4154700730 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9a94003a00 0x7f9a94003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:36.603 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.601+0000 7f9aaea00640 1 --2- 172.21.15.145:0/4154700730 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a94004800 0x7f9a94004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:36.603 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.601+0000 7f9aaf400640 1 -- 172.21.15.145:0/4154700730 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9ab00712f0 msgr2=0x7f9a94005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:36.603 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.601+0000 7f9aaf400640 1 --2- 172.21.15.145:0/4154700730 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9ab00712f0 0x7f9a94005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:36.603 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.601+0000 7f9aaf400640 1 -- 172.21.15.145:0/4154700730 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a94004800 msgr2=0x7f9a94004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:36.603 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.601+0000 7f9aaf400640 1 --2- 172.21.15.145:0/4154700730 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a94004800 0x7f9a94004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:36.603 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.601+0000 7f9aaf400640 1 -- 172.21.15.145:0/4154700730 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a94003520 con 0x7f9a94003a00 2024-05-09T04:35:36.603 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.601+0000 7f9aaea00640 1 --2- 172.21.15.145:0/4154700730 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a94004800 0x7f9a94004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:35:36.604 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.602+0000 7f9aaf400640 1 --2- 172.21.15.145:0/4154700730 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9a94003a00 0x7f9a94003e20 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0007360 tx=0x7f9a94005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=210787a66697cd29 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:36.604 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.603+0000 7f9aae000640 1 -- 172.21.15.145:0/4154700730 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa0009be0 con 0x7f9a94003a00 2024-05-09T04:35:36.605 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.603+0000 7f9aae000640 1 -- 172.21.15.145:0/4154700730 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9aa002e620 con 0x7f9a94003a00 2024-05-09T04:35:36.605 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.603+0000 7f9aae000640 1 -- 172.21.15.145:0/4154700730 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa002e9f0 con 0x7f9a94003a00 2024-05-09T04:35:36.605 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.603+0000 7f9ab6800640 1 -- 172.21.15.145:0/4154700730 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9a94003a00 msgr2=0x7f9a94003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:36.605 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.603+0000 7f9ab6800640 1 --2- 172.21.15.145:0/4154700730 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9a94003a00 0x7f9a94003e20 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0007360 tx=0x7f9a94005830 comp rx=0 tx=0).stop 2024-05-09T04:35:36.605 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.603+0000 7f9aafe00640 1 -- 172.21.15.145:0/4154700730 reap_dead start 2024-05-09T04:35:36.605 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.603+0000 7f9ab6800640 1 -- 172.21.15.145:0/4154700730 shutdown_connections 2024-05-09T04:35:36.605 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.603+0000 7f9ab6800640 1 -- 172.21.15.145:0/4154700730 >> 172.21.15.145:0/4154700730 conn(0x7f9ab00b4af0 msgr2=0x7f9ab00b4ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:36.605 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.603+0000 7f9ab6800640 1 -- 172.21.15.145:0/4154700730 shutdown_connections 2024-05-09T04:35:36.605 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.603+0000 7f9ab6800640 1 -- 172.21.15.145:0/4154700730 wait complete. 2024-05-09T04:35:36.605 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.605+0000 7f9ab6800640 1 Processor -- start 2024-05-09T04:35:36.607 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.605+0000 7f9ab6800640 1 -- start start 2024-05-09T04:35:36.608 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.606+0000 7f9ab6800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a94003a00 0x7f9ab01541d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:36.608 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.606+0000 7f9ab6800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9ab014d4c0 0x7f9ab014d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:36.608 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.606+0000 7f9ab6800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9ab0150430 0x7f9ab01508c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:36.608 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.606+0000 7f9ab6800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab006e320 con 0x7f9ab0150430 2024-05-09T04:35:36.608 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.606+0000 7f9ab6800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab006e040 con 0x7f9a94003a00 2024-05-09T04:35:36.608 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.606+0000 7f9ab6800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9ab0070650 con 0x7f9ab014d4c0 2024-05-09T04:35:36.608 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.607+0000 7f9aafe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9ab0150430 0x7f9ab01508c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:36.608 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.607+0000 7f9aafe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9ab0150430 0x7f9ab01508c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:56402/0 (socket says 172.21.15.145:56402) 2024-05-09T04:35:36.608 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.607+0000 7f9aafe00640 1 -- 172.21.15.145:0/1440418597 learned_addr learned my addr 172.21.15.145:0/1440418597 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:36.608 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.607+0000 7f9aaea00640 1 --2- 172.21.15.145:0/1440418597 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9ab014d4c0 0x7f9ab014d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:36.608 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.607+0000 7f9aaf400640 1 --2- 172.21.15.145:0/1440418597 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a94003a00 0x7f9ab01541d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:36.608 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.607+0000 7f9aafe00640 1 -- 172.21.15.145:0/1440418597 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9ab014d4c0 msgr2=0x7f9ab014d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:36.608 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.607+0000 7f9aafe00640 1 --2- 172.21.15.145:0/1440418597 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9ab014d4c0 0x7f9ab014d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:36.608 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.607+0000 7f9aafe00640 1 -- 172.21.15.145:0/1440418597 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a94003a00 msgr2=0x7f9ab01541d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:36.609 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.607+0000 7f9aafe00640 1 --2- 172.21.15.145:0/1440418597 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a94003a00 0x7f9ab01541d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:36.609 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.607+0000 7f9aafe00640 1 -- 172.21.15.145:0/1440418597 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ab00c8c10 con 0x7f9ab0150430 2024-05-09T04:35:36.609 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.607+0000 7f9aaf400640 1 --2- 172.21.15.145:0/1440418597 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a94003a00 0x7f9ab01541d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:35:36.609 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.607+0000 7f9aafe00640 1 --2- 172.21.15.145:0/1440418597 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9ab0150430 0x7f9ab01508c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f9ab00b2a30 tx=0x7f9aa800e830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:36.609 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.608+0000 7f9aa7400640 1 -- 172.21.15.145:0/1440418597 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa800f040 con 0x7f9ab0150430 2024-05-09T04:35:36.609 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.608+0000 7f9aa7400640 1 -- 172.21.15.145:0/1440418597 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9aa8019070 con 0x7f9ab0150430 2024-05-09T04:35:36.609 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.608+0000 7f9aa7400640 1 -- 172.21.15.145:0/1440418597 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa8013810 con 0x7f9ab0150430 2024-05-09T04:35:36.609 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.608+0000 7f9ab6800640 1 -- 172.21.15.145:0/1440418597 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ab00c8eb0 con 0x7f9ab0150430 2024-05-09T04:35:36.609 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.608+0000 7f9ab6800640 1 -- 172.21.15.145:0/1440418597 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ab00c9470 con 0x7f9ab0150430 2024-05-09T04:35:36.610 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.610+0000 7f9aa7400640 1 -- 172.21.15.145:0/1440418597 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f9aa80139b0 con 0x7f9ab0150430 2024-05-09T04:35:36.611 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.611+0000 7f9aa7400640 1 --2- 172.21.15.145:0/1440418597 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f9a8c0762d0 0x7f9a8c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:36.611 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.611+0000 7f9aa7400640 1 -- 172.21.15.145:0/1440418597 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6172+0+0 (secure 0 0 0) 0x7f9aa8096fe0 con 0x7f9ab0150430 2024-05-09T04:35:36.611 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.611+0000 7f9aaf400640 1 --2- 172.21.15.145:0/1440418597 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f9a8c0762d0 0x7f9a8c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:36.611 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.611+0000 7f9ab97b8740 1 Processor -- start 2024-05-09T04:35:36.611 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.612+0000 7f9ab97b8740 1 -- start start 2024-05-09T04:35:36.612 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.612+0000 7f9aaf400640 1 --2- 172.21.15.145:0/1440418597 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f9a8c0762d0 0x7f9a8c078790 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0007360 tx=0x7f9aa0038040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:36.612 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.613+0000 7f9ab97b8740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x556915200d40 0x5569151b36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:36.612 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.613+0000 7f9ab97b8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569151e3e60 0x5569151e31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:36.613 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.613+0000 7f9ab97b8740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5569151e3720 0x5569151b8530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:36.613 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.613+0000 7f9ab97b8740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x55691511dd50 con 0x5569151e3720 2024-05-09T04:35:36.613 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.613+0000 7f9ab97b8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5569150c2ef0 con 0x5569151e3e60 2024-05-09T04:35:36.613 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.613+0000 7f9ab97b8740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x556915119d10 con 0x556915200d40 2024-05-09T04:35:36.613 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.613+0000 7f9aafe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5569151e3720 0x5569151b8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:36.613 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.613+0000 7f9aafe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5569151e3720 0x5569151b8530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:56414/0 (socket says 172.21.15.145:56414) 2024-05-09T04:35:36.613 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.613+0000 7f9aafe00640 1 -- 172.21.15.145:0/2837836559 learned_addr learned my addr 172.21.15.145:0/2837836559 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:36.615 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.613+0000 7f9aafe00640 1 -- 172.21.15.145:0/2837836559 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x556915200d40 msgr2=0x5569151b36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:36.615 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.613+0000 7f9aafe00640 1 --2- 172.21.15.145:0/2837836559 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x556915200d40 0x5569151b36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:36.615 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.613+0000 7f9aafe00640 1 -- 172.21.15.145:0/2837836559 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569151e3e60 msgr2=0x5569151e31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:36.615 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.613+0000 7f9aafe00640 1 --2- 172.21.15.145:0/2837836559 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569151e3e60 0x5569151e31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:36.615 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.613+0000 7f9aafe00640 1 -- 172.21.15.145:0/2837836559 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556914ef97d0 con 0x5569151e3720 2024-05-09T04:35:36.615 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.613+0000 7f9aafe00640 1 --2- 172.21.15.145:0/2837836559 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5569151e3720 0x5569151b8530 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8095210 tx=0x7f9aa805fa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10fa5d168601bf12 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:36.615 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.614+0000 7f9a9fe00640 1 -- 172.21.15.145:0/2837836559 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa805ffd0 con 0x5569151e3720 2024-05-09T04:35:36.615 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.614+0000 7f9a9fe00640 1 -- 172.21.15.145:0/2837836559 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9aa8064380 con 0x5569151e3720 2024-05-09T04:35:36.615 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.614+0000 7f9a9fe00640 1 -- 172.21.15.145:0/2837836559 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa805f450 con 0x5569151e3720 2024-05-09T04:35:36.616 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.614+0000 7f9ab97b8740 1 -- 172.21.15.145:0/2837836559 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5569151e3720 msgr2=0x5569151b8530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:36.616 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.614+0000 7f9ab97b8740 1 --2- 172.21.15.145:0/2837836559 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5569151e3720 0x5569151b8530 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8095210 tx=0x7f9aa805fa20 comp rx=0 tx=0).stop 2024-05-09T04:35:36.616 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.614+0000 7f9ab97b8740 1 -- 172.21.15.145:0/2837836559 shutdown_connections 2024-05-09T04:35:36.616 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.614+0000 7f9ab97b8740 1 --2- 172.21.15.145:0/2837836559 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5569151e3720 0x5569151b8530 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:36.616 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.614+0000 7f9ab97b8740 1 --2- 172.21.15.145:0/2837836559 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569151e3e60 0x5569151e31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:36.616 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.614+0000 7f9ab97b8740 1 --2- 172.21.15.145:0/2837836559 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x556915200d40 0x5569151b36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:36.616 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.614+0000 7f9ab97b8740 1 -- 172.21.15.145:0/2837836559 >> 172.21.15.145:0/2837836559 conn(0x5569151b5940 msgr2=0x5569151fe1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:36.616 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.615+0000 7f9ab97b8740 1 -- 172.21.15.145:0/2837836559 shutdown_connections 2024-05-09T04:35:36.616 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.615+0000 7f9ab97b8740 1 -- 172.21.15.145:0/2837836559 wait complete. 2024-05-09T04:35:36.616 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.615+0000 7f9ab97b8740 1 -- start start 2024-05-09T04:35:36.620 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.616+0000 7f9ab97b8740 1 Processor -- start 2024-05-09T04:35:36.620 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.616+0000 7f9ab97b8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569151e3e60 0x5569151e5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:36.621 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.617+0000 7f9ab97b8740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x556915200d40 0x5569151f4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:36.621 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.617+0000 7f9ab97b8740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5569151f53e0 0x556915211b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:36.621 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.617+0000 7f9ab97b8740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5569151b7af0 con 0x5569151f53e0 2024-05-09T04:35:36.621 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.617+0000 7f9ab97b8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x5569150d6990 con 0x5569151e3e60 2024-05-09T04:35:36.621 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.617+0000 7f9ab97b8740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x5569151b6ee0 con 0x556915200d40 2024-05-09T04:35:36.621 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.617+0000 7f9ab97b8740 10 client.0 _ll_register_callbacks cb 0x5569150d5110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:35:36.621 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.617+0000 7f9aafe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5569151f53e0 0x556915211b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:36.621 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.617+0000 7f9aafe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5569151f53e0 0x556915211b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:56428/0 (socket says 172.21.15.145:56428) 2024-05-09T04:35:36.621 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.617+0000 7f9aafe00640 1 -- 172.21.15.145:0/1979045587 learned_addr learned my addr 172.21.15.145:0/1979045587 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:36.621 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.617+0000 7f9aafe00640 1 -- 172.21.15.145:0/1979045587 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x556915200d40 msgr2=0x5569151f4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:36.621 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.617+0000 7f9aafe00640 1 --2- 172.21.15.145:0/1979045587 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x556915200d40 0x5569151f4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:36.621 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.617+0000 7f9aafe00640 1 -- 172.21.15.145:0/1979045587 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569151e3e60 msgr2=0x5569151e5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:36.622 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.617+0000 7f9aafe00640 1 --2- 172.21.15.145:0/1979045587 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569151e3e60 0x5569151e5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:36.622 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.617+0000 7f9aafe00640 1 -- 172.21.15.145:0/1979045587 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556914e2f3f0 con 0x5569151f53e0 2024-05-09T04:35:36.622 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.617+0000 7f9aafe00640 1 --2- 172.21.15.145:0/1979045587 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5569151f53e0 0x556915211b40 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8006da0 tx=0x7f9aa8006d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70c7c397d2d70dde server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:36.622 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.617+0000 7f9a9cc00640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-05-09T04:35:36.622 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.618+0000 7f9a9cc00640 1 -- 172.21.15.145:0/1979045587 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa807e060 con 0x5569151f53e0 2024-05-09T04:35:36.622 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.618+0000 7f9a9cc00640 1 -- 172.21.15.145:0/1979045587 <== mon.0 v2:172.21.15.145:3300/0 2 ==== osd_map(51..51 src has 1..51) ==== 6172+0+0 (secure 0 0 0) 0x7f9aa805eca0 con 0x5569151f53e0 2024-05-09T04:35:36.622 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.618+0000 7f9ab97b8740 1 -- 172.21.15.145:0/1979045587 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9aa8064380 con 0x5569151f53e0 2024-05-09T04:35:36.622 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.618+0000 7f9a9cc00640 1 -- 172.21.15.145:0/1979045587 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9aa8068da0 con 0x5569151f53e0 2024-05-09T04:35:36.622 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.618+0000 7f9ab97b8740 10 client.5523 fetch_fsmap learned FSMap version 28 2024-05-09T04:35:36.622 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.618+0000 7f9ab97b8740 1 -- 172.21.15.145:0/1979045587 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) -- 0x556914def0f0 con 0x5569151f53e0 2024-05-09T04:35:36.622 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.618+0000 7f9a9cc00640 1 -- 172.21.15.145:0/1979045587 <== mon.0 v2:172.21.15.145:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9aa8064380 con 0x5569151f53e0 2024-05-09T04:35:36.622 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.618+0000 7f9a9cc00640 1 -- 172.21.15.145:0/1979045587 <== mon.0 v2:172.21.15.145:3300/0 5 ==== fsmap(e 28) ==== 3502+0+0 (secure 0 0 0) 0x7f9aa8012050 con 0x5569151f53e0 2024-05-09T04:35:36.622 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.618+0000 7f9ab97b8740 10 client.5523 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T04:35:36.622 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.618+0000 7f9ab97b8740 10 client.5523 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-05-09T04:35:36.622 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.618+0000 7f9ab97b8740 20 client.5523 populate_metadata read hostname 'smithi145' 2024-05-09T04:35:36.623 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.618+0000 7f9ab97b8740 1 --2- 172.21.15.145:0/1979045587 >> [v2:172.21.15.145:6835/2593908438,v1:172.21.15.145:6837/2593908438] conn(0x556915212080 0x5569151fbd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:36.623 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.618+0000 7f9ab97b8740 4 client.5523 mds_command: new command op to 5316 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:35:36.623 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.618+0000 7f9ab97b8740 1 -- 172.21.15.145:0/1979045587 --> [v2:172.21.15.145:6835/2593908438,v1:172.21.15.145:6837/2593908438] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556915131ef0 con 0x556915212080 2024-05-09T04:35:36.623 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.619+0000 7f9aaf400640 1 --2- 172.21.15.145:0/1979045587 >> [v2:172.21.15.145:6835/2593908438,v1:172.21.15.145:6837/2593908438] conn(0x556915212080 0x5569151fbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:36.623 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.619+0000 7f9aaf400640 1 --2- 172.21.15.145:0/1979045587 >> [v2:172.21.15.145:6835/2593908438,v1:172.21.15.145:6837/2593908438] conn(0x556915212080 0x5569151fbd90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffdc3ba2a8317eb4 server_cookie=ef9addd9824eb8a in_seq=0 out_seq=0 2024-05-09T04:35:36.623 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.619+0000 7f9a9cc00640 10 client.5523 ms_handle_connect on v2:172.21.15.145:6835/2593908438 2024-05-09T04:35:36.623 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.620+0000 7f9a9cc00640 1 -- 172.21.15.145:0/1979045587 <== mds.0 v2:172.21.15.145:6835/2593908438 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556915131ef0 con 0x556915212080 2024-05-09T04:35:36.623 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.620+0000 7f9a9cc00640 10 client.5523 handle_command_reply: tid=0 2024-05-09T04:35:36.624 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.620+0000 7f9ab97b8740 1 client.5523 shutdown 2024-05-09T04:35:36.624 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.620+0000 7f9ab97b8740 1 -- 172.21.15.145:0/1979045587 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5569151f53e0 msgr2=0x556915211b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:36.624 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.620+0000 7f9ab97b8740 1 --2- 172.21.15.145:0/1979045587 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5569151f53e0 0x556915211b40 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8006da0 tx=0x7f9aa8006d10 comp rx=0 tx=0).stop 2024-05-09T04:35:36.624 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.620+0000 7f9ab97b8740 1 -- 172.21.15.145:0/1979045587 shutdown_connections 2024-05-09T04:35:36.624 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.620+0000 7f9ab97b8740 1 --2- 172.21.15.145:0/1979045587 >> [v2:172.21.15.145:6835/2593908438,v1:172.21.15.145:6837/2593908438] conn(0x556915212080 0x5569151fbd90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:36.624 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.620+0000 7f9ab97b8740 1 --2- 172.21.15.145:0/1979045587 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5569151f53e0 0x556915211b40 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:36.624 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.620+0000 7f9ab97b8740 1 --2- 172.21.15.145:0/1979045587 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x556915200d40 0x5569151f4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:36.624 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.620+0000 7f9ab97b8740 1 --2- 172.21.15.145:0/1979045587 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569151e3e60 0x5569151e5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:36.624 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.620+0000 7f9a9cc00640 0 client.5523 ms_handle_reset on v2:172.21.15.145:6835/2593908438 2024-05-09T04:35:36.624 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.620+0000 7f9ab97b8740 1 -- 172.21.15.145:0/1979045587 >> 172.21.15.145:0/1979045587 conn(0x5569151e3720 msgr2=0x5569151fe1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:36.624 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.621+0000 7f9ab97b8740 1 -- 172.21.15.145:0/1979045587 shutdown_connections 2024-05-09T04:35:36.624 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.621+0000 7f9ab97b8740 1 -- 172.21.15.145:0/1979045587 wait complete. 2024-05-09T04:35:36.624 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.621+0000 7f9ab97b8740 20 client.5523 trim_cache size 0 max 16384 2024-05-09T04:35:36.640 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.637+0000 7f9ab97b8740 1 Processor -- start 2024-05-09T04:35:36.640 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.637+0000 7f9ab97b8740 1 -- start start 2024-05-09T04:35:36.640 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.637+0000 7f9ab97b8740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5569151e3e60 0x5569151e68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:36.640 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.638+0000 7f9ab97b8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569151f53e0 0x556915222500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:36.640 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.638+0000 7f9ab97b8740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556915200d40 0x5569151fcd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:36.640 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.638+0000 7f9ab97b8740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab0202d50 con 0x556915200d40 2024-05-09T04:35:36.640 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.638+0000 7f9ab97b8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab0201d90 con 0x5569151f53e0 2024-05-09T04:35:36.640 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.638+0000 7f9ab97b8740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f9ab02010f0 con 0x5569151e3e60 2024-05-09T04:35:36.640 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.638+0000 7f9aafe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556915200d40 0x5569151fcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:36.640 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.638+0000 7f9aafe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556915200d40 0x5569151fcd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:56440/0 (socket says 172.21.15.145:56440) 2024-05-09T04:35:36.640 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.638+0000 7f9aafe00640 1 -- 172.21.15.145:0/3919843038 learned_addr learned my addr 172.21.15.145:0/3919843038 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:36.640 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.638+0000 7f9aafe00640 1 -- 172.21.15.145:0/3919843038 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5569151e3e60 msgr2=0x5569151e68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:36.641 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.638+0000 7f9aafe00640 1 --2- 172.21.15.145:0/3919843038 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5569151e3e60 0x5569151e68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:36.641 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.638+0000 7f9aafe00640 1 -- 172.21.15.145:0/3919843038 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569151f53e0 msgr2=0x556915222500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:36.641 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.638+0000 7f9aafe00640 1 --2- 172.21.15.145:0/3919843038 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569151f53e0 0x556915222500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:36.641 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.638+0000 7f9aafe00640 1 -- 172.21.15.145:0/3919843038 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569151ded00 con 0x556915200d40 2024-05-09T04:35:36.641 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.638+0000 7f9aafe00640 1 --2- 172.21.15.145:0/3919843038 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556915200d40 0x5569151fcd00 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f9aa805c570 tx=0x7f9aa808f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf31e9f26f285e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:36.641 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.639+0000 7f9a9f400640 1 -- 172.21.15.145:0/3919843038 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa805fda0 con 0x556915200d40 2024-05-09T04:35:36.641 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.639+0000 7f9a9f400640 1 -- 172.21.15.145:0/3919843038 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9aa8094070 con 0x556915200d40 2024-05-09T04:35:36.641 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.639+0000 7f9a9f400640 1 -- 172.21.15.145:0/3919843038 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa8060020 con 0x556915200d40 2024-05-09T04:35:36.641 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.639+0000 7f9ab97b8740 1 -- 172.21.15.145:0/3919843038 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556915200d40 msgr2=0x5569151fcd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:36.641 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.639+0000 7f9ab97b8740 1 --2- 172.21.15.145:0/3919843038 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556915200d40 0x5569151fcd00 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f9aa805c570 tx=0x7f9aa808f000 comp rx=0 tx=0).stop 2024-05-09T04:35:36.641 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.639+0000 7f9ab97b8740 1 -- 172.21.15.145:0/3919843038 shutdown_connections 2024-05-09T04:35:36.641 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.639+0000 7f9ab97b8740 1 --2- 172.21.15.145:0/3919843038 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x556915200d40 0x5569151fcd00 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:36.641 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.639+0000 7f9ab97b8740 1 --2- 172.21.15.145:0/3919843038 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5569151f53e0 0x556915222500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:36.641 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.639+0000 7f9ab97b8740 1 --2- 172.21.15.145:0/3919843038 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5569151e3e60 0x5569151e68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:36.641 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.639+0000 7f9ab97b8740 1 -- 172.21.15.145:0/3919843038 >> 172.21.15.145:0/3919843038 conn(0x5569151e3720 msgr2=0x5569151b4130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:36.642 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.639+0000 7f9ab97b8740 1 -- 172.21.15.145:0/3919843038 shutdown_connections 2024-05-09T04:35:36.642 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.639+0000 7f9ab97b8740 1 -- 172.21.15.145:0/3919843038 wait complete. 2024-05-09T04:35:36.642 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.640+0000 7f9ab97b8740 1 -- start start 2024-05-09T04:35:36.642 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.640+0000 7f9ab97b8740 1 Processor -- start 2024-05-09T04:35:36.644 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.641+0000 7f9ab97b8740 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5569151e3e60 0x55691527bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:36.644 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.641+0000 7f9ab97b8740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5569151f53e0 0x55691527fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:36.644 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.641+0000 7f9ab97b8740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556915200d40 0x5569151b89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:36.645 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.641+0000 7f9ab97b8740 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x5569151f66e0 con 0x5569151e3e60 2024-05-09T04:35:36.645 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.641+0000 7f9ab97b8740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab0201740 con 0x556915200d40 2024-05-09T04:35:36.645 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.641+0000 7f9ab97b8740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x5569151af420 con 0x5569151f53e0 2024-05-09T04:35:36.645 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.641+0000 7f9ab97b8740 10 client.0 _ll_register_callbacks cb 0x7f9aa8079070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:35:36.645 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.642+0000 7f9aaf400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5569151e3e60 0x55691527bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:36.645 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.642+0000 7f9aaf400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5569151e3e60 0x55691527bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:56448/0 (socket says 172.21.15.145:56448) 2024-05-09T04:35:36.645 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.642+0000 7f9aaf400640 1 -- 172.21.15.145:0/795190285 learned_addr learned my addr 172.21.15.145:0/795190285 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:36.645 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.642+0000 7f9aaf400640 1 -- 172.21.15.145:0/795190285 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5569151f53e0 msgr2=0x55691527fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:36.645 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.642+0000 7f9aaf400640 1 --2- 172.21.15.145:0/795190285 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5569151f53e0 0x55691527fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:36.645 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.642+0000 7f9aaf400640 1 -- 172.21.15.145:0/795190285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556915200d40 msgr2=0x5569151b89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:36.645 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.642+0000 7f9aaf400640 1 --2- 172.21.15.145:0/795190285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556915200d40 0x5569151b89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:36.645 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.642+0000 7f9aaf400640 1 -- 172.21.15.145:0/795190285 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569151f3ee0 con 0x5569151e3e60 2024-05-09T04:35:36.645 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.642+0000 7f9aaf400640 1 --2- 172.21.15.145:0/795190285 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5569151e3e60 0x55691527bf10 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0000f80 tx=0x7f9aa0009dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ad4f9537d2792e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:36.645 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.642+0000 7f9a9cc00640 10 client.0 ms_handle_connect on v2:172.21.15.145:3300/0 2024-05-09T04:35:36.646 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.642+0000 7f9a9cc00640 1 -- 172.21.15.145:0/795190285 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa0057060 con 0x5569151e3e60 2024-05-09T04:35:36.646 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.642+0000 7f9a9cc00640 1 -- 172.21.15.145:0/795190285 <== mon.0 v2:172.21.15.145:3300/0 2 ==== osd_map(51..51 src has 1..51) ==== 6172+0+0 (secure 0 0 0) 0x7f9aa002ead0 con 0x5569151e3e60 2024-05-09T04:35:36.646 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.642+0000 7f9ab97b8740 1 -- 172.21.15.145:0/795190285 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9aa8094070 con 0x5569151e3e60 2024-05-09T04:35:36.646 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.643+0000 7f9a9cc00640 1 -- 172.21.15.145:0/795190285 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9aa005d050 con 0x5569151e3e60 2024-05-09T04:35:36.646 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.643+0000 7f9ab97b8740 10 client.5529 fetch_fsmap learned FSMap version 28 2024-05-09T04:35:36.646 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.643+0000 7f9ab97b8740 1 -- 172.21.15.145:0/795190285 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) -- 0x5569151ffd80 con 0x5569151e3e60 2024-05-09T04:35:36.646 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.643+0000 7f9a9cc00640 1 -- 172.21.15.145:0/795190285 <== mon.0 v2:172.21.15.145:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9aa8094070 con 0x5569151e3e60 2024-05-09T04:35:36.646 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.643+0000 7f9a9cc00640 1 -- 172.21.15.145:0/795190285 <== mon.0 v2:172.21.15.145:3300/0 5 ==== fsmap(e 28) ==== 3502+0+0 (secure 0 0 0) 0x7f9aa0060020 con 0x5569151e3e60 2024-05-09T04:35:36.646 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.643+0000 7f9ab97b8740 10 client.5529 fetch_fsmap finished waiting for FSMap version 28 2024-05-09T04:35:36.646 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.643+0000 7f9ab97b8740 10 client.5529 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2024-05-09T04:35:36.646 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.643+0000 7f9ab97b8740 20 client.5529 populate_metadata read hostname 'smithi145' 2024-05-09T04:35:36.646 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.643+0000 7f9ab97b8740 1 --2- 172.21.15.145:0/795190285 >> [v2:172.21.15.145:6835/2593908438,v1:172.21.15.145:6837/2593908438] conn(0x556915200670 0x5569151b8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:36.646 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.643+0000 7f9ab97b8740 4 client.5529 mds_command: new command op to 5316 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T04:35:36.646 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.643+0000 7f9ab97b8740 1 -- 172.21.15.145:0/795190285 --> [v2:172.21.15.145:6835/2593908438,v1:172.21.15.145:6837/2593908438] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55691512b960 con 0x556915200670 2024-05-09T04:35:36.646 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.643+0000 7f9aaea00640 1 --2- 172.21.15.145:0/795190285 >> [v2:172.21.15.145:6835/2593908438,v1:172.21.15.145:6837/2593908438] conn(0x556915200670 0x5569151b8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:36.647 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.644+0000 7f9aaea00640 1 --2- 172.21.15.145:0/795190285 >> [v2:172.21.15.145:6835/2593908438,v1:172.21.15.145:6837/2593908438] conn(0x556915200670 0x5569151b8f10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5e6bbc946cd7e96 server_cookie=877848bfb6015c5b in_seq=0 out_seq=0 2024-05-09T04:35:36.647 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.644+0000 7f9a9cc00640 10 client.5529 ms_handle_connect on v2:172.21.15.145:6835/2593908438 2024-05-09T04:35:36.667 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.666+0000 7f9a9cc00640 1 -- 172.21.15.145:0/795190285 <== mds.0 v2:172.21.15.145:6835/2593908438 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55691512b960 con 0x556915200670 2024-05-09T04:35:36.667 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.666+0000 7f9a9cc00640 10 client.5529 handle_command_reply: tid=0 2024-05-09T04:35:36.667 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.666+0000 7f9ab97b8740 1 client.5529 shutdown 2024-05-09T04:35:36.667 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.667+0000 7f9ab97b8740 1 -- 172.21.15.145:0/795190285 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5569151e3e60 msgr2=0x55691527bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:36.667 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.667+0000 7f9ab97b8740 1 --2- 172.21.15.145:0/795190285 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5569151e3e60 0x55691527bf10 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0000f80 tx=0x7f9aa0009dd0 comp rx=0 tx=0).stop 2024-05-09T04:35:36.667 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.667+0000 7f9ab97b8740 1 -- 172.21.15.145:0/795190285 shutdown_connections 2024-05-09T04:35:36.667 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.667+0000 7f9ab97b8740 1 --2- 172.21.15.145:0/795190285 >> [v2:172.21.15.145:6835/2593908438,v1:172.21.15.145:6837/2593908438] conn(0x556915200670 0x5569151b8f10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:36.667 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.667+0000 7f9ab97b8740 1 --2- 172.21.15.145:0/795190285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x556915200d40 0x5569151b89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:36.667 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.667+0000 7f9ab97b8740 1 --2- 172.21.15.145:0/795190285 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5569151f53e0 0x55691527fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:36.667 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.667+0000 7f9ab97b8740 1 --2- 172.21.15.145:0/795190285 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x5569151e3e60 0x55691527bf10 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:36.667 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.667+0000 7f9a9cc00640 0 client.5529 ms_handle_reset on v2:172.21.15.145:6835/2593908438 2024-05-09T04:35:36.667 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.667+0000 7f9ab97b8740 1 -- 172.21.15.145:0/795190285 >> 172.21.15.145:0/795190285 conn(0x5569151e3720 msgr2=0x5569151b7b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:36.668 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.667+0000 7f9ab97b8740 1 -- 172.21.15.145:0/795190285 shutdown_connections 2024-05-09T04:35:36.668 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.667+0000 7f9ab97b8740 1 -- 172.21.15.145:0/795190285 wait complete. 2024-05-09T04:35:36.668 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.667+0000 7f9ab97b8740 20 client.5529 trim_cache size 0 max 16384 2024-05-09T04:35:36.668 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-05-09T04:35:36.668 INFO:teuthology.orchestra.run.smithi145.stdout: "message": "", 2024-05-09T04:35:36.668 INFO:teuthology.orchestra.run.smithi145.stdout: "return_code": 0 2024-05-09T04:35:36.668 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-05-09T04:35:36.669 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.668+0000 7f9ab6800640 1 -- 172.21.15.145:0/1440418597 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f9a8c0762d0 msgr2=0x7f9a8c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:36.669 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.668+0000 7f9ab6800640 1 --2- 172.21.15.145:0/1440418597 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f9a8c0762d0 0x7f9a8c078790 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0007360 tx=0x7f9aa0038040 comp rx=0 tx=0).stop 2024-05-09T04:35:36.669 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.668+0000 7f9ab6800640 1 -- 172.21.15.145:0/1440418597 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9ab0150430 msgr2=0x7f9ab01508c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:36.669 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.668+0000 7f9ab6800640 1 --2- 172.21.15.145:0/1440418597 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9ab0150430 0x7f9ab01508c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f9ab00b2a30 tx=0x7f9aa800e830 comp rx=0 tx=0).stop 2024-05-09T04:35:36.669 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.668+0000 7f9ab6800640 1 -- 172.21.15.145:0/1440418597 shutdown_connections 2024-05-09T04:35:36.669 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.668+0000 7f9ab6800640 1 --2- 172.21.15.145:0/1440418597 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f9a8c0762d0 0x7f9a8c078790 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:36.669 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.668+0000 7f9ab6800640 1 --2- 172.21.15.145:0/1440418597 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9ab0150430 0x7f9ab01508c0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:36.669 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.668+0000 7f9ab6800640 1 --2- 172.21.15.145:0/1440418597 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9ab014d4c0 0x7f9ab014d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:36.669 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.668+0000 7f9ab6800640 1 --2- 172.21.15.145:0/1440418597 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a94003a00 0x7f9ab01541d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:36.669 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.668+0000 7f9ab6800640 1 -- 172.21.15.145:0/1440418597 >> 172.21.15.145:0/1440418597 conn(0x7f9ab00b4af0 msgr2=0x7f9ab00cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:36.669 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.668+0000 7f9ab6800640 1 -- 172.21.15.145:0/1440418597 shutdown_connections 2024-05-09T04:35:36.670 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:36.668+0000 7f9ab6800640 1 -- 172.21.15.145:0/1440418597 wait complete. 2024-05-09T04:35:36.685 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-05-09T04:35:36.685 DEBUG:teuthology.orchestra.run.smithi145:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-09T04:35:36.775 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-05-09T04:35:36.881 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-05-09T04:35:36.882 INFO:tasks.cephfs.mount:Terminating background process 2024-05-09T04:35:36.882 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-05-09T04:35:36.882 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T04:35:36.936 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-05-09T04:35:36.936 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T04:35:36.987 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T04:35:37.188 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.188+0000 7f0f95e00640 1 Processor -- start 2024-05-09T04:35:37.188 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.188+0000 7f0f95e00640 1 -- start start 2024-05-09T04:35:37.189 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.190+0000 7f0f95e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f0f900ce060 0x7f0f9006e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:37.190 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.190+0000 7f0f95e00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f9006ebd0 con 0x7f0f900ce440 2024-05-09T04:35:37.190 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.190+0000 7f0f95e00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f9006eeb0 con 0x7f0f90145ad0 2024-05-09T04:35:37.190 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.190+0000 7f0f95e00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f0f90073460 con 0x7f0f900ce060 2024-05-09T04:35:37.190 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.190+0000 7f0f8ea00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f0f900ce060 0x7f0f9006e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:37.190 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.190+0000 7f0f87e00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f0f900ce440 0x7f0f901453c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:52748/0 (socket says 172.21.15.145:52748) 2024-05-09T04:35:37.190 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.190+0000 7f0f87e00640 1 -- 172.21.15.145:0/2430316135 learned_addr learned my addr 172.21.15.145:0/2430316135 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:35:37.191 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.191+0000 7f0f8e000640 1 -- 172.21.15.145:0/2430316135 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4156165060 0 0) 0x7f0f9006ebd0 con 0x7f0f900ce440 2024-05-09T04:35:37.191 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.191+0000 7f0f8ea00640 1 -- 172.21.15.145:0/2430316135 >> v1:172.21.15.192:6789/0 conn(0x7f0f90145ad0 legacy=0x7f0f900738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:37.191 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.191+0000 7f0f8ea00640 1 -- 172.21.15.145:0/2430316135 >> v1:172.21.15.145:6789/0 conn(0x7f0f900ce440 legacy=0x7f0f901453c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:37.191 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.191+0000 7f0f8ea00640 1 -- 172.21.15.145:0/2430316135 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f9006f190 con 0x7f0f900ce060 2024-05-09T04:35:37.191 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.192+0000 7f0f8ea00640 1 --2- 172.21.15.145:0/2430316135 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0f900ce060 0x7f0f9006e510 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f0f800097a0 tx=0x7f0f8002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49e198484c242f31 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:37.192 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.192+0000 7f0f8e000640 1 -- 172.21.15.145:0/2430316135 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f80037070 con 0x7f0f900ce060 2024-05-09T04:35:37.192 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.192+0000 7f0f8e000640 1 -- 172.21.15.145:0/2430316135 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f80032440 con 0x7f0f900ce060 2024-05-09T04:35:37.192 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.192+0000 7f0f8e000640 1 -- 172.21.15.145:0/2430316135 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f80032810 con 0x7f0f900ce060 2024-05-09T04:35:37.192 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.192+0000 7f0f95e00640 1 -- 172.21.15.145:0/2430316135 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0f900ce060 msgr2=0x7f0f9006e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:37.193 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.192+0000 7f0f95e00640 1 --2- 172.21.15.145:0/2430316135 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0f900ce060 0x7f0f9006e510 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f0f800097a0 tx=0x7f0f8002a180 comp rx=0 tx=0).stop 2024-05-09T04:35:37.193 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.193+0000 7f0f95e00640 1 -- 172.21.15.145:0/2430316135 shutdown_connections 2024-05-09T04:35:37.193 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.193+0000 7f0f95e00640 1 --2- 172.21.15.145:0/2430316135 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0f900ce060 0x7f0f9006e510 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:37.193 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.193+0000 7f0f95e00640 1 -- 172.21.15.145:0/2430316135 >> 172.21.15.145:0/2430316135 conn(0x7f0f9006c8d0 msgr2=0x7f0f9006cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:37.193 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.193+0000 7f0f95e00640 1 -- 172.21.15.145:0/2430316135 shutdown_connections 2024-05-09T04:35:37.193 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.193+0000 7f0f95e00640 1 -- 172.21.15.145:0/2430316135 wait complete. 2024-05-09T04:35:37.194 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.194+0000 7f0f95e00640 1 Processor -- start 2024-05-09T04:35:37.194 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.195+0000 7f0f95e00640 1 -- start start 2024-05-09T04:35:37.196 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.196+0000 7f0f95e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0f900ce060 0x7f0f9013d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:37.196 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.196+0000 7f0f95e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f900ce440 0x7f0f9015aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:37.196 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.196+0000 7f0f95e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0f90145ad0 0x7f0f900ced50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:37.196 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.196+0000 7f0f95e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9015d8c0 con 0x7f0f900ce060 2024-05-09T04:35:37.196 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.196+0000 7f0f95e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9006f750 con 0x7f0f900ce440 2024-05-09T04:35:37.196 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.196+0000 7f0f95e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0f9015da40 con 0x7f0f90145ad0 2024-05-09T04:35:37.197 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.196+0000 7f0f8ea00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0f900ce060 0x7f0f9013d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:37.197 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.196+0000 7f0f8ea00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0f900ce060 0x7f0f9013d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:56460/0 (socket says 172.21.15.145:56460) 2024-05-09T04:35:37.197 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.196+0000 7f0f8ea00640 1 -- 172.21.15.145:0/1468597058 learned_addr learned my addr 172.21.15.145:0/1468597058 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:37.197 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.196+0000 7f0f8ea00640 1 -- 172.21.15.145:0/1468597058 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0f90145ad0 msgr2=0x7f0f900ced50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:37.197 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.196+0000 7f0f8ea00640 1 --2- 172.21.15.145:0/1468597058 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0f90145ad0 0x7f0f900ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:37.197 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.196+0000 7f0f8ea00640 1 -- 172.21.15.145:0/1468597058 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f900ce440 msgr2=0x7f0f9015aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:37.197 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.196+0000 7f0f8ea00640 1 --2- 172.21.15.145:0/1468597058 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f900ce440 0x7f0f9015aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:37.197 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.196+0000 7f0f8ea00640 1 -- 172.21.15.145:0/1468597058 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f900ae880 con 0x7f0f900ce060 2024-05-09T04:35:37.197 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.197+0000 7f0f8ea00640 1 --2- 172.21.15.145:0/1468597058 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0f900ce060 0x7f0f9013d700 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f0f80035040 tx=0x7f0f800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:37.198 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.197+0000 7f0f87400640 1 -- 172.21.15.145:0/1468597058 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f80037040 con 0x7f0f900ce060 2024-05-09T04:35:37.199 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.197+0000 7f0f87400640 1 -- 172.21.15.145:0/1468597058 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f80004250 con 0x7f0f900ce060 2024-05-09T04:35:37.199 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.197+0000 7f0f95e00640 1 -- 172.21.15.145:0/1468597058 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f900aeb60 con 0x7f0f900ce060 2024-05-09T04:35:37.199 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.197+0000 7f0f87400640 1 -- 172.21.15.145:0/1468597058 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f80032e20 con 0x7f0f900ce060 2024-05-09T04:35:37.199 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.197+0000 7f0f95e00640 1 -- 172.21.15.145:0/1468597058 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f900af120 con 0x7f0f900ce060 2024-05-09T04:35:37.199 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.198+0000 7f0f95e00640 1 -- 172.21.15.145:0/1468597058 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f9006a310 con 0x7f0f900ce060 2024-05-09T04:35:37.199 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.199+0000 7f0f87400640 1 -- 172.21.15.145:0/1468597058 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0f80030770 con 0x7f0f900ce060 2024-05-09T04:35:37.200 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.200+0000 7f0f87400640 1 --2- 172.21.15.145:0/1468597058 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f0f6c0762d0 0x7f0f6c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:37.200 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.200+0000 7f0f87400640 1 -- 172.21.15.145:0/1468597058 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6172+0+0 (secure 0 0 0) 0x7f0f800b8b40 con 0x7f0f900ce060 2024-05-09T04:35:37.200 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.201+0000 7f0f87e00640 1 --2- 172.21.15.145:0/1468597058 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f0f6c0762d0 0x7f0f6c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:37.201 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.201+0000 7f0f87e00640 1 --2- 172.21.15.145:0/1468597058 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f0f6c0762d0 0x7f0f6c078790 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0f90160d70 tx=0x7f0f780078d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:37.205 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.205+0000 7f0f87400640 1 -- 172.21.15.145:0/1468597058 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0f80082230 con 0x7f0f900ce060 2024-05-09T04:35:37.426 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.426+0000 7f0f95e00640 1 -- 172.21.15.145:0/1468597058 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0f900d03f0 con 0x7f0f900ce060 2024-05-09T04:35:37.427 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.427+0000 7f0f87400640 1 -- 172.21.15.145:0/1468597058 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v51) ==== 80+0+0 (secure 0 0 0) 0x7f0f80081b30 con 0x7f0f900ce060 2024-05-09T04:35:37.427 INFO:teuthology.orchestra.run.smithi145.stderr:listed 0 entries 2024-05-09T04:35:37.429 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.429+0000 7f0f95e00640 1 -- 172.21.15.145:0/1468597058 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f0f6c0762d0 msgr2=0x7f0f6c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:37.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.429+0000 7f0f95e00640 1 --2- 172.21.15.145:0/1468597058 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f0f6c0762d0 0x7f0f6c078790 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0f90160d70 tx=0x7f0f780078d0 comp rx=0 tx=0).stop 2024-05-09T04:35:37.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.429+0000 7f0f95e00640 1 -- 172.21.15.145:0/1468597058 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0f900ce060 msgr2=0x7f0f9013d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:37.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.429+0000 7f0f95e00640 1 --2- 172.21.15.145:0/1468597058 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0f900ce060 0x7f0f9013d700 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f0f80035040 tx=0x7f0f800046c0 comp rx=0 tx=0).stop 2024-05-09T04:35:37.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.430+0000 7f0f95e00640 1 -- 172.21.15.145:0/1468597058 shutdown_connections 2024-05-09T04:35:37.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.430+0000 7f0f95e00640 1 --2- 172.21.15.145:0/1468597058 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f0f6c0762d0 0x7f0f6c078790 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:37.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.430+0000 7f0f95e00640 1 --2- 172.21.15.145:0/1468597058 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0f90145ad0 0x7f0f900ced50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:37.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.430+0000 7f0f95e00640 1 --2- 172.21.15.145:0/1468597058 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f900ce440 0x7f0f9015aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:37.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.430+0000 7f0f95e00640 1 --2- 172.21.15.145:0/1468597058 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0f900ce060 0x7f0f9013d700 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:37.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.430+0000 7f0f95e00640 1 -- 172.21.15.145:0/1468597058 >> 172.21.15.145:0/1468597058 conn(0x7f0f9006c8d0 msgr2=0x7f0f900b2380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:37.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.430+0000 7f0f95e00640 1 -- 172.21.15.145:0/1468597058 shutdown_connections 2024-05-09T04:35:37.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.430+0000 7f0f95e00640 1 -- 172.21.15.145:0/1468597058 wait complete. 2024-05-09T04:35:37.448 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi145.front.sepia.ceph.com... 2024-05-09T04:35:37.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:35:37.449 DEBUG:teuthology.orchestra.run.smithi145:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-09T04:35:37.516 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi145.stderr:ceph-fuse[50203]: fuse finished with error 0 and tester_r 0 2024-05-09T04:35:37.522 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi145.front.sepia.ceph.com 2024-05-09T04:35:37.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:35:37.522 DEBUG:teuthology.orchestra.run.smithi145:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-09T04:35:37.620 INFO:tasks.cephfs.mount:Terminating background process 2024-05-09T04:35:37.621 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-09T04:35:37.621 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T04:35:37.663 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-09T04:35:37.663 INFO:tasks.cephfs.mount:Terminating background process 2024-05-09T04:35:37.663 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-05-09T04:35:37.663 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T04:35:37.700 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-09T04:35:37.700 INFO:tasks.cephfs.mount:Terminating background process 2024-05-09T04:35:37.700 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-09T04:35:37.700 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T04:35:37.752 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-09T04:35:37.752 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T04:35:37.938 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.939+0000 7f8dc7200640 1 Processor -- start 2024-05-09T04:35:37.938 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.939+0000 7f8dc7200640 1 -- start start 2024-05-09T04:35:37.940 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.941+0000 7f8dc7200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8dc815bec0 0x7f8dc815f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:37.940 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.941+0000 7f8dc7200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f8dc815fde0 0x7f8dc80cfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:37.940 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.941+0000 7f8dc7200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8dc8160280 con 0x7f8dc815bae0 2024-05-09T04:35:37.940 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.941+0000 7f8dc7200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f8dc8160840 con 0x7f8dc815fde0 2024-05-09T04:35:37.940 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.941+0000 7f8dc7200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8dc80b4190 con 0x7f8dc815bec0 2024-05-09T04:35:37.941 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.941+0000 7f8dbfe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8dc815bec0 0x7f8dc815f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:37.941 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.941+0000 7f8dbfe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8dc815bec0 0x7f8dc815f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.145:57404/0 (socket says 172.21.15.145:57404) 2024-05-09T04:35:37.941 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.941+0000 7f8dbfe00640 1 -- 172.21.15.145:0/2068696665 learned_addr learned my addr 172.21.15.145:0/2068696665 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:37.941 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.941+0000 7f8dc6800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f8dc815fde0 0x7f8dc80cfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:37.941 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.942+0000 7f8dbfe00640 1 -- 172.21.15.145:0/2068696665 >> v2:172.21.15.192:3300/0 conn(0x7f8dc815fde0 msgr2=0x7f8dc80cfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:37.942 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.942+0000 7f8dbfe00640 1 --2- 172.21.15.145:0/2068696665 >> v2:172.21.15.192:3300/0 conn(0x7f8dc815fde0 0x7f8dc80cfe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:37.942 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.942+0000 7f8dbfe00640 1 -- 172.21.15.145:0/2068696665 >> v1:172.21.15.192:6789/0 conn(0x7f8dc815bae0 legacy=0x7f8dc80721b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:37.942 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.942+0000 7f8dbfe00640 1 -- 172.21.15.145:0/2068696665 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dc8160560 con 0x7f8dc815bec0 2024-05-09T04:35:37.942 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.943+0000 7f8dbfe00640 1 --2- 172.21.15.145:0/2068696665 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8dc815bec0 0x7f8dc815f8a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8db000bb40 tx=0x7f8db002c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f7568f18ca64901 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:37.943 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.943+0000 7f8dc5400640 1 -- 172.21.15.145:0/2068696665 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db0032780 con 0x7f8dc815bec0 2024-05-09T04:35:37.943 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.944+0000 7f8dc5400640 1 -- 172.21.15.145:0/2068696665 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8db0038070 con 0x7f8dc815bec0 2024-05-09T04:35:37.943 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.944+0000 7f8dc5400640 1 -- 172.21.15.145:0/2068696665 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db0032b40 con 0x7f8dc815bec0 2024-05-09T04:35:37.944 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.944+0000 7f8dc7200640 1 -- 172.21.15.145:0/2068696665 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8dc815bec0 msgr2=0x7f8dc815f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:37.944 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.944+0000 7f8dc7200640 1 --2- 172.21.15.145:0/2068696665 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8dc815bec0 0x7f8dc815f8a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8db000bb40 tx=0x7f8db002c8c0 comp rx=0 tx=0).stop 2024-05-09T04:35:37.944 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.944+0000 7f8dc7200640 1 -- 172.21.15.145:0/2068696665 shutdown_connections 2024-05-09T04:35:37.944 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.944+0000 7f8dc7200640 1 --2- 172.21.15.145:0/2068696665 >> v2:172.21.15.192:3300/0 conn(0x7f8dc815fde0 0x7f8dc80cfe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:37.944 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.944+0000 7f8dc7200640 1 --2- 172.21.15.145:0/2068696665 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8dc815bec0 0x7f8dc815f8a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:37.944 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.944+0000 7f8dc7200640 1 -- 172.21.15.145:0/2068696665 >> 172.21.15.145:0/2068696665 conn(0x7f8dc806c8a0 msgr2=0x7f8dc806ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:37.944 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.944+0000 7f8dc7200640 1 -- 172.21.15.145:0/2068696665 shutdown_connections 2024-05-09T04:35:37.945 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.944+0000 7f8dc7200640 1 -- 172.21.15.145:0/2068696665 wait complete. 2024-05-09T04:35:37.946 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.946+0000 7f8dc7200640 1 Processor -- start 2024-05-09T04:35:37.946 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.946+0000 7f8dc7200640 1 -- start start 2024-05-09T04:35:37.948 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.947+0000 7f8dc7200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8dc815bae0 0x7f8dc80af7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:37.948 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.947+0000 7f8dc7200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8dc815bec0 0x7f8dc81f54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:37.948 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.947+0000 7f8dc7200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8dc815fde0 0x7f8dc815f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:37.948 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.947+0000 7f8dc7200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc80ae050 con 0x7f8dc815bae0 2024-05-09T04:35:37.948 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.947+0000 7f8dc7200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc80aded0 con 0x7f8dc815fde0 2024-05-09T04:35:37.948 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.947+0000 7f8dc7200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f8dc80d0370 con 0x7f8dc815bec0 2024-05-09T04:35:37.948 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.947+0000 7f8dc5e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8dc815bae0 0x7f8dc80af7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:37.948 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.948+0000 7f8dc5e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8dc815bae0 0x7f8dc80af7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:56462/0 (socket says 172.21.15.145:56462) 2024-05-09T04:35:37.948 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.948+0000 7f8dc5e00640 1 -- 172.21.15.145:0/1700546060 learned_addr learned my addr 172.21.15.145:0/1700546060 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:37.948 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.948+0000 7f8dbfe00640 1 --2- 172.21.15.145:0/1700546060 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8dc815bec0 0x7f8dc81f54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:37.948 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.948+0000 7f8dc6800640 1 --2- 172.21.15.145:0/1700546060 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8dc815fde0 0x7f8dc815f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:37.948 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.948+0000 7f8dc5e00640 1 -- 172.21.15.145:0/1700546060 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8dc815bec0 msgr2=0x7f8dc81f54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:37.949 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.948+0000 7f8dc5e00640 1 --2- 172.21.15.145:0/1700546060 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8dc815bec0 0x7f8dc81f54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:37.949 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.948+0000 7f8dc5e00640 1 -- 172.21.15.145:0/1700546060 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8dc815fde0 msgr2=0x7f8dc815f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:37.949 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.948+0000 7f8dc5e00640 1 --2- 172.21.15.145:0/1700546060 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8dc815fde0 0x7f8dc815f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:37.949 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.948+0000 7f8dc5e00640 1 -- 172.21.15.145:0/1700546060 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dc81f5a30 con 0x7f8dc815bae0 2024-05-09T04:35:37.950 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.948+0000 7f8dc5e00640 1 --2- 172.21.15.145:0/1700546060 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8dc815bae0 0x7f8dc80af7f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f8db800c500 tx=0x7f8db800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:37.950 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.949+0000 7f8dbe000640 1 -- 172.21.15.145:0/1700546060 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db80052d0 con 0x7f8dc815bae0 2024-05-09T04:35:37.950 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.949+0000 7f8dbe000640 1 -- 172.21.15.145:0/1700546060 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8db8005470 con 0x7f8dc815bae0 2024-05-09T04:35:37.950 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.949+0000 7f8dc7200640 1 -- 172.21.15.145:0/1700546060 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dc81f5cd0 con 0x7f8dc815bae0 2024-05-09T04:35:37.950 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.949+0000 7f8dbe000640 1 -- 172.21.15.145:0/1700546060 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db8012870 con 0x7f8dc815bae0 2024-05-09T04:35:37.950 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.949+0000 7f8dc7200640 1 -- 172.21.15.145:0/1700546060 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dc81f6290 con 0x7f8dc815bae0 2024-05-09T04:35:37.950 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.950+0000 7f8dc7200640 1 -- 172.21.15.145:0/1700546060 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dc806a310 con 0x7f8dc815bae0 2024-05-09T04:35:37.951 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.951+0000 7f8dbe000640 1 -- 172.21.15.145:0/1700546060 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f8db8012a10 con 0x7f8dc815bae0 2024-05-09T04:35:37.952 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.953+0000 7f8dbe000640 1 --2- 172.21.15.145:0/1700546060 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f8da40762d0 0x7f8da4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:37.952 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.953+0000 7f8dbe000640 1 -- 172.21.15.145:0/1700546060 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6172+0+0 (secure 0 0 0) 0x7f8db8097480 con 0x7f8dc815bae0 2024-05-09T04:35:37.952 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.953+0000 7f8dbfe00640 1 --2- 172.21.15.145:0/1700546060 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f8da40762d0 0x7f8da4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:37.952 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.953+0000 7f8dbfe00640 1 --2- 172.21.15.145:0/1700546060 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f8da40762d0 0x7f8da4078790 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8db0037000 tx=0x7f8db0032000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:37.958 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:37.958+0000 7f8dbe000640 1 -- 172.21.15.145:0/1700546060 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8db8060b80 con 0x7f8dc815bae0 2024-05-09T04:35:38.203 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.203+0000 7f8dc7200640 1 -- 172.21.15.145:0/1700546060 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8dc815b300 con 0x7f8dc815bae0 2024-05-09T04:35:38.204 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.204+0000 7f8dbe000640 1 -- 172.21.15.145:0/1700546060 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5189 (secure 0 0 0) 0x7f8db8060480 con 0x7f8dc815bae0 2024-05-09T04:35:38.204 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:35:38.204 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4661,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6834/1033874179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":1033874179},{"type":"v1","addr":"172.21.15.192:6834","nonce":1033874179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4664,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1787962921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6833","nonce":1787962921},{"type":"v1","addr":"172.21.15.192:6835","nonce":1787962921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5313,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6836/65063948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":65063948},{"type":"v1","addr":"172.21.15.145:6836","nonce":65063948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T04:35:23.667303+0000","modified":"2024-05-09T04:35:34.804314+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5316},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5316":{"gid":5316,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.145:6837/2593908438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6835","nonce":2593908438},{"type":"v1","addr":"172.21.15.145:6837","nonce":2593908438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8,9],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5316,"qdb_cluster":[5316]},"id":3}]} 2024-05-09T04:35:38.204 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 28 2024-05-09T04:35:38.207 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.207+0000 7f8dc7200640 1 -- 172.21.15.145:0/1700546060 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f8da40762d0 msgr2=0x7f8da4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:38.207 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.207+0000 7f8dc7200640 1 --2- 172.21.15.145:0/1700546060 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f8da40762d0 0x7f8da4078790 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8db0037000 tx=0x7f8db0032000 comp rx=0 tx=0).stop 2024-05-09T04:35:38.207 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.207+0000 7f8dc7200640 1 -- 172.21.15.145:0/1700546060 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8dc815bae0 msgr2=0x7f8dc80af7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:38.208 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.207+0000 7f8dc7200640 1 --2- 172.21.15.145:0/1700546060 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8dc815bae0 0x7f8dc80af7f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f8db800c500 tx=0x7f8db800c9c0 comp rx=0 tx=0).stop 2024-05-09T04:35:38.208 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.207+0000 7f8dc7200640 1 -- 172.21.15.145:0/1700546060 shutdown_connections 2024-05-09T04:35:38.208 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.207+0000 7f8dc7200640 1 --2- 172.21.15.145:0/1700546060 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f8da40762d0 0x7f8da4078790 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:38.208 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.207+0000 7f8dc7200640 1 --2- 172.21.15.145:0/1700546060 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8dc815fde0 0x7f8dc815f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:38.208 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.207+0000 7f8dc7200640 1 --2- 172.21.15.145:0/1700546060 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8dc815bec0 0x7f8dc81f54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:38.208 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.207+0000 7f8dc7200640 1 --2- 172.21.15.145:0/1700546060 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8dc815bae0 0x7f8dc80af7f0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:38.208 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.207+0000 7f8dc7200640 1 -- 172.21.15.145:0/1700546060 >> 172.21.15.145:0/1700546060 conn(0x7f8dc806c8a0 msgr2=0x7f8dc8073670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:38.208 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.207+0000 7f8dc7200640 1 -- 172.21.15.145:0/1700546060 shutdown_connections 2024-05-09T04:35:38.208 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.207+0000 7f8dc7200640 1 -- 172.21.15.145:0/1700546060 wait complete. 2024-05-09T04:35:38.226 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T04:35:38.406 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.407+0000 7f73d2800640 1 Processor -- start 2024-05-09T04:35:38.406 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.407+0000 7f73d2800640 1 -- start start 2024-05-09T04:35:38.407 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.408+0000 7f73d2800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f73cc15fbc0 0x7f73cc142200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:38.407 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.408+0000 7f73d2800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f73cc0726b0 0x7f73cc072b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:38.407 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.408+0000 7f73d2800640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73cc13f340 con 0x7f73cc15ffa0 2024-05-09T04:35:38.407 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.408+0000 7f73d2800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f73cc072220 con 0x7f73cc15fbc0 2024-05-09T04:35:38.408 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.408+0000 7f73d2800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f73cc0cffe0 con 0x7f73cc0726b0 2024-05-09T04:35:38.408 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.408+0000 7f73cbe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f73cc0726b0 0x7f73cc072b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:38.408 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.408+0000 7f73cb400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f73cc15fbc0 0x7f73cc142200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:38.408 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.409+0000 7f73cbe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f73cc0726b0 0x7f73cc072b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.145:57408/0 (socket says 172.21.15.145:57408) 2024-05-09T04:35:38.408 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.409+0000 7f73cbe00640 1 -- 172.21.15.145:0/1656983967 learned_addr learned my addr 172.21.15.145:0/1656983967 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:38.409 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.409+0000 7f73cbe00640 1 -- 172.21.15.145:0/1656983967 >> v2:172.21.15.192:3300/0 conn(0x7f73cc15fbc0 msgr2=0x7f73cc142200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:38.409 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.409+0000 7f73cbe00640 1 --2- 172.21.15.145:0/1656983967 >> v2:172.21.15.192:3300/0 conn(0x7f73cc15fbc0 0x7f73cc142200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:38.409 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.409+0000 7f73cbe00640 1 -- 172.21.15.145:0/1656983967 >> v1:172.21.15.192:6789/0 conn(0x7f73cc15ffa0 legacy=0x7f73cc0cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:38.409 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.409+0000 7f73cbe00640 1 -- 172.21.15.145:0/1656983967 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73cc0d02b0 con 0x7f73cc0726b0 2024-05-09T04:35:38.409 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.409+0000 7f73cb400640 1 --2- 172.21.15.145:0/1656983967 >> v2:172.21.15.192:3300/0 conn(0x7f73cc15fbc0 0x7f73cc142200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:35:38.410 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.411+0000 7f73cbe00640 1 --2- 172.21.15.145:0/1656983967 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f73cc0726b0 0x7f73cc072b40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f73c4009e50 tx=0x7f73c402a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f2ecea477475148 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:38.410 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.411+0000 7f73ca000640 1 -- 172.21.15.145:0/1656983967 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c4036070 con 0x7f73cc0726b0 2024-05-09T04:35:38.411 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.411+0000 7f73ca000640 1 -- 172.21.15.145:0/1656983967 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73c4031600 con 0x7f73cc0726b0 2024-05-09T04:35:38.411 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.411+0000 7f73ca000640 1 -- 172.21.15.145:0/1656983967 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c40319d0 con 0x7f73cc0726b0 2024-05-09T04:35:38.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.412+0000 7f73d2800640 1 -- 172.21.15.145:0/1656983967 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f73cc0726b0 msgr2=0x7f73cc072b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:38.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.412+0000 7f73d2800640 1 --2- 172.21.15.145:0/1656983967 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f73cc0726b0 0x7f73cc072b40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f73c4009e50 tx=0x7f73c402a680 comp rx=0 tx=0).stop 2024-05-09T04:35:38.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.412+0000 7f73d2800640 1 -- 172.21.15.145:0/1656983967 shutdown_connections 2024-05-09T04:35:38.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.412+0000 7f73d2800640 1 --2- 172.21.15.145:0/1656983967 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f73cc0726b0 0x7f73cc072b40 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:38.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.412+0000 7f73d2800640 1 --2- 172.21.15.145:0/1656983967 >> v2:172.21.15.192:3300/0 conn(0x7f73cc15fbc0 0x7f73cc142200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:38.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.412+0000 7f73d2800640 1 -- 172.21.15.145:0/1656983967 >> 172.21.15.145:0/1656983967 conn(0x7f73cc06c870 msgr2=0x7f73cc06cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:38.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.412+0000 7f73d2800640 1 -- 172.21.15.145:0/1656983967 shutdown_connections 2024-05-09T04:35:38.412 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.412+0000 7f73d2800640 1 -- 172.21.15.145:0/1656983967 wait complete. 2024-05-09T04:35:38.413 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.413+0000 7f73d2800640 1 Processor -- start 2024-05-09T04:35:38.413 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.414+0000 7f73d2800640 1 -- start start 2024-05-09T04:35:38.415 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.414+0000 7f73d2800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f73cc0726b0 0x7f73cc15e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:38.415 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.414+0000 7f73d2800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73cc15fbc0 0x7f73cc140f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:38.415 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.415+0000 7f73d2800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73cc15ffa0 0x7f73cc13d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:38.415 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.415+0000 7f73d2800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f73cc0d0b50 con 0x7f73cc15ffa0 2024-05-09T04:35:38.415 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.415+0000 7f73d2800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f73cc0d0870 con 0x7f73cc15fbc0 2024-05-09T04:35:38.415 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.415+0000 7f73d2800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f73cc160430 con 0x7f73cc0726b0 2024-05-09T04:35:38.415 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.415+0000 7f73cbe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73cc15ffa0 0x7f73cc13d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:38.415 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.415+0000 7f73cbe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73cc15ffa0 0x7f73cc13d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:56472/0 (socket says 172.21.15.145:56472) 2024-05-09T04:35:38.415 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.415+0000 7f73cbe00640 1 -- 172.21.15.145:0/4129524031 learned_addr learned my addr 172.21.15.145:0/4129524031 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:38.416 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.415+0000 7f73cbe00640 1 -- 172.21.15.145:0/4129524031 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f73cc0726b0 msgr2=0x7f73cc15e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:38.416 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.415+0000 7f73cbe00640 1 --2- 172.21.15.145:0/4129524031 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f73cc0726b0 0x7f73cc15e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:38.416 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.415+0000 7f73cbe00640 1 -- 172.21.15.145:0/4129524031 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73cc15fbc0 msgr2=0x7f73cc140f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:38.416 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.415+0000 7f73cbe00640 1 --2- 172.21.15.145:0/4129524031 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73cc15fbc0 0x7f73cc140f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:38.416 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.415+0000 7f73cbe00640 1 -- 172.21.15.145:0/4129524031 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73cc152820 con 0x7f73cc15ffa0 2024-05-09T04:35:38.416 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.416+0000 7f73cbe00640 1 --2- 172.21.15.145:0/4129524031 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73cc15ffa0 0x7f73cc13d640 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f73c4009e50 tx=0x7f73c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:38.417 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.416+0000 7f73c3400640 1 -- 172.21.15.145:0/4129524031 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c4036050 con 0x7f73cc15ffa0 2024-05-09T04:35:38.417 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.416+0000 7f73c3400640 1 -- 172.21.15.145:0/4129524031 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73c4042070 con 0x7f73cc15ffa0 2024-05-09T04:35:38.417 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.416+0000 7f73c3400640 1 -- 172.21.15.145:0/4129524031 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c4009510 con 0x7f73cc15ffa0 2024-05-09T04:35:38.417 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.416+0000 7f73d2800640 1 -- 172.21.15.145:0/4129524031 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73cc15e7a0 con 0x7f73cc15ffa0 2024-05-09T04:35:38.417 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.416+0000 7f73d2800640 1 -- 172.21.15.145:0/4129524031 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73cc15ed60 con 0x7f73cc15ffa0 2024-05-09T04:35:38.417 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.417+0000 7f73d2800640 1 -- 172.21.15.145:0/4129524031 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73cc06a310 con 0x7f73cc15ffa0 2024-05-09T04:35:38.417 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.418+0000 7f73c3400640 1 -- 172.21.15.145:0/4129524031 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f73c4007bf0 con 0x7f73cc15ffa0 2024-05-09T04:35:38.419 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.419+0000 7f73c3400640 1 --2- 172.21.15.145:0/4129524031 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f73ac0762d0 0x7f73ac078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:38.419 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.419+0000 7f73c3400640 1 -- 172.21.15.145:0/4129524031 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6172+0+0 (secure 0 0 0) 0x7f73c40b8710 con 0x7f73cc15ffa0 2024-05-09T04:35:38.419 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.419+0000 7f73cb400640 1 --2- 172.21.15.145:0/4129524031 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f73ac0762d0 0x7f73ac078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:38.419 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.420+0000 7f73cb400640 1 --2- 172.21.15.145:0/4129524031 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f73ac0762d0 0x7f73ac078790 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f73bc006e80 tx=0x7f73bc006dd0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:38.423 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.425+0000 7f73c3400640 1 -- 172.21.15.145:0/4129524031 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f73c4081d70 con 0x7f73cc15ffa0 2024-05-09T04:35:38.672 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.672+0000 7f73d2800640 1 -- 172.21.15.145:0/4129524031 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f73cc0b4070 con 0x7f73cc15ffa0 2024-05-09T04:35:38.673 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.675+0000 7f73c3400640 1 -- 172.21.15.145:0/4129524031 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5189 (secure 0 0 0) 0x7f73c4081670 con 0x7f73cc15ffa0 2024-05-09T04:35:38.674 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:35:38.674 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4661,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6834/1033874179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":1033874179},{"type":"v1","addr":"172.21.15.192:6834","nonce":1033874179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4664,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1787962921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6833","nonce":1787962921},{"type":"v1","addr":"172.21.15.192:6835","nonce":1787962921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5313,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6836/65063948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":65063948},{"type":"v1","addr":"172.21.15.145:6836","nonce":65063948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T04:35:23.667303+0000","modified":"2024-05-09T04:35:34.804314+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5316},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5316":{"gid":5316,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.145:6837/2593908438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6835","nonce":2593908438},{"type":"v1","addr":"172.21.15.145:6837","nonce":2593908438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8,9],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5316,"qdb_cluster":[5316]},"id":3}]} 2024-05-09T04:35:38.674 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 28 2024-05-09T04:35:38.677 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.677+0000 7f73d2800640 1 -- 172.21.15.145:0/4129524031 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f73ac0762d0 msgr2=0x7f73ac078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:38.677 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.677+0000 7f73d2800640 1 --2- 172.21.15.145:0/4129524031 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f73ac0762d0 0x7f73ac078790 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f73bc006e80 tx=0x7f73bc006dd0 comp rx=0 tx=0).stop 2024-05-09T04:35:38.677 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.677+0000 7f73d2800640 1 -- 172.21.15.145:0/4129524031 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73cc15ffa0 msgr2=0x7f73cc13d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:38.677 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.677+0000 7f73d2800640 1 --2- 172.21.15.145:0/4129524031 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73cc15ffa0 0x7f73cc13d640 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f73c4009e50 tx=0x7f73c4004a40 comp rx=0 tx=0).stop 2024-05-09T04:35:38.677 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.677+0000 7f73d2800640 1 -- 172.21.15.145:0/4129524031 shutdown_connections 2024-05-09T04:35:38.677 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.677+0000 7f73d2800640 1 --2- 172.21.15.145:0/4129524031 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f73ac0762d0 0x7f73ac078790 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:38.677 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.677+0000 7f73d2800640 1 --2- 172.21.15.145:0/4129524031 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f73cc15ffa0 0x7f73cc13d640 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:38.677 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.677+0000 7f73d2800640 1 --2- 172.21.15.145:0/4129524031 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73cc15fbc0 0x7f73cc140f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:38.677 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.677+0000 7f73d2800640 1 --2- 172.21.15.145:0/4129524031 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f73cc0726b0 0x7f73cc15e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:38.677 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.677+0000 7f73d2800640 1 -- 172.21.15.145:0/4129524031 >> 172.21.15.145:0/4129524031 conn(0x7f73cc06c870 msgr2=0x7f73cc06cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:38.677 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.677+0000 7f73d2800640 1 -- 172.21.15.145:0/4129524031 shutdown_connections 2024-05-09T04:35:38.677 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.677+0000 7f73d2800640 1 -- 172.21.15.145:0/4129524031 wait complete. 2024-05-09T04:35:38.694 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T04:35:38.878 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.879+0000 7f856be00640 1 Processor -- start 2024-05-09T04:35:38.878 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.879+0000 7f856be00640 1 -- start start 2024-05-09T04:35:38.880 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.881+0000 7f856be00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f856c0af610 0x7f856c0afa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:38.880 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.881+0000 7f856be00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f856c0b4ae0 0x7f856c0b4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:38.880 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.881+0000 7f856be00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f856c072c00 con 0x7f856c0b4700 2024-05-09T04:35:38.880 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.881+0000 7f856be00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f856c0aea70 con 0x7f856c0b4ae0 2024-05-09T04:35:38.880 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.881+0000 7f856be00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f856c0737f0 con 0x7f856c0af610 2024-05-09T04:35:38.880 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.881+0000 7f856b400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f856c0b4ae0 0x7f856c0b4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:38.881 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.881+0000 7f856b400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f856c0b4ae0 0x7f856c0b4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:56476/0 (socket says 172.21.15.145:56476) 2024-05-09T04:35:38.881 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.881+0000 7f856b400640 1 -- 172.21.15.145:0/1793027876 learned_addr learned my addr 172.21.15.145:0/1793027876 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:38.881 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.881+0000 7f856a000640 1 --2- 172.21.15.145:0/1793027876 >> v2:172.21.15.192:3300/0 conn(0x7f856c0af610 0x7f856c0afa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:38.881 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.881+0000 7f856b400640 1 -- 172.21.15.145:0/1793027876 >> v2:172.21.15.192:3300/0 conn(0x7f856c0af610 msgr2=0x7f856c0afa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:38.881 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.881+0000 7f856b400640 1 --2- 172.21.15.145:0/1793027876 >> v2:172.21.15.192:3300/0 conn(0x7f856c0af610 0x7f856c0afa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:38.881 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.881+0000 7f856b400640 1 -- 172.21.15.145:0/1793027876 >> v1:172.21.15.192:6789/0 conn(0x7f856c0b4700 legacy=0x7f856c073c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:38.881 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.882+0000 7f856b400640 1 -- 172.21.15.145:0/1793027876 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f856c072ec0 con 0x7f856c0b4ae0 2024-05-09T04:35:38.882 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.883+0000 7f856b400640 1 --2- 172.21.15.145:0/1793027876 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f856c0b4ae0 0x7f856c0b4f10 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f856c06fe70 tx=0x7f856402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34363926d6ec333 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:38.882 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.883+0000 7f8569600640 1 -- 172.21.15.145:0/1793027876 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8564036070 con 0x7f856c0b4ae0 2024-05-09T04:35:38.882 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.883+0000 7f8569600640 1 -- 172.21.15.145:0/1793027876 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8564031600 con 0x7f856c0b4ae0 2024-05-09T04:35:38.882 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.883+0000 7f8569600640 1 -- 172.21.15.145:0/1793027876 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85640319d0 con 0x7f856c0b4ae0 2024-05-09T04:35:38.883 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.883+0000 7f856be00640 1 -- 172.21.15.145:0/1793027876 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f856c0b4ae0 msgr2=0x7f856c0b4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:38.883 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.883+0000 7f856be00640 1 --2- 172.21.15.145:0/1793027876 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f856c0b4ae0 0x7f856c0b4f10 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f856c06fe70 tx=0x7f856402a6a0 comp rx=0 tx=0).stop 2024-05-09T04:35:38.883 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.884+0000 7f856be00640 1 -- 172.21.15.145:0/1793027876 shutdown_connections 2024-05-09T04:35:38.883 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.884+0000 7f856be00640 1 --2- 172.21.15.145:0/1793027876 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f856c0b4ae0 0x7f856c0b4f10 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:38.883 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.884+0000 7f856be00640 1 --2- 172.21.15.145:0/1793027876 >> v2:172.21.15.192:3300/0 conn(0x7f856c0af610 0x7f856c0afa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:38.883 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.884+0000 7f856be00640 1 -- 172.21.15.145:0/1793027876 >> 172.21.15.145:0/1793027876 conn(0x7f856c06c8a0 msgr2=0x7f856c06ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:38.883 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.884+0000 7f856be00640 1 -- 172.21.15.145:0/1793027876 shutdown_connections 2024-05-09T04:35:38.883 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.884+0000 7f856be00640 1 -- 172.21.15.145:0/1793027876 wait complete. 2024-05-09T04:35:38.884 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.885+0000 7f856be00640 1 Processor -- start 2024-05-09T04:35:38.884 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.885+0000 7f856be00640 1 -- start start 2024-05-09T04:35:38.885 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.886+0000 7f856be00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f856c0af610 0x7f856c15ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:38.885 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.886+0000 7f856be00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f856c0b4700 0x7f856c0b1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:38.885 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.886+0000 7f856be00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f856c0b4ae0 0x7f856c15b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:38.885 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.886+0000 7f856be00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f856c0b5450 con 0x7f856c0b4700 2024-05-09T04:35:38.885 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.886+0000 7f856be00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f856c073480 con 0x7f856c0af610 2024-05-09T04:35:38.885 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.886+0000 7f856be00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f856c06e320 con 0x7f856c0b4ae0 2024-05-09T04:35:38.886 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.887+0000 7f856a000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f856c0b4700 0x7f856c0b1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:38.886 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.887+0000 7f856a000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f856c0b4700 0x7f856c0b1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:56482/0 (socket says 172.21.15.145:56482) 2024-05-09T04:35:38.886 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.887+0000 7f856a000640 1 -- 172.21.15.145:0/3613871638 learned_addr learned my addr 172.21.15.145:0/3613871638 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:38.887 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.887+0000 7f856a000640 1 -- 172.21.15.145:0/3613871638 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f856c0b4ae0 msgr2=0x7f856c15b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:38.887 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.887+0000 7f856a000640 1 --2- 172.21.15.145:0/3613871638 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f856c0b4ae0 0x7f856c15b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:38.887 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.887+0000 7f856a000640 1 -- 172.21.15.145:0/3613871638 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f856c0af610 msgr2=0x7f856c15ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:38.887 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.887+0000 7f856a000640 1 --2- 172.21.15.145:0/3613871638 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f856c0af610 0x7f856c15ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:38.887 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.887+0000 7f856a000640 1 -- 172.21.15.145:0/3613871638 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f856c15d3d0 con 0x7f856c0b4700 2024-05-09T04:35:38.887 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.887+0000 7f856a000640 1 --2- 172.21.15.145:0/3613871638 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f856c0b4700 0x7f856c0b1970 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f855800d830 tx=0x7f855800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:38.887 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.887+0000 7f8562a00640 1 -- 172.21.15.145:0/3613871638 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855800b6c0 con 0x7f856c0b4700 2024-05-09T04:35:38.887 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.888+0000 7f8562a00640 1 -- 172.21.15.145:0/3613871638 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8558019070 con 0x7f856c0b4700 2024-05-09T04:35:38.887 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.888+0000 7f8562a00640 1 -- 172.21.15.145:0/3613871638 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855800ba80 con 0x7f856c0b4700 2024-05-09T04:35:38.887 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.888+0000 7f856be00640 1 -- 172.21.15.145:0/3613871638 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f856c1ebb00 con 0x7f856c0b4700 2024-05-09T04:35:38.887 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.888+0000 7f856be00640 1 -- 172.21.15.145:0/3613871638 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f856c1ec0c0 con 0x7f856c0b4700 2024-05-09T04:35:38.890 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.889+0000 7f856be00640 1 -- 172.21.15.145:0/3613871638 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f856c06a310 con 0x7f856c0b4700 2024-05-09T04:35:38.890 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.890+0000 7f8562a00640 1 -- 172.21.15.145:0/3613871638 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f855801e460 con 0x7f856c0b4700 2024-05-09T04:35:38.890 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.891+0000 7f8562a00640 1 --2- 172.21.15.145:0/3613871638 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f854c0762d0 0x7f854c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:38.890 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.891+0000 7f8562a00640 1 -- 172.21.15.145:0/3613871638 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6172+0+0 (secure 0 0 0) 0x7f8558097ec0 con 0x7f856c0b4700 2024-05-09T04:35:38.890 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.891+0000 7f856aa00640 1 --2- 172.21.15.145:0/3613871638 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f854c0762d0 0x7f854c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:38.890 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.892+0000 7f856aa00640 1 --2- 172.21.15.145:0/3613871638 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f854c0762d0 0x7f854c078790 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f855c001560 tx=0x7f855c007990 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:38.893 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:38.895+0000 7f8562a00640 1 -- 172.21.15.145:0/3613871638 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8558061640 con 0x7f856c0b4700 2024-05-09T04:35:39.100 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.102+0000 7f856be00640 1 -- 172.21.15.145:0/3613871638 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f856c0af250 con 0x7f856c0b4700 2024-05-09T04:35:39.102 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.104+0000 7f8562a00640 1 -- 172.21.15.145:0/3613871638 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) ==== 74+0+18633 (secure 0 0 0) 0x7f8558060f40 con 0x7f856c0b4700 2024-05-09T04:35:39.103 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:35:39.104 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":51,"fsid":"957ee6c1-073b-44cf-9b93-9491b3fdfc1d","created":"2024-05-09T04:33:26.544367+0000","modified":"2024-05-09T04:35:36.336628+0000","last_up_change":"2024-05-09T04:33:39.039318+0000","last_in_change":"2024-05-09T04:33:29.902361+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T04:33:39.795344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-09T04:35:21.570048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-09T04:35:22.639737+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data_extra1","create_time":"2024-05-09T04:35:33.506234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22e550ad-847b-4952-80e6-63d859ed1a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6801","nonce":3182572912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6803","nonce":3182572912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6807","nonce":3182572912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6805","nonce":3182572912}]},"public_addr":"172.21.15.145:6801/3182572912","cluster_addr":"172.21.15.145:6803/3182572912","heartbeat_back_addr":"172.21.15.145:6807/3182572912","heartbeat_front_addr":"172.21.15.145:6805/3182572912","state":["exists","up"]},{"osd":1,"uuid":"0f6b9fca-0c41-4145-8d74-598ddb42f2a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6825","nonce":1845225555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6827","nonce":1845225555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6831","nonce":1845225555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6829","nonce":1845225555}]},"public_addr":"172.21.15.145:6825/1845225555","cluster_addr":"172.21.15.145:6827/1845225555","heartbeat_back_addr":"172.21.15.145:6831/1845225555","heartbeat_front_addr":"172.21.15.145:6829/1845225555","state":["exists","up"]},{"osd":2,"uuid":"b3b1f67e-5790-483f-8610-e05a5bce4c30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6817","nonce":2168958993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6819","nonce":2168958993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6823","nonce":2168958993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6821","nonce":2168958993}]},"public_addr":"172.21.15.145:6817/2168958993","cluster_addr":"172.21.15.145:6819/2168958993","heartbeat_back_addr":"172.21.15.145:6823/2168958993","heartbeat_front_addr":"172.21.15.145:6821/2168958993","state":["exists","up"]},{"osd":3,"uuid":"884787a5-e037-4d31-9aae-743030eba4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6809","nonce":337009826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6811","nonce":337009826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6815","nonce":337009826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6813","nonce":337009826}]},"public_addr":"172.21.15.145:6809/337009826","cluster_addr":"172.21.15.145:6811/337009826","heartbeat_back_addr":"172.21.15.145:6815/337009826","heartbeat_front_addr":"172.21.15.145:6813/337009826","state":["exists","up"]},{"osd":4,"uuid":"0759fc1e-c5fc-4a2f-86bb-e8eccd83f833","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6818","nonce":3117700995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6821","nonce":3117700995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6827","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6829","nonce":3117700995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6823","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6825","nonce":3117700995}]},"public_addr":"172.21.15.192:6818/3117700995","cluster_addr":"172.21.15.192:6821/3117700995","heartbeat_back_addr":"172.21.15.192:6829/3117700995","heartbeat_front_addr":"172.21.15.192:6825/3117700995","state":["exists","up"]},{"osd":5,"uuid":"9443cc22-9379-46e3-9173-98b592bf2c16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6801","nonce":1328373268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6803","nonce":1328373268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6807","nonce":1328373268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6805","nonce":1328373268}]},"public_addr":"172.21.15.192:6801/1328373268","cluster_addr":"172.21.15.192:6803/1328373268","heartbeat_back_addr":"172.21.15.192:6807/1328373268","heartbeat_front_addr":"172.21.15.192:6805/1328373268","state":["exists","up"]},{"osd":6,"uuid":"d05a3311-505a-45b5-85b9-aa3113e96987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6817","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6820","nonce":1868011057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6824","nonce":1868011057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6831","nonce":1868011057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6828","nonce":1868011057}]},"public_addr":"172.21.15.192:6820/1868011057","cluster_addr":"172.21.15.192:6824/1868011057","heartbeat_back_addr":"172.21.15.192:6831/1868011057","heartbeat_front_addr":"172.21.15.192:6828/1868011057","state":["exists","up"]},{"osd":7,"uuid":"c7c23dbc-2029-43ec-9f53-25f3d58019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6809","nonce":3368512157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6811","nonce":3368512157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6815","nonce":3368512157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6813","nonce":3368512157}]},"public_addr":"172.21.15.192:6809/3368512157","cluster_addr":"172.21.15.192:6811/3368512157","heartbeat_back_addr":"172.21.15.192:6815/3368512157","heartbeat_front_addr":"172.21.15.192:6813/3368512157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.091188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.032567+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:34.797468+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.228839+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.622400+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.473103+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.566092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.349395+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.c","mappings":[{"from":0,"to":6}]},{"pgid":"8.e","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T04:35:39.106 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.106+0000 7f856be00640 1 -- 172.21.15.145:0/3613871638 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f854c0762d0 msgr2=0x7f854c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:39.106 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.106+0000 7f856be00640 1 --2- 172.21.15.145:0/3613871638 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f854c0762d0 0x7f854c078790 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f855c001560 tx=0x7f855c007990 comp rx=0 tx=0).stop 2024-05-09T04:35:39.106 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.107+0000 7f856be00640 1 -- 172.21.15.145:0/3613871638 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f856c0b4700 msgr2=0x7f856c0b1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:39.106 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.107+0000 7f856be00640 1 --2- 172.21.15.145:0/3613871638 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f856c0b4700 0x7f856c0b1970 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f855800d830 tx=0x7f855800dcf0 comp rx=0 tx=0).stop 2024-05-09T04:35:39.106 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.107+0000 7f856be00640 1 -- 172.21.15.145:0/3613871638 shutdown_connections 2024-05-09T04:35:39.106 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.107+0000 7f856be00640 1 --2- 172.21.15.145:0/3613871638 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f854c0762d0 0x7f854c078790 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:39.106 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.107+0000 7f856be00640 1 --2- 172.21.15.145:0/3613871638 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f856c0b4ae0 0x7f856c15b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:39.106 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.107+0000 7f856be00640 1 --2- 172.21.15.145:0/3613871638 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f856c0b4700 0x7f856c0b1970 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:39.106 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.107+0000 7f856be00640 1 --2- 172.21.15.145:0/3613871638 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f856c0af610 0x7f856c15ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:39.106 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.107+0000 7f856be00640 1 -- 172.21.15.145:0/3613871638 >> 172.21.15.145:0/3613871638 conn(0x7f856c06c8a0 msgr2=0x7f856c073c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:39.106 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.107+0000 7f856be00640 1 -- 172.21.15.145:0/3613871638 shutdown_connections 2024-05-09T04:35:39.107 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.107+0000 7f856be00640 1 -- 172.21.15.145:0/3613871638 wait complete. 2024-05-09T04:35:39.125 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-09T04:35:39.125 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T04:35:39.327 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.328+0000 7f5525c00640 1 Processor -- start 2024-05-09T04:35:39.327 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.328+0000 7f5525c00640 1 -- start start 2024-05-09T04:35:39.328 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.330+0000 7f5525c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f55201572c0 0x7f552015deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:39.328 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.330+0000 7f5525c00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f55200ae710 0x7f5520161220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:39.329 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.330+0000 7f5525c00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55200aecd0 con 0x7f55200cff30 2024-05-09T04:35:39.329 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.330+0000 7f5525c00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f55200b6290 con 0x7f55200ae710 2024-05-09T04:35:39.329 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.330+0000 7f5525c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f55200aefb0 con 0x7f55201572c0 2024-05-09T04:35:39.329 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.330+0000 7f5525200640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f55200cff30 0x7f55200b4d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:52762/0 (socket says 172.21.15.145:52762) 2024-05-09T04:35:39.329 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.330+0000 7f5525200640 1 -- 172.21.15.145:0/1479028836 learned_addr learned my addr 172.21.15.145:0/1479028836 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:35:39.331 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.330+0000 7f551fe00640 1 --2- 172.21.15.145:0/1479028836 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f55201572c0 0x7f552015deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:39.332 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.330+0000 7f551f400640 1 --2- 172.21.15.145:0/1479028836 >> v2:172.21.15.192:3300/0 conn(0x7f55200ae710 0x7f5520161220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:39.332 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.330+0000 7f551ea00640 1 -- 172.21.15.145:0/1479028836 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2123979005 0 0) 0x7f55200aecd0 con 0x7f55200cff30 2024-05-09T04:35:39.332 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.330+0000 7f551ea00640 1 -- 172.21.15.145:0/1479028836 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5504003520 con 0x7f55200cff30 2024-05-09T04:35:39.332 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.331+0000 7f551ea00640 1 -- 172.21.15.145:0/1479028836 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1163001496 0 0) 0x7f5504003520 con 0x7f55200cff30 2024-05-09T04:35:39.332 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.331+0000 7f551ea00640 1 -- 172.21.15.145:0/1479028836 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f55201572c0 msgr2=0x7f552015deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:39.332 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.331+0000 7f551ea00640 1 --2- 172.21.15.145:0/1479028836 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f55201572c0 0x7f552015deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:39.332 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.331+0000 7f551ea00640 1 -- 172.21.15.145:0/1479028836 >> v2:172.21.15.192:3300/0 conn(0x7f55200ae710 msgr2=0x7f5520161220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:39.332 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.331+0000 7f551ea00640 1 --2- 172.21.15.145:0/1479028836 >> v2:172.21.15.192:3300/0 conn(0x7f55200ae710 0x7f5520161220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:39.332 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.331+0000 7f551ea00640 1 -- 172.21.15.145:0/1479028836 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55200af290 con 0x7f55200cff30 2024-05-09T04:35:39.332 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.331+0000 7f551ea00640 1 -- 172.21.15.145:0/1479028836 <== mon.0 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3474413261 0 0) 0x7f5518001a70 con 0x7f55200cff30 2024-05-09T04:35:39.332 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.331+0000 7f551f400640 1 --2- 172.21.15.145:0/1479028836 >> v2:172.21.15.192:3300/0 conn(0x7f55200ae710 0x7f5520161220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:35:39.332 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.331+0000 7f551ea00640 1 -- 172.21.15.145:0/1479028836 >> v1:172.21.15.145:6789/0 conn(0x7f55200cff30 legacy=0x7f55200b4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:39.332 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.331+0000 7f551ea00640 1 --2- 172.21.15.145:0/1479028836 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f55201572c0 0x7f5504003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:39.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.331+0000 7f551ea00640 1 --2- 172.21.15.145:0/1479028836 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5504004080 0x7f5504004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:39.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.331+0000 7f551ea00640 1 --2- 172.21.15.145:0/1479028836 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5504004d40 0x7f55040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:39.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.331+0000 7f551ea00640 1 -- 172.21.15.145:0/1479028836 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f55200b6290 con 0x7f5504004d40 2024-05-09T04:35:39.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.331+0000 7f551ea00640 1 -- 172.21.15.145:0/1479028836 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f55200aefb0 con 0x7f5504004080 2024-05-09T04:35:39.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.331+0000 7f551ea00640 1 -- 172.21.15.145:0/1479028836 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f550400bb80 con 0x7f55201572c0 2024-05-09T04:35:39.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.331+0000 7f5525200640 1 --2- 172.21.15.145:0/1479028836 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5504004d40 0x7f55040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:39.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.331+0000 7f551fe00640 1 --2- 172.21.15.145:0/1479028836 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f55201572c0 0x7f5504003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:39.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.332+0000 7f551f400640 1 --2- 172.21.15.145:0/1479028836 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5504004080 0x7f5504004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:39.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.332+0000 7f5525200640 1 -- 172.21.15.145:0/1479028836 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f55201572c0 msgr2=0x7f5504003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:39.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.332+0000 7f5525200640 1 --2- 172.21.15.145:0/1479028836 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f55201572c0 0x7f5504003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:39.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.332+0000 7f5525200640 1 -- 172.21.15.145:0/1479028836 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5504004080 msgr2=0x7f5504004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:39.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.332+0000 7f5525200640 1 --2- 172.21.15.145:0/1479028836 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5504004080 0x7f5504004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:39.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.332+0000 7f5525200640 1 -- 172.21.15.145:0/1479028836 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5504003520 con 0x7f5504004d40 2024-05-09T04:35:39.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.332+0000 7f551f400640 1 --2- 172.21.15.145:0/1479028836 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5504004080 0x7f5504004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:35:39.334 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.333+0000 7f5525200640 1 --2- 172.21.15.145:0/1479028836 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5504004d40 0x7f55040051c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f5518002230 tx=0x7f5504005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eec7cb71c2dfaa5f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:39.334 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.333+0000 7f551ea00640 1 -- 172.21.15.145:0/1479028836 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f551800a4a0 con 0x7f5504004d40 2024-05-09T04:35:39.334 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.333+0000 7f551ea00640 1 -- 172.21.15.145:0/1479028836 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f551802f070 con 0x7f5504004d40 2024-05-09T04:35:39.334 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.333+0000 7f551ea00640 1 -- 172.21.15.145:0/1479028836 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f551802bc00 con 0x7f5504004d40 2024-05-09T04:35:39.336 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.334+0000 7f5525c00640 1 -- 172.21.15.145:0/1479028836 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5504004d40 msgr2=0x7f55040051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:39.336 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.334+0000 7f5525c00640 1 --2- 172.21.15.145:0/1479028836 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5504004d40 0x7f55040051c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f5518002230 tx=0x7f5504005830 comp rx=0 tx=0).stop 2024-05-09T04:35:39.336 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.334+0000 7f5525200640 1 -- 172.21.15.145:0/1479028836 reap_dead start 2024-05-09T04:35:39.336 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.334+0000 7f5525c00640 1 -- 172.21.15.145:0/1479028836 shutdown_connections 2024-05-09T04:35:39.336 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.334+0000 7f5525c00640 1 -- 172.21.15.145:0/1479028836 >> 172.21.15.145:0/1479028836 conn(0x7f552006c8a0 msgr2=0x7f552006ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:39.336 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.334+0000 7f5525c00640 1 -- 172.21.15.145:0/1479028836 shutdown_connections 2024-05-09T04:35:39.336 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.334+0000 7f5525c00640 1 -- 172.21.15.145:0/1479028836 wait complete. 2024-05-09T04:35:39.336 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.335+0000 7f5525c00640 1 Processor -- start 2024-05-09T04:35:39.336 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.336+0000 7f5525c00640 1 -- start start 2024-05-09T04:35:39.337 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.336+0000 7f5525c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5504004d40 0x7f55200b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:39.337 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.336+0000 7f5525c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5520073d20 0x7f55200724a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:39.337 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.336+0000 7f5525c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5520150580 0x7f552015bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:39.337 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.336+0000 7f5525c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5520161760 con 0x7f5504004d40 2024-05-09T04:35:39.337 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.336+0000 7f5525c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5520160d00 con 0x7f5520073d20 2024-05-09T04:35:39.337 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.336+0000 7f5525c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f55201618e0 con 0x7f5520150580 2024-05-09T04:35:39.337 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.337+0000 7f551fe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5504004d40 0x7f55200b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:39.337 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.337+0000 7f551fe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5504004d40 0x7f55200b32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:56506/0 (socket says 172.21.15.145:56506) 2024-05-09T04:35:39.337 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.337+0000 7f551fe00640 1 -- 172.21.15.145:0/1300240656 learned_addr learned my addr 172.21.15.145:0/1300240656 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:39.337 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.337+0000 7f551f400640 1 --2- 172.21.15.145:0/1300240656 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5520073d20 0x7f55200724a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:39.337 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.337+0000 7f5525200640 1 --2- 172.21.15.145:0/1300240656 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5520150580 0x7f552015bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:39.337 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.337+0000 7f551fe00640 1 -- 172.21.15.145:0/1300240656 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5520150580 msgr2=0x7f552015bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:39.337 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.337+0000 7f551fe00640 1 --2- 172.21.15.145:0/1300240656 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5520150580 0x7f552015bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:39.337 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.337+0000 7f551fe00640 1 -- 172.21.15.145:0/1300240656 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5520073d20 msgr2=0x7f55200724a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:39.337 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.337+0000 7f551fe00640 1 --2- 172.21.15.145:0/1300240656 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5520073d20 0x7f55200724a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:39.338 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.337+0000 7f551fe00640 1 -- 172.21.15.145:0/1300240656 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55200729e0 con 0x7f5504004d40 2024-05-09T04:35:39.338 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.337+0000 7f551fe00640 1 --2- 172.21.15.145:0/1300240656 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5504004d40 0x7f55200b32c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f551000f580 tx=0x7f551000fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:39.339 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.338+0000 7f5517e00640 1 -- 172.21.15.145:0/1300240656 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5510019020 con 0x7f5504004d40 2024-05-09T04:35:39.339 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.338+0000 7f5517e00640 1 -- 172.21.15.145:0/1300240656 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5510014870 con 0x7f5504004d40 2024-05-09T04:35:39.339 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.338+0000 7f5517e00640 1 -- 172.21.15.145:0/1300240656 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5510014c60 con 0x7f5504004d40 2024-05-09T04:35:39.339 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.338+0000 7f5525c00640 1 -- 172.21.15.145:0/1300240656 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5520072c80 con 0x7f5504004d40 2024-05-09T04:35:39.339 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.338+0000 7f5525c00640 1 -- 172.21.15.145:0/1300240656 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5520073240 con 0x7f5504004d40 2024-05-09T04:35:39.339 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.339+0000 7f5525c00640 1 -- 172.21.15.145:0/1300240656 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55200768c0 con 0x7f5504004d40 2024-05-09T04:35:39.339 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.340+0000 7f5517e00640 1 -- 172.21.15.145:0/1300240656 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f551001c020 con 0x7f5504004d40 2024-05-09T04:35:39.340 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.341+0000 7f5517e00640 1 --2- 172.21.15.145:0/1300240656 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f5500076350 0x7f5500078810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:39.340 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.342+0000 7f5517e00640 1 -- 172.21.15.145:0/1300240656 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6172+0+0 (secure 0 0 0) 0x7f5510097ff0 con 0x7f5504004d40 2024-05-09T04:35:39.340 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.342+0000 7f551f400640 1 --2- 172.21.15.145:0/1300240656 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f5500076350 0x7f5500078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:39.341 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.342+0000 7f551f400640 1 --2- 172.21.15.145:0/1300240656 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f5500076350 0x7f5500078810 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f552015c900 tx=0x7f550c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:39.344 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.346+0000 7f5517e00640 1 -- 172.21.15.145:0/1300240656 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f55200768c0 con 0x7f5504004d40 2024-05-09T04:35:39.590 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.592+0000 7f5525c00640 1 -- 172.21.15.145:0/1300240656 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f552006a310 con 0x7f5504004d40 2024-05-09T04:35:39.591 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.593+0000 7f5517e00640 1 -- 172.21.15.145:0/1300240656 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5189 (secure 0 0 0) 0x7f5510023020 con 0x7f5504004d40 2024-05-09T04:35:39.592 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:35:39.592 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4661,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6834/1033874179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":1033874179},{"type":"v1","addr":"172.21.15.192:6834","nonce":1033874179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4664,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1787962921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6833","nonce":1787962921},{"type":"v1","addr":"172.21.15.192:6835","nonce":1787962921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5313,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6836/65063948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":65063948},{"type":"v1","addr":"172.21.15.145:6836","nonce":65063948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T04:35:23.667303+0000","modified":"2024-05-09T04:35:34.804314+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5316},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5316":{"gid":5316,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.145:6837/2593908438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6835","nonce":2593908438},{"type":"v1","addr":"172.21.15.145:6837","nonce":2593908438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8,9],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5316,"qdb_cluster":[5316]},"id":3}]} 2024-05-09T04:35:39.592 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 28 2024-05-09T04:35:39.594 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.595+0000 7f5525c00640 1 -- 172.21.15.145:0/1300240656 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f5500076350 msgr2=0x7f5500078810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:39.594 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.595+0000 7f5525c00640 1 --2- 172.21.15.145:0/1300240656 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f5500076350 0x7f5500078810 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f552015c900 tx=0x7f550c008040 comp rx=0 tx=0).stop 2024-05-09T04:35:39.594 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.595+0000 7f5525c00640 1 -- 172.21.15.145:0/1300240656 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5504004d40 msgr2=0x7f55200b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:39.594 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.595+0000 7f5525c00640 1 --2- 172.21.15.145:0/1300240656 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5504004d40 0x7f55200b32c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f551000f580 tx=0x7f551000fa40 comp rx=0 tx=0).stop 2024-05-09T04:35:39.594 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.595+0000 7f5525c00640 1 -- 172.21.15.145:0/1300240656 shutdown_connections 2024-05-09T04:35:39.594 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.595+0000 7f5525c00640 1 --2- 172.21.15.145:0/1300240656 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f5500076350 0x7f5500078810 secure :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f552015c900 tx=0x7f550c008040 comp rx=0 tx=0).stop 2024-05-09T04:35:39.594 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.595+0000 7f5525c00640 1 --2- 172.21.15.145:0/1300240656 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5520150580 0x7f552015bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:39.595 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.595+0000 7f5525c00640 1 --2- 172.21.15.145:0/1300240656 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5520073d20 0x7f55200724a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:39.595 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.595+0000 7f5525c00640 1 --2- 172.21.15.145:0/1300240656 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5504004d40 0x7f55200b32c0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:39.595 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.595+0000 7f5525c00640 1 -- 172.21.15.145:0/1300240656 >> 172.21.15.145:0/1300240656 conn(0x7f552006c8a0 msgr2=0x7f552015e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:39.595 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.596+0000 7f5525c00640 1 -- 172.21.15.145:0/1300240656 shutdown_connections 2024-05-09T04:35:39.595 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.596+0000 7f5525c00640 1 -- 172.21.15.145:0/1300240656 wait complete. 2024-05-09T04:35:39.612 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T04:35:39.804 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.805+0000 7f0b0fe00640 1 Processor -- start 2024-05-09T04:35:39.804 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.806+0000 7f0b0fe00640 1 -- start start 2024-05-09T04:35:39.806 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.807+0000 7f0b0fe00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f0b101602a0 0x7f0b1006f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:39.806 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.807+0000 7f0b0fe00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b10070c30 con 0x7f0b101633e0 2024-05-09T04:35:39.806 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.807+0000 7f0b0fe00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b10070f10 con 0x7f0b10163000 2024-05-09T04:35:39.806 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.807+0000 7f0b0fe00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f0b1006c690 con 0x7f0b101602a0 2024-05-09T04:35:39.807 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.808+0000 7f0b0f400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f0b101602a0 0x7f0b1006f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:39.807 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.808+0000 7f0b0f400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f0b101602a0 0x7f0b1006f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:56512/0 (socket says 172.21.15.145:56512) 2024-05-09T04:35:39.807 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.808+0000 7f0b0f400640 1 -- 172.21.15.145:0/1342125329 learned_addr learned my addr 172.21.15.145:0/1342125329 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:39.807 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.808+0000 7f0b0f400640 1 -- 172.21.15.145:0/1342125329 >> v1:172.21.15.192:6789/0 conn(0x7f0b10163000 legacy=0x7f0b100d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:39.807 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.808+0000 7f0b0f400640 1 -- 172.21.15.145:0/1342125329 >> v1:172.21.15.145:6789/0 conn(0x7f0b101633e0 legacy=0x7f0b10073200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:39.807 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.808+0000 7f0b0f400640 1 -- 172.21.15.145:0/1342125329 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b1006c970 con 0x7f0b101602a0 2024-05-09T04:35:39.808 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.809+0000 7f0b0f400640 1 --2- 172.21.15.145:0/1342125329 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0b101602a0 0x7f0b1006f1c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f0b08009e80 tx=0x7f0b0802a6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9853784dbeb31857 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:39.808 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.809+0000 7f0b0e000640 1 -- 172.21.15.145:0/1342125329 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b08036070 con 0x7f0b101602a0 2024-05-09T04:35:39.808 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.809+0000 7f0b0e000640 1 -- 172.21.15.145:0/1342125329 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b08031600 con 0x7f0b101602a0 2024-05-09T04:35:39.808 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.809+0000 7f0b0e000640 1 -- 172.21.15.145:0/1342125329 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b080319d0 con 0x7f0b101602a0 2024-05-09T04:35:39.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.810+0000 7f0b0fe00640 1 -- 172.21.15.145:0/1342125329 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0b101602a0 msgr2=0x7f0b1006f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:39.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.810+0000 7f0b0fe00640 1 --2- 172.21.15.145:0/1342125329 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0b101602a0 0x7f0b1006f1c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f0b08009e80 tx=0x7f0b0802a6b0 comp rx=0 tx=0).stop 2024-05-09T04:35:39.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.810+0000 7f0b0fe00640 1 -- 172.21.15.145:0/1342125329 shutdown_connections 2024-05-09T04:35:39.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.810+0000 7f0b0fe00640 1 --2- 172.21.15.145:0/1342125329 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0b101602a0 0x7f0b1006f1c0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:39.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.810+0000 7f0b0fe00640 1 -- 172.21.15.145:0/1342125329 >> 172.21.15.145:0/1342125329 conn(0x7f0b100b3ae0 msgr2=0x7f0b100b3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:39.810 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.810+0000 7f0b0fe00640 1 -- 172.21.15.145:0/1342125329 shutdown_connections 2024-05-09T04:35:39.810 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.810+0000 7f0b0fe00640 1 -- 172.21.15.145:0/1342125329 wait complete. 2024-05-09T04:35:39.813 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.812+0000 7f0b0fe00640 1 Processor -- start 2024-05-09T04:35:39.813 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.812+0000 7f0b0fe00640 1 -- start start 2024-05-09T04:35:39.814 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.813+0000 7f0b0fe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b101602a0 0x7f0b10155360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:39.814 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.813+0000 7f0b0fe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0b10163000 0x7f0b10156540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:39.814 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.813+0000 7f0b0fe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b101633e0 0x7f0b1015e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:39.814 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.813+0000 7f0b0fe00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0b10073ca0 con 0x7f0b10163000 2024-05-09T04:35:39.814 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.813+0000 7f0b0fe00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0b10073b20 con 0x7f0b101602a0 2024-05-09T04:35:39.814 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.813+0000 7f0b0fe00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0b10073e20 con 0x7f0b101633e0 2024-05-09T04:35:39.814 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.813+0000 7f0b07e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0b10163000 0x7f0b10156540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:39.814 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.813+0000 7f0b07e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0b10163000 0x7f0b10156540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:56514/0 (socket says 172.21.15.145:56514) 2024-05-09T04:35:39.814 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.813+0000 7f0b07e00640 1 -- 172.21.15.145:0/2959971219 learned_addr learned my addr 172.21.15.145:0/2959971219 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:39.814 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.814+0000 7f0b07e00640 1 -- 172.21.15.145:0/2959971219 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b101633e0 msgr2=0x7f0b1015e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:39.814 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.814+0000 7f0b07e00640 1 --2- 172.21.15.145:0/2959971219 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b101633e0 0x7f0b1015e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:39.814 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.814+0000 7f0b07e00640 1 -- 172.21.15.145:0/2959971219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b101602a0 msgr2=0x7f0b10155360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:39.814 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.814+0000 7f0b0ea00640 1 --2- 172.21.15.145:0/2959971219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b101602a0 0x7f0b10155360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:39.814 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.814+0000 7f0b07e00640 1 --2- 172.21.15.145:0/2959971219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b101602a0 0x7f0b10155360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:39.815 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.814+0000 7f0b07e00640 1 -- 172.21.15.145:0/2959971219 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b10161b80 con 0x7f0b10163000 2024-05-09T04:35:39.815 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.814+0000 7f0b07e00640 1 --2- 172.21.15.145:0/2959971219 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0b10163000 0x7f0b10156540 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f0af800c450 tx=0x7f0af800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:39.815 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.814+0000 7f0b06a00640 1 -- 172.21.15.145:0/2959971219 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af80052d0 con 0x7f0b10163000 2024-05-09T04:35:39.815 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.814+0000 7f0b06a00640 1 -- 172.21.15.145:0/2959971219 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0af8005470 con 0x7f0b10163000 2024-05-09T04:35:39.815 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.814+0000 7f0b0fe00640 1 -- 172.21.15.145:0/2959971219 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b10156a80 con 0x7f0b10163000 2024-05-09T04:35:39.815 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.814+0000 7f0b06a00640 1 -- 172.21.15.145:0/2959971219 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af8012870 con 0x7f0b10163000 2024-05-09T04:35:39.815 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.815+0000 7f0b0fe00640 1 -- 172.21.15.145:0/2959971219 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b101618a0 con 0x7f0b10163000 2024-05-09T04:35:39.815 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.816+0000 7f0b0fe00640 1 -- 172.21.15.145:0/2959971219 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b100694e0 con 0x7f0b10163000 2024-05-09T04:35:39.815 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.817+0000 7f0b06a00640 1 -- 172.21.15.145:0/2959971219 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0af8012a10 con 0x7f0b10163000 2024-05-09T04:35:39.817 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.818+0000 7f0b06a00640 1 --2- 172.21.15.145:0/2959971219 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f0aec0762d0 0x7f0aec078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:39.817 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.818+0000 7f0b06a00640 1 -- 172.21.15.145:0/2959971219 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6172+0+0 (secure 0 0 0) 0x7f0af80973c0 con 0x7f0b10163000 2024-05-09T04:35:39.817 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.818+0000 7f0b0ea00640 1 --2- 172.21.15.145:0/2959971219 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f0aec0762d0 0x7f0aec078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:39.817 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.819+0000 7f0b0ea00640 1 --2- 172.21.15.145:0/2959971219 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f0aec0762d0 0x7f0aec078790 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0b00003d90 tx=0x7f0b00009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:39.821 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:39.822+0000 7f0b06a00640 1 -- 172.21.15.145:0/2959971219 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0af8060b40 con 0x7f0b10163000 2024-05-09T04:35:40.047 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.049+0000 7f0b0fe00640 1 -- 172.21.15.145:0/2959971219 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b100743c0 con 0x7f0b10163000 2024-05-09T04:35:40.048 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.050+0000 7f0b06a00640 1 -- 172.21.15.145:0/2959971219 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+5189 (secure 0 0 0) 0x7f0af8060440 con 0x7f0b10163000 2024-05-09T04:35:40.048 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:35:40.048 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4661,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6834/1033874179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":1033874179},{"type":"v1","addr":"172.21.15.192:6834","nonce":1033874179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4664,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1787962921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6833","nonce":1787962921},{"type":"v1","addr":"172.21.15.192:6835","nonce":1787962921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5313,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6836/65063948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":65063948},{"type":"v1","addr":"172.21.15.145:6836","nonce":65063948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T04:35:23.667303+0000","modified":"2024-05-09T04:35:34.804314+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5316},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5316":{"gid":5316,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.145:6837/2593908438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6835","nonce":2593908438},{"type":"v1","addr":"172.21.15.145:6837","nonce":2593908438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8,9],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5316,"qdb_cluster":[5316]},"id":3}]} 2024-05-09T04:35:40.049 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 28 2024-05-09T04:35:40.051 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.052+0000 7f0b0fe00640 1 -- 172.21.15.145:0/2959971219 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f0aec0762d0 msgr2=0x7f0aec078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:40.051 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.052+0000 7f0b0fe00640 1 --2- 172.21.15.145:0/2959971219 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f0aec0762d0 0x7f0aec078790 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0b00003d90 tx=0x7f0b00009040 comp rx=0 tx=0).stop 2024-05-09T04:35:40.051 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.052+0000 7f0b0fe00640 1 -- 172.21.15.145:0/2959971219 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0b10163000 msgr2=0x7f0b10156540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:40.051 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.052+0000 7f0b0fe00640 1 --2- 172.21.15.145:0/2959971219 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0b10163000 0x7f0b10156540 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f0af800c450 tx=0x7f0af800c910 comp rx=0 tx=0).stop 2024-05-09T04:35:40.051 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.052+0000 7f0b0fe00640 1 -- 172.21.15.145:0/2959971219 shutdown_connections 2024-05-09T04:35:40.051 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.052+0000 7f0b0fe00640 1 --2- 172.21.15.145:0/2959971219 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f0aec0762d0 0x7f0aec078790 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:40.052 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.052+0000 7f0b0fe00640 1 --2- 172.21.15.145:0/2959971219 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b101633e0 0x7f0b1015e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:40.052 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.052+0000 7f0b0fe00640 1 --2- 172.21.15.145:0/2959971219 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0b10163000 0x7f0b10156540 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:40.052 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.052+0000 7f0b0fe00640 1 --2- 172.21.15.145:0/2959971219 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b101602a0 0x7f0b10155360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:40.052 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.052+0000 7f0b0fe00640 1 -- 172.21.15.145:0/2959971219 >> 172.21.15.145:0/2959971219 conn(0x7f0b100b3ae0 msgr2=0x7f0b1015caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:40.052 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.052+0000 7f0b0fe00640 1 -- 172.21.15.145:0/2959971219 shutdown_connections 2024-05-09T04:35:40.052 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.052+0000 7f0b0fe00640 1 -- 172.21.15.145:0/2959971219 wait complete. 2024-05-09T04:35:40.070 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T04:35:40.263 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.264+0000 7faadda00640 1 Processor -- start 2024-05-09T04:35:40.263 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.264+0000 7faadda00640 1 -- start start 2024-05-09T04:35:40.265 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.266+0000 7faadda00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7faad80d5ad0 0x7faad80d22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:40.265 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.266+0000 7faadda00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faad80b4760 0x7faad80b4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:40.265 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.266+0000 7faadda00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faad80d3e70 con 0x7faad80d3180 2024-05-09T04:35:40.265 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.266+0000 7faadda00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7faad80d4150 con 0x7faad80d5ad0 2024-05-09T04:35:40.265 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.266+0000 7faadda00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7faad80d2830 con 0x7faad80b4760 2024-05-09T04:35:40.265 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.266+0000 7faad7e00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7faad80d3180 0x7faad80d3560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:52782/0 (socket says 172.21.15.145:52782) 2024-05-09T04:35:40.265 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.266+0000 7faad7e00640 1 -- 172.21.15.145:0/3038946615 learned_addr learned my addr 172.21.15.145:0/3038946615 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:35:40.265 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.266+0000 7faad7400640 1 --2- 172.21.15.145:0/3038946615 >> v2:172.21.15.145:3300/0 conn(0x7faad80d5ad0 0x7faad80d22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:40.266 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.267+0000 7faad7400640 1 -- 172.21.15.145:0/3038946615 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faad80b4760 msgr2=0x7faad80b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:40.266 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.267+0000 7faad7400640 1 --2- 172.21.15.145:0/3038946615 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faad80b4760 0x7faad80b4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:40.266 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.267+0000 7faad7400640 1 -- 172.21.15.145:0/3038946615 >> v1:172.21.15.145:6789/0 conn(0x7faad80d3180 legacy=0x7faad80d3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:40.266 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.267+0000 7faad7400640 1 -- 172.21.15.145:0/3038946615 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faad80b6450 con 0x7faad80d5ad0 2024-05-09T04:35:40.266 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.267+0000 7faad6a00640 1 -- 172.21.15.145:0/3038946615 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2082542376 0 0) 0x7faad80d3e70 con 0x7faad80d3180 2024-05-09T04:35:40.267 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.268+0000 7faad7400640 1 --2- 172.21.15.145:0/3038946615 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faad80d5ad0 0x7faad80d22f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7faac0009d90 tx=0x7faac002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=907aa70c06f1cd36 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:40.267 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.268+0000 7faad6a00640 1 -- 172.21.15.145:0/3038946615 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac0031600 con 0x7faad80d5ad0 2024-05-09T04:35:40.267 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.268+0000 7faad6a00640 1 -- 172.21.15.145:0/3038946615 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faac00317a0 con 0x7faad80d5ad0 2024-05-09T04:35:40.267 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.268+0000 7faad6a00640 1 -- 172.21.15.145:0/3038946615 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac0031b70 con 0x7faad80d5ad0 2024-05-09T04:35:40.267 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.268+0000 7faadda00640 1 -- 172.21.15.145:0/3038946615 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faad80d5ad0 msgr2=0x7faad80d22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:40.267 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.268+0000 7faadda00640 1 --2- 172.21.15.145:0/3038946615 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faad80d5ad0 0x7faad80d22f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7faac0009d90 tx=0x7faac002a5d0 comp rx=0 tx=0).stop 2024-05-09T04:35:40.267 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.268+0000 7faadda00640 1 -- 172.21.15.145:0/3038946615 shutdown_connections 2024-05-09T04:35:40.268 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.268+0000 7faadda00640 1 --2- 172.21.15.145:0/3038946615 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faad80b4760 0x7faad80b4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:40.268 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.268+0000 7faadda00640 1 --2- 172.21.15.145:0/3038946615 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faad80d5ad0 0x7faad80d22f0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:40.268 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.268+0000 7faadda00640 1 -- 172.21.15.145:0/3038946615 >> 172.21.15.145:0/3038946615 conn(0x7faad80d4fb0 msgr2=0x7faad80d53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:40.268 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.269+0000 7faadda00640 1 -- 172.21.15.145:0/3038946615 shutdown_connections 2024-05-09T04:35:40.268 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.269+0000 7faadda00640 1 -- 172.21.15.145:0/3038946615 wait complete. 2024-05-09T04:35:40.269 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.270+0000 7faadda00640 1 Processor -- start 2024-05-09T04:35:40.269 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.270+0000 7faadda00640 1 -- start start 2024-05-09T04:35:40.271 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.271+0000 7faadda00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faad80b4760 0x7faad80d12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:40.271 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.271+0000 7faadda00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faad80d3180 0x7faad80cc080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:40.271 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.271+0000 7faadda00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faad80d5ad0 0x7faad815b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:40.271 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.271+0000 7faadda00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7faad80d1e60 con 0x7faad80b4760 2024-05-09T04:35:40.272 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.271+0000 7faadda00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7faad80b6200 con 0x7faad80d3180 2024-05-09T04:35:40.272 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.271+0000 7faadda00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7faad80b3fa0 con 0x7faad80d5ad0 2024-05-09T04:35:40.272 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.271+0000 7faad7e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faad80b4760 0x7faad80d12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:40.272 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.271+0000 7faad7e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faad80b4760 0x7faad80d12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:56532/0 (socket says 172.21.15.145:56532) 2024-05-09T04:35:40.272 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.271+0000 7faad7e00640 1 -- 172.21.15.145:0/2396029150 learned_addr learned my addr 172.21.15.145:0/2396029150 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:40.272 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.272+0000 7faad7e00640 1 -- 172.21.15.145:0/2396029150 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faad80d5ad0 msgr2=0x7faad815b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:40.272 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.272+0000 7faad7e00640 1 --2- 172.21.15.145:0/2396029150 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faad80d5ad0 0x7faad815b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:40.272 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.272+0000 7faad7e00640 1 -- 172.21.15.145:0/2396029150 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faad80d3180 msgr2=0x7faad80cc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:40.272 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.272+0000 7faad7e00640 1 --2- 172.21.15.145:0/2396029150 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faad80d3180 0x7faad80cc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:40.272 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.272+0000 7faad7e00640 1 -- 172.21.15.145:0/2396029150 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faad80ccb40 con 0x7faad80b4760 2024-05-09T04:35:40.272 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.272+0000 7faad7e00640 1 --2- 172.21.15.145:0/2396029150 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faad80b4760 0x7faad80d12a0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7faac800c4c0 tx=0x7faac800c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:40.272 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.272+0000 7faacfe00640 1 -- 172.21.15.145:0/2396029150 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac8018070 con 0x7faad80b4760 2024-05-09T04:35:40.272 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.272+0000 7faacfe00640 1 -- 172.21.15.145:0/2396029150 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faac8007500 con 0x7faad80b4760 2024-05-09T04:35:40.272 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.272+0000 7faadda00640 1 -- 172.21.15.145:0/2396029150 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faad80cce20 con 0x7faad80b4760 2024-05-09T04:35:40.272 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.272+0000 7faacfe00640 1 -- 172.21.15.145:0/2396029150 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac8004a20 con 0x7faad80b4760 2024-05-09T04:35:40.273 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.272+0000 7faadda00640 1 -- 172.21.15.145:0/2396029150 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faad80cd3e0 con 0x7faad80b4760 2024-05-09T04:35:40.273 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.274+0000 7faadda00640 1 -- 172.21.15.145:0/2396029150 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faad80cd710 con 0x7faad80b4760 2024-05-09T04:35:40.273 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.275+0000 7faacfe00640 1 -- 172.21.15.145:0/2396029150 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7faac801b070 con 0x7faad80b4760 2024-05-09T04:35:40.275 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.276+0000 7faacfe00640 1 --2- 172.21.15.145:0/2396029150 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7faab40762d0 0x7faab4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:40.275 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.276+0000 7faacfe00640 1 -- 172.21.15.145:0/2396029150 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6172+0+0 (secure 0 0 0) 0x7faac8096b40 con 0x7faad80b4760 2024-05-09T04:35:40.275 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.276+0000 7faad7400640 1 --2- 172.21.15.145:0/2396029150 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7faab40762d0 0x7faab4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:40.275 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.277+0000 7faad7400640 1 --2- 172.21.15.145:0/2396029150 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7faab40762d0 0x7faab4078790 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7faac0034000 tx=0x7faac002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:40.279 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.281+0000 7faacfe00640 1 -- 172.21.15.145:0/2396029150 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7faac80602d0 con 0x7faad80b4760 2024-05-09T04:35:40.476 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.477+0000 7faadda00640 1 -- 172.21.15.145:0/2396029150 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faad806a310 con 0x7faad80b4760 2024-05-09T04:35:40.478 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.479+0000 7faacfe00640 1 -- 172.21.15.145:0/2396029150 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) ==== 74+0+18633 (secure 0 0 0) 0x7faac805fbd0 con 0x7faad80b4760 2024-05-09T04:35:40.478 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:35:40.478 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":51,"fsid":"957ee6c1-073b-44cf-9b93-9491b3fdfc1d","created":"2024-05-09T04:33:26.544367+0000","modified":"2024-05-09T04:35:36.336628+0000","last_up_change":"2024-05-09T04:33:39.039318+0000","last_in_change":"2024-05-09T04:33:29.902361+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T04:33:39.795344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-05-09T04:35:21.570048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-05-09T04:35:22.639737+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data_extra1","create_time":"2024-05-09T04:35:33.506234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22e550ad-847b-4952-80e6-63d859ed1a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6801","nonce":3182572912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6803","nonce":3182572912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6807","nonce":3182572912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6805","nonce":3182572912}]},"public_addr":"172.21.15.145:6801/3182572912","cluster_addr":"172.21.15.145:6803/3182572912","heartbeat_back_addr":"172.21.15.145:6807/3182572912","heartbeat_front_addr":"172.21.15.145:6805/3182572912","state":["exists","up"]},{"osd":1,"uuid":"0f6b9fca-0c41-4145-8d74-598ddb42f2a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6825","nonce":1845225555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6827","nonce":1845225555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6831","nonce":1845225555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6829","nonce":1845225555}]},"public_addr":"172.21.15.145:6825/1845225555","cluster_addr":"172.21.15.145:6827/1845225555","heartbeat_back_addr":"172.21.15.145:6831/1845225555","heartbeat_front_addr":"172.21.15.145:6829/1845225555","state":["exists","up"]},{"osd":2,"uuid":"b3b1f67e-5790-483f-8610-e05a5bce4c30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6817","nonce":2168958993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6819","nonce":2168958993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6823","nonce":2168958993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6821","nonce":2168958993}]},"public_addr":"172.21.15.145:6817/2168958993","cluster_addr":"172.21.15.145:6819/2168958993","heartbeat_back_addr":"172.21.15.145:6823/2168958993","heartbeat_front_addr":"172.21.15.145:6821/2168958993","state":["exists","up"]},{"osd":3,"uuid":"884787a5-e037-4d31-9aae-743030eba4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6809","nonce":337009826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6811","nonce":337009826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6815","nonce":337009826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6813","nonce":337009826}]},"public_addr":"172.21.15.145:6809/337009826","cluster_addr":"172.21.15.145:6811/337009826","heartbeat_back_addr":"172.21.15.145:6815/337009826","heartbeat_front_addr":"172.21.15.145:6813/337009826","state":["exists","up"]},{"osd":4,"uuid":"0759fc1e-c5fc-4a2f-86bb-e8eccd83f833","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6818","nonce":3117700995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6821","nonce":3117700995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6827","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6829","nonce":3117700995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6823","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6825","nonce":3117700995}]},"public_addr":"172.21.15.192:6818/3117700995","cluster_addr":"172.21.15.192:6821/3117700995","heartbeat_back_addr":"172.21.15.192:6829/3117700995","heartbeat_front_addr":"172.21.15.192:6825/3117700995","state":["exists","up"]},{"osd":5,"uuid":"9443cc22-9379-46e3-9173-98b592bf2c16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6801","nonce":1328373268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6803","nonce":1328373268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6807","nonce":1328373268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6805","nonce":1328373268}]},"public_addr":"172.21.15.192:6801/1328373268","cluster_addr":"172.21.15.192:6803/1328373268","heartbeat_back_addr":"172.21.15.192:6807/1328373268","heartbeat_front_addr":"172.21.15.192:6805/1328373268","state":["exists","up"]},{"osd":6,"uuid":"d05a3311-505a-45b5-85b9-aa3113e96987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6817","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6820","nonce":1868011057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6824","nonce":1868011057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6831","nonce":1868011057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6828","nonce":1868011057}]},"public_addr":"172.21.15.192:6820/1868011057","cluster_addr":"172.21.15.192:6824/1868011057","heartbeat_back_addr":"172.21.15.192:6831/1868011057","heartbeat_front_addr":"172.21.15.192:6828/1868011057","state":["exists","up"]},{"osd":7,"uuid":"c7c23dbc-2029-43ec-9f53-25f3d58019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6809","nonce":3368512157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6811","nonce":3368512157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6815","nonce":3368512157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6813","nonce":3368512157}]},"public_addr":"172.21.15.192:6809/3368512157","cluster_addr":"172.21.15.192:6811/3368512157","heartbeat_back_addr":"172.21.15.192:6815/3368512157","heartbeat_front_addr":"172.21.15.192:6813/3368512157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.091188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.032567+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:34.797468+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.228839+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.622400+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.473103+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.566092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.349395+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.c","mappings":[{"from":0,"to":6}]},{"pgid":"8.e","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T04:35:40.481 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.482+0000 7faadda00640 1 -- 172.21.15.145:0/2396029150 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7faab40762d0 msgr2=0x7faab4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:40.481 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.482+0000 7faadda00640 1 --2- 172.21.15.145:0/2396029150 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7faab40762d0 0x7faab4078790 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7faac0034000 tx=0x7faac002e000 comp rx=0 tx=0).stop 2024-05-09T04:35:40.481 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.482+0000 7faadda00640 1 -- 172.21.15.145:0/2396029150 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faad80b4760 msgr2=0x7faad80d12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:40.481 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.482+0000 7faadda00640 1 --2- 172.21.15.145:0/2396029150 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faad80b4760 0x7faad80d12a0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7faac800c4c0 tx=0x7faac800c980 comp rx=0 tx=0).stop 2024-05-09T04:35:40.481 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.482+0000 7faadda00640 1 -- 172.21.15.145:0/2396029150 shutdown_connections 2024-05-09T04:35:40.481 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.482+0000 7faadda00640 1 --2- 172.21.15.145:0/2396029150 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7faab40762d0 0x7faab4078790 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:40.482 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.482+0000 7faadda00640 1 --2- 172.21.15.145:0/2396029150 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faad80d5ad0 0x7faad815b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:40.482 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.482+0000 7faadda00640 1 --2- 172.21.15.145:0/2396029150 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faad80d3180 0x7faad80cc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:40.482 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.482+0000 7faadda00640 1 --2- 172.21.15.145:0/2396029150 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7faad80b4760 0x7faad80d12a0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:40.482 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.482+0000 7faadda00640 1 -- 172.21.15.145:0/2396029150 >> 172.21.15.145:0/2396029150 conn(0x7faad80d4fb0 msgr2=0x7faad8162680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:40.482 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.482+0000 7faadda00640 1 -- 172.21.15.145:0/2396029150 shutdown_connections 2024-05-09T04:35:40.482 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.482+0000 7faadda00640 1 -- 172.21.15.145:0/2396029150 wait complete. 2024-05-09T04:35:40.500 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-09T04:35:40.702 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.703+0000 7fc054000640 1 Processor -- start 2024-05-09T04:35:40.702 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.703+0000 7fc054000640 1 -- start start 2024-05-09T04:35:40.704 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.705+0000 7fc054000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fc04c145ad0 0x7fc04c06e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:40.704 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.705+0000 7fc054000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc04c0ce440 0x7fc04c15d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:40.704 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.705+0000 7fc054000640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc04c06ebd0 con 0x7fc04c0ce060 2024-05-09T04:35:40.704 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.705+0000 7fc054000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fc04c073460 con 0x7fc04c145ad0 2024-05-09T04:35:40.704 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.705+0000 7fc054000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc04c06f750 con 0x7fc04c0ce440 2024-05-09T04:35:40.704 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.705+0000 7fc050c00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fc04c145ad0 0x7fc04c06e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:40.704 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.705+0000 7fc050c00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fc04c145ad0 0x7fc04c06e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:56548/0 (socket says 172.21.15.145:56548) 2024-05-09T04:35:40.704 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.705+0000 7fc050c00640 1 -- 172.21.15.145:0/2793665749 learned_addr learned my addr 172.21.15.145:0/2793665749 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:40.705 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.705+0000 7fc052000640 1 --2- 172.21.15.145:0/2793665749 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc04c0ce440 0x7fc04c15d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:40.705 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.706+0000 7fc050c00640 1 -- 172.21.15.145:0/2793665749 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc04c0ce440 msgr2=0x7fc04c15d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:40.705 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.706+0000 7fc050c00640 1 --2- 172.21.15.145:0/2793665749 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc04c0ce440 0x7fc04c15d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:40.705 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.706+0000 7fc050c00640 1 -- 172.21.15.145:0/2793665749 >> v1:172.21.15.192:6789/0 conn(0x7fc04c0ce060 legacy=0x7fc04c1453c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:40.705 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.706+0000 7fc050c00640 1 -- 172.21.15.145:0/2793665749 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc04c06eeb0 con 0x7fc04c145ad0 2024-05-09T04:35:40.706 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.707+0000 7fc050c00640 1 --2- 172.21.15.145:0/2793665749 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc04c145ad0 0x7fc04c06e510 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fc0380098d0 tx=0x7fc03802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec023c825da00972 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:40.706 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.707+0000 7fc04be00640 1 -- 172.21.15.145:0/2793665749 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc038037070 con 0x7fc04c145ad0 2024-05-09T04:35:40.706 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.707+0000 7fc04be00640 1 -- 172.21.15.145:0/2793665749 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc038032440 con 0x7fc04c145ad0 2024-05-09T04:35:40.707 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.707+0000 7fc04be00640 1 -- 172.21.15.145:0/2793665749 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc038032810 con 0x7fc04c145ad0 2024-05-09T04:35:40.707 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.708+0000 7fc054000640 1 -- 172.21.15.145:0/2793665749 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc04c145ad0 msgr2=0x7fc04c06e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:40.707 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.708+0000 7fc054000640 1 --2- 172.21.15.145:0/2793665749 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc04c145ad0 0x7fc04c06e510 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fc0380098d0 tx=0x7fc03802a210 comp rx=0 tx=0).stop 2024-05-09T04:35:40.707 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.708+0000 7fc054000640 1 -- 172.21.15.145:0/2793665749 shutdown_connections 2024-05-09T04:35:40.707 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.708+0000 7fc054000640 1 --2- 172.21.15.145:0/2793665749 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc04c0ce440 0x7fc04c15d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:40.707 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.708+0000 7fc054000640 1 --2- 172.21.15.145:0/2793665749 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc04c145ad0 0x7fc04c06e510 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:40.707 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.708+0000 7fc054000640 1 -- 172.21.15.145:0/2793665749 >> 172.21.15.145:0/2793665749 conn(0x7fc04c06c8d0 msgr2=0x7fc04c06cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:40.707 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.708+0000 7fc054000640 1 -- 172.21.15.145:0/2793665749 shutdown_connections 2024-05-09T04:35:40.707 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.708+0000 7fc054000640 1 -- 172.21.15.145:0/2793665749 wait complete. 2024-05-09T04:35:40.708 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.709+0000 7fc054000640 1 Processor -- start 2024-05-09T04:35:40.708 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.710+0000 7fc054000640 1 -- start start 2024-05-09T04:35:40.711 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.710+0000 7fc054000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc04c0ce060 0x7fc04c137730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:40.711 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.710+0000 7fc054000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc04c0ce440 0x7fc04c13c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:40.711 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.710+0000 7fc054000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc04c145ad0 0x7fc04c13dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:40.711 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.710+0000 7fc054000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc04c15d8c0 con 0x7fc04c0ce440 2024-05-09T04:35:40.711 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.710+0000 7fc054000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc04c06f470 con 0x7fc04c0ce060 2024-05-09T04:35:40.711 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.710+0000 7fc054000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc04c15da40 con 0x7fc04c145ad0 2024-05-09T04:35:40.711 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.711+0000 7fc050c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc04c0ce440 0x7fc04c13c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:40.711 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.711+0000 7fc050c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc04c0ce440 0x7fc04c13c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:56550/0 (socket says 172.21.15.145:56550) 2024-05-09T04:35:40.711 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.711+0000 7fc050c00640 1 -- 172.21.15.145:0/2767437855 learned_addr learned my addr 172.21.15.145:0/2767437855 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:40.712 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.711+0000 7fc052000640 1 --2- 172.21.15.145:0/2767437855 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc04c145ad0 0x7fc04c13dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:40.712 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.711+0000 7fc051600640 1 --2- 172.21.15.145:0/2767437855 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc04c0ce060 0x7fc04c137730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:40.712 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.711+0000 7fc050c00640 1 -- 172.21.15.145:0/2767437855 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc04c145ad0 msgr2=0x7fc04c13dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:40.712 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.711+0000 7fc050c00640 1 --2- 172.21.15.145:0/2767437855 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc04c145ad0 0x7fc04c13dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:40.712 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.711+0000 7fc050c00640 1 -- 172.21.15.145:0/2767437855 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc04c0ce060 msgr2=0x7fc04c137730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:40.712 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.711+0000 7fc050c00640 1 --2- 172.21.15.145:0/2767437855 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc04c0ce060 0x7fc04c137730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:40.712 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.711+0000 7fc050c00640 1 -- 172.21.15.145:0/2767437855 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc04c15abd0 con 0x7fc04c0ce440 2024-05-09T04:35:40.712 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.711+0000 7fc051600640 1 --2- 172.21.15.145:0/2767437855 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc04c0ce060 0x7fc04c137730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:35:40.712 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.712+0000 7fc050c00640 1 --2- 172.21.15.145:0/2767437855 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc04c0ce440 0x7fc04c13c0d0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fc038035040 tx=0x7fc0380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:40.712 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.712+0000 7fc049600640 1 -- 172.21.15.145:0/2767437855 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc038041070 con 0x7fc04c0ce440 2024-05-09T04:35:40.712 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.712+0000 7fc049600640 1 -- 172.21.15.145:0/2767437855 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc038004220 con 0x7fc04c0ce440 2024-05-09T04:35:40.712 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.712+0000 7fc049600640 1 -- 172.21.15.145:0/2767437855 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc038037040 con 0x7fc04c0ce440 2024-05-09T04:35:40.712 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.712+0000 7fc054000640 1 -- 172.21.15.145:0/2767437855 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc04c15ae70 con 0x7fc04c0ce440 2024-05-09T04:35:40.712 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.712+0000 7fc054000640 1 -- 172.21.15.145:0/2767437855 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc04c15b430 con 0x7fc04c0ce440 2024-05-09T04:35:40.713 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.713+0000 7fc054000640 1 -- 172.21.15.145:0/2767437855 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc04c06a310 con 0x7fc04c0ce440 2024-05-09T04:35:40.713 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.714+0000 7fc049600640 1 -- 172.21.15.145:0/2767437855 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc038003610 con 0x7fc04c0ce440 2024-05-09T04:35:40.714 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.715+0000 7fc049600640 1 --2- 172.21.15.145:0/2767437855 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fc02c076300 0x7fc02c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:40.714 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.715+0000 7fc049600640 1 -- 172.21.15.145:0/2767437855 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6172+0+0 (secure 0 0 0) 0x7fc0380b7ed0 con 0x7fc04c0ce440 2024-05-09T04:35:40.714 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.715+0000 7fc051600640 1 --2- 172.21.15.145:0/2767437855 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fc02c076300 0x7fc02c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:40.715 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.716+0000 7fc051600640 1 --2- 172.21.15.145:0/2767437855 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fc02c076300 0x7fc02c0787c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc040003eb0 tx=0x7fc040009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:40.718 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.720+0000 7fc049600640 1 -- 172.21.15.145:0/2767437855 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc038081650 con 0x7fc04c0ce440 2024-05-09T04:35:40.964 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.965+0000 7fc054000640 1 -- 172.21.15.145:0/2767437855 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fc04c0cf810 con 0x7fc04c0ce440 2024-05-09T04:35:40.965 INFO:tasks.ceph.mon.a.smithi145.stderr:2024-05-09T04:35:40.966+0000 7f50dee00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T04:35:40.968 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.970+0000 7fc049600640 1 -- 172.21.15.145:0/2767437855 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v29) ==== 161+0+0 (secure 0 0 0) 0x7fc038080f50 con 0x7fc04c0ce440 2024-05-09T04:35:40.969 INFO:teuthology.orchestra.run.smithi145.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-09T04:35:40.972 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.972+0000 7fc054000640 1 -- 172.21.15.145:0/2767437855 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fc02c076300 msgr2=0x7fc02c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:40.972 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.973+0000 7fc054000640 1 --2- 172.21.15.145:0/2767437855 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fc02c076300 0x7fc02c0787c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc040003eb0 tx=0x7fc040009040 comp rx=0 tx=0).stop 2024-05-09T04:35:40.972 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.973+0000 7fc054000640 1 -- 172.21.15.145:0/2767437855 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc04c0ce440 msgr2=0x7fc04c13c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:40.972 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.973+0000 7fc054000640 1 --2- 172.21.15.145:0/2767437855 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc04c0ce440 0x7fc04c13c0d0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fc038035040 tx=0x7fc0380046c0 comp rx=0 tx=0).stop 2024-05-09T04:35:40.972 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.973+0000 7fc054000640 1 -- 172.21.15.145:0/2767437855 shutdown_connections 2024-05-09T04:35:40.972 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.973+0000 7fc054000640 1 --2- 172.21.15.145:0/2767437855 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fc02c076300 0x7fc02c0787c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:40.972 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.973+0000 7fc054000640 1 --2- 172.21.15.145:0/2767437855 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc04c145ad0 0x7fc04c13dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:40.972 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.973+0000 7fc054000640 1 --2- 172.21.15.145:0/2767437855 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc04c0ce440 0x7fc04c13c0d0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:40.972 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.973+0000 7fc054000640 1 --2- 172.21.15.145:0/2767437855 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc04c0ce060 0x7fc04c137730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:40.973 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.973+0000 7fc054000640 1 -- 172.21.15.145:0/2767437855 >> 172.21.15.145:0/2767437855 conn(0x7fc04c06c8d0 msgr2=0x7fc04c0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:40.973 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.973+0000 7fc054000640 1 -- 172.21.15.145:0/2767437855 shutdown_connections 2024-05-09T04:35:40.973 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:40.973+0000 7fc054000640 1 -- 172.21.15.145:0/2767437855 wait complete. 2024-05-09T04:35:40.993 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-09T04:35:40.999 INFO:tasks.ceph.mds.c.smithi145.stderr: -3253> 2024-05-09T04:35:26.174+0000 7fcbd3a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-09T04:35:41.070 INFO:tasks.ceph.mds.c.smithi145.stdout:starting mds.c at 2024-05-09T04:35:41.173 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.174+0000 7f6773e00640 1 Processor -- start 2024-05-09T04:35:41.173 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.174+0000 7f6773e00640 1 -- start start 2024-05-09T04:35:41.174 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.176+0000 7f6773e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f677415f100 0x7f67740d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:41.175 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.176+0000 7f6773e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f67740d2060 0x7f67740d3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:41.175 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.176+0000 7f6773e00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67740cfda0 con 0x7f677415f4e0 2024-05-09T04:35:41.175 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.176+0000 7f6773e00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f67740b8e70 con 0x7f677415f100 2024-05-09T04:35:41.175 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.176+0000 7f6773e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f67740d0080 con 0x7f67740d2060 2024-05-09T04:35:41.175 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.176+0000 7f6772a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f677415f100 0x7f67740d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:41.175 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.176+0000 7f676be00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f677415f4e0 0x7f67740d1950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:52030/0 (socket says 172.21.15.145:52030) 2024-05-09T04:35:41.175 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.176+0000 7f676be00640 1 -- 172.21.15.145:0/3967535864 learned_addr learned my addr 172.21.15.145:0/3967535864 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:35:41.176 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.176+0000 7f6773400640 1 --2- 172.21.15.145:0/3967535864 >> v2:172.21.15.192:3300/0 conn(0x7f67740d2060 0x7f67740d3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:41.176 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.176+0000 7f6772a00640 1 -- 172.21.15.145:0/3967535864 >> v2:172.21.15.192:3300/0 conn(0x7f67740d2060 msgr2=0x7f67740d3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:41.176 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.176+0000 7f6772a00640 1 --2- 172.21.15.145:0/3967535864 >> v2:172.21.15.192:3300/0 conn(0x7f67740d2060 0x7f67740d3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:41.176 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.176+0000 7f6772a00640 1 -- 172.21.15.145:0/3967535864 >> v1:172.21.15.145:6789/0 conn(0x7f677415f4e0 legacy=0x7f67740d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:41.176 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.177+0000 7f6772a00640 1 -- 172.21.15.145:0/3967535864 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67740d0360 con 0x7f677415f100 2024-05-09T04:35:41.176 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.176+0000 7f6772000640 1 -- 172.21.15.145:0/3967535864 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3447085253 0 0) 0x7f67740cfda0 con 0x7f677415f4e0 2024-05-09T04:35:41.176 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.177+0000 7f6773400640 1 --2- 172.21.15.145:0/3967535864 >> v2:172.21.15.192:3300/0 conn(0x7f67740d2060 0x7f67740d3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:35:41.177 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.178+0000 7f6772a00640 1 --2- 172.21.15.145:0/3967535864 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f677415f100 0x7f67740d35c0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f67740d40a0 tx=0x7f676402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da0954c84d188b99 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:41.177 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.178+0000 7f6772000640 1 -- 172.21.15.145:0/3967535864 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6764037070 con 0x7f677415f100 2024-05-09T04:35:41.177 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.178+0000 7f6772000640 1 -- 172.21.15.145:0/3967535864 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6764032440 con 0x7f677415f100 2024-05-09T04:35:41.177 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.178+0000 7f6772000640 1 -- 172.21.15.145:0/3967535864 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6764032810 con 0x7f677415f100 2024-05-09T04:35:41.177 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.178+0000 7f6773e00640 1 -- 172.21.15.145:0/3967535864 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f677415f100 msgr2=0x7f67740d35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:41.178 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.178+0000 7f6773e00640 1 --2- 172.21.15.145:0/3967535864 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f677415f100 0x7f67740d35c0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f67740d40a0 tx=0x7f676402a1a0 comp rx=0 tx=0).stop 2024-05-09T04:35:41.178 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.178+0000 7f6773e00640 1 -- 172.21.15.145:0/3967535864 shutdown_connections 2024-05-09T04:35:41.178 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.178+0000 7f6773e00640 1 --2- 172.21.15.145:0/3967535864 >> v2:172.21.15.192:3300/0 conn(0x7f67740d2060 0x7f67740d3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:41.178 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.178+0000 7f6773e00640 1 --2- 172.21.15.145:0/3967535864 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f677415f100 0x7f67740d35c0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:41.178 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.178+0000 7f6773e00640 1 -- 172.21.15.145:0/3967535864 >> 172.21.15.145:0/3967535864 conn(0x7f677409e2e0 msgr2=0x7f67740d5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:41.178 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.179+0000 7f6773e00640 1 -- 172.21.15.145:0/3967535864 shutdown_connections 2024-05-09T04:35:41.178 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.179+0000 7f6773e00640 1 -- 172.21.15.145:0/3967535864 wait complete. 2024-05-09T04:35:41.179 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.180+0000 7f6773e00640 1 Processor -- start 2024-05-09T04:35:41.179 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.180+0000 7f6773e00640 1 -- start start 2024-05-09T04:35:41.180 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.181+0000 7f6773e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f67740d2060 0x7f67741522c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:41.181 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.181+0000 7f6773e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f677415f100 0x7f67740cec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:41.181 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.181+0000 7f6773e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f677415f4e0 0x7f6774160940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:41.181 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.181+0000 7f6773e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f67740d0e40 con 0x7f67740d2060 2024-05-09T04:35:41.181 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.181+0000 7f6773e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f67740d0920 con 0x7f677415f4e0 2024-05-09T04:35:41.181 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.181+0000 7f6773e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f67740c8b50 con 0x7f677415f100 2024-05-09T04:35:41.181 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.182+0000 7f6772a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f67740d2060 0x7f67741522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:41.181 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.182+0000 7f6772a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f67740d2060 0x7f67741522c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:42298/0 (socket says 172.21.15.145:42298) 2024-05-09T04:35:41.181 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.182+0000 7f6772a00640 1 -- 172.21.15.145:0/3229793290 learned_addr learned my addr 172.21.15.145:0/3229793290 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:41.181 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.182+0000 7f6772a00640 1 -- 172.21.15.145:0/3229793290 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f677415f100 msgr2=0x7f67740cec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:41.181 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.182+0000 7f6772a00640 1 --2- 172.21.15.145:0/3229793290 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f677415f100 0x7f67740cec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:41.181 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.182+0000 7f6772a00640 1 -- 172.21.15.145:0/3229793290 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f677415f4e0 msgr2=0x7f6774160940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:41.181 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.182+0000 7f6772a00640 1 --2- 172.21.15.145:0/3229793290 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f677415f4e0 0x7f6774160940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:41.181 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.182+0000 7f6772a00640 1 -- 172.21.15.145:0/3229793290 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67740c8df0 con 0x7f67740d2060 2024-05-09T04:35:41.182 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.182+0000 7f6772a00640 1 --2- 172.21.15.145:0/3229793290 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f67740d2060 0x7f67741522c0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f6764000c00 tx=0x7f67640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:41.182 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.183+0000 7f676aa00640 1 -- 172.21.15.145:0/3229793290 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6764037040 con 0x7f67740d2060 2024-05-09T04:35:41.183 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.183+0000 7f676aa00640 1 -- 172.21.15.145:0/3229793290 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6764042070 con 0x7f67740d2060 2024-05-09T04:35:41.183 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.183+0000 7f676aa00640 1 -- 172.21.15.145:0/3229793290 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6764030bd0 con 0x7f67740d2060 2024-05-09T04:35:41.183 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.183+0000 7f6773e00640 1 -- 172.21.15.145:0/3229793290 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67740c90d0 con 0x7f67740d2060 2024-05-09T04:35:41.183 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.183+0000 7f6773e00640 1 -- 172.21.15.145:0/3229793290 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6774152800 con 0x7f67740d2060 2024-05-09T04:35:41.183 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.184+0000 7f6773e00640 1 -- 172.21.15.145:0/3229793290 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f677406a310 con 0x7f67740d2060 2024-05-09T04:35:41.183 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.185+0000 7f676aa00640 1 -- 172.21.15.145:0/3229793290 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f6764047460 con 0x7f67740d2060 2024-05-09T04:35:41.185 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.186+0000 7f676aa00640 1 --2- 172.21.15.145:0/3229793290 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f67500762d0 0x7f6750078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:41.185 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.186+0000 7f676aa00640 1 -- 172.21.15.145:0/3229793290 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6258+0+0 (secure 0 0 0) 0x7f67640b8520 con 0x7f67740d2060 2024-05-09T04:35:41.185 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.186+0000 7f676be00640 1 --2- 172.21.15.145:0/3229793290 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f67500762d0 0x7f6750078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:41.185 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.187+0000 7f676be00640 1 --2- 172.21.15.145:0/3229793290 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f67500762d0 0x7f6750078790 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f675c007fd0 tx=0x7f675c007960 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:41.189 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.191+0000 7f676aa00640 1 -- 172.21.15.145:0/3229793290 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6764032e20 con 0x7f67740d2060 2024-05-09T04:35:41.437 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.438+0000 7f6773e00640 1 -- 172.21.15.145:0/3229793290 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f677408f090 con 0x7f67740d2060 2024-05-09T04:35:41.972 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.974+0000 7f676aa00640 1 -- 172.21.15.145:0/3229793290 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v30) ==== 104+0+0 (secure 0 0 0) 0x7f6764081bb0 con 0x7f67740d2060 2024-05-09T04:35:41.976 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.977+0000 7f6773e00640 1 -- 172.21.15.145:0/3229793290 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f67500762d0 msgr2=0x7f6750078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:41.976 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.977+0000 7f6773e00640 1 --2- 172.21.15.145:0/3229793290 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f67500762d0 0x7f6750078790 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f675c007fd0 tx=0x7f675c007960 comp rx=0 tx=0).stop 2024-05-09T04:35:41.976 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.977+0000 7f6773e00640 1 -- 172.21.15.145:0/3229793290 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f67740d2060 msgr2=0x7f67741522c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:41.976 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.977+0000 7f6773e00640 1 --2- 172.21.15.145:0/3229793290 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f67740d2060 0x7f67741522c0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f6764000c00 tx=0x7f67640046c0 comp rx=0 tx=0).stop 2024-05-09T04:35:41.977 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.978+0000 7f6773e00640 1 -- 172.21.15.145:0/3229793290 shutdown_connections 2024-05-09T04:35:41.977 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.978+0000 7f6773e00640 1 --2- 172.21.15.145:0/3229793290 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f67500762d0 0x7f6750078790 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:41.977 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.978+0000 7f6773e00640 1 --2- 172.21.15.145:0/3229793290 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f677415f4e0 0x7f6774160940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:41.977 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.978+0000 7f6773e00640 1 --2- 172.21.15.145:0/3229793290 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f677415f100 0x7f67740cec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:41.977 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.978+0000 7f6773e00640 1 --2- 172.21.15.145:0/3229793290 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f67740d2060 0x7f67741522c0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:41.977 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.978+0000 7f6773e00640 1 -- 172.21.15.145:0/3229793290 >> 172.21.15.145:0/3229793290 conn(0x7f677409e2e0 msgr2=0x7f67740b9630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:41.977 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.978+0000 7f6773e00640 1 -- 172.21.15.145:0/3229793290 shutdown_connections 2024-05-09T04:35:41.978 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:41.978+0000 7f6773e00640 1 -- 172.21.15.145:0/3229793290 wait complete. 2024-05-09T04:35:41.998 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-09T04:35:42.191 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.192+0000 7f7738e00640 1 Processor -- start 2024-05-09T04:35:42.191 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.192+0000 7f7738e00640 1 -- start start 2024-05-09T04:35:42.193 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.194+0000 7f7738e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f77340d33f0 0x7f77340d5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:42.193 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.194+0000 7f7738e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f77340b2600 0x7f77340b2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:42.193 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.194+0000 7f7738e00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77340d0fd0 con 0x7f77340b40a0 2024-05-09T04:35:42.194 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.194+0000 7f7738e00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f77340b2170 con 0x7f77340d33f0 2024-05-09T04:35:42.194 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.194+0000 7f7738e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f77340d0a10 con 0x7f77340b2600 2024-05-09T04:35:42.194 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.194+0000 7f7733400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f77340d33f0 0x7f77340d5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:42.194 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.194+0000 7f7733400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f77340d33f0 0x7f77340d5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:42306/0 (socket says 172.21.15.145:42306) 2024-05-09T04:35:42.194 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.194+0000 7f7733400640 1 -- 172.21.15.145:0/1456172313 learned_addr learned my addr 172.21.15.145:0/1456172313 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:42.194 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.194+0000 7f7732a00640 1 --1- 172.21.15.145:0/1456172313 >> v1:172.21.15.145:6789/0 conn(0x7f77340b40a0 0x7f77340d37d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:52044/0 (socket says 172.21.15.145:52044) 2024-05-09T04:35:42.194 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.195+0000 7f7733400640 1 -- 172.21.15.145:0/1456172313 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f77340b2600 msgr2=0x7f77340b2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:42.194 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.195+0000 7f7733e00640 1 --2- 172.21.15.145:0/1456172313 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f77340b2600 0x7f77340b2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:42.194 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.195+0000 7f7733400640 1 --2- 172.21.15.145:0/1456172313 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f77340b2600 0x7f77340b2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:42.194 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.195+0000 7f7733400640 1 -- 172.21.15.145:0/1456172313 >> v1:172.21.15.145:6789/0 conn(0x7f77340b40a0 legacy=0x7f77340d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:42.194 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.195+0000 7f7733400640 1 -- 172.21.15.145:0/1456172313 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77340d0cf0 con 0x7f77340d33f0 2024-05-09T04:35:42.194 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.195+0000 7f7732000640 1 -- 172.21.15.145:0/1456172313 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1192424086 0 0) 0x7f77340d0fd0 con 0x7f77340b40a0 2024-05-09T04:35:42.195 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.196+0000 7f7733400640 1 --2- 172.21.15.145:0/1456172313 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f77340d33f0 0x7f77340d5990 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7724009e50 tx=0x7f772402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86a6f1f2da135d42 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:42.196 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.196+0000 7f7732000640 1 -- 172.21.15.145:0/1456172313 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7724036070 con 0x7f77340d33f0 2024-05-09T04:35:42.196 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.196+0000 7f7732000640 1 -- 172.21.15.145:0/1456172313 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7724031600 con 0x7f77340d33f0 2024-05-09T04:35:42.196 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.196+0000 7f7732000640 1 -- 172.21.15.145:0/1456172313 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77240319d0 con 0x7f77340d33f0 2024-05-09T04:35:42.196 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.197+0000 7f7738e00640 1 -- 172.21.15.145:0/1456172313 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f77340d33f0 msgr2=0x7f77340d5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:42.196 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.197+0000 7f7738e00640 1 --2- 172.21.15.145:0/1456172313 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f77340d33f0 0x7f77340d5990 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7724009e50 tx=0x7f772402a680 comp rx=0 tx=0).stop 2024-05-09T04:35:42.196 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.197+0000 7f7738e00640 1 -- 172.21.15.145:0/1456172313 shutdown_connections 2024-05-09T04:35:42.196 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.197+0000 7f7738e00640 1 --2- 172.21.15.145:0/1456172313 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f77340b2600 0x7f77340b2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:42.196 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.197+0000 7f7738e00640 1 --2- 172.21.15.145:0/1456172313 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f77340d33f0 0x7f77340d5990 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:42.196 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.197+0000 7f7738e00640 1 -- 172.21.15.145:0/1456172313 >> 172.21.15.145:0/1456172313 conn(0x7f773409e400 msgr2=0x7f77340d50c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:42.196 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.197+0000 7f7738e00640 1 -- 172.21.15.145:0/1456172313 shutdown_connections 2024-05-09T04:35:42.196 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.197+0000 7f7738e00640 1 -- 172.21.15.145:0/1456172313 wait complete. 2024-05-09T04:35:42.197 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.198+0000 7f7738e00640 1 Processor -- start 2024-05-09T04:35:42.197 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.198+0000 7f7738e00640 1 -- start start 2024-05-09T04:35:42.199 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.199+0000 7f7738e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f77340b2600 0x7f77340b9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:42.199 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.199+0000 7f7738e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77340b40a0 0x7f77340ccba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:42.199 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.199+0000 7f7738e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f77340d33f0 0x7f77341544e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:42.199 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.199+0000 7f7738e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7734162c10 con 0x7f77340d33f0 2024-05-09T04:35:42.199 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.199+0000 7f7738e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f77340d06e0 con 0x7f77340b40a0 2024-05-09T04:35:42.200 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.199+0000 7f7738e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f7734162d90 con 0x7f77340b2600 2024-05-09T04:35:42.200 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.200+0000 7f7733e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f77340d33f0 0x7f77341544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:42.200 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.200+0000 7f7733e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f77340d33f0 0x7f77341544e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:42318/0 (socket says 172.21.15.145:42318) 2024-05-09T04:35:42.200 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.200+0000 7f7733e00640 1 -- 172.21.15.145:0/2629896796 learned_addr learned my addr 172.21.15.145:0/2629896796 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:42.200 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.200+0000 7f7732a00640 1 --2- 172.21.15.145:0/2629896796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77340b40a0 0x7f77340ccba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:42.200 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.200+0000 7f7733400640 1 --2- 172.21.15.145:0/2629896796 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f77340b2600 0x7f77340b9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:42.200 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.200+0000 7f7733e00640 1 -- 172.21.15.145:0/2629896796 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f77340b2600 msgr2=0x7f77340b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:42.200 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.200+0000 7f7733e00640 1 --2- 172.21.15.145:0/2629896796 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f77340b2600 0x7f77340b9ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:42.200 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.200+0000 7f7733e00640 1 -- 172.21.15.145:0/2629896796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77340b40a0 msgr2=0x7f77340ccba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:42.200 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.200+0000 7f7733e00640 1 --2- 172.21.15.145:0/2629896796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77340b40a0 0x7f77340ccba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:42.200 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.200+0000 7f7733e00640 1 -- 172.21.15.145:0/2629896796 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77340cd0e0 con 0x7f77340d33f0 2024-05-09T04:35:42.200 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.200+0000 7f7732a00640 1 --2- 172.21.15.145:0/2629896796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77340b40a0 0x7f77340ccba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:35:42.200 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.200+0000 7f7733e00640 1 --2- 172.21.15.145:0/2629896796 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f77340d33f0 0x7f77341544e0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f772c00c4d0 tx=0x7f772c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:42.200 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.201+0000 7f772b400640 1 -- 172.21.15.145:0/2629896796 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f772c015070 con 0x7f77340d33f0 2024-05-09T04:35:42.200 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.201+0000 7f772b400640 1 -- 172.21.15.145:0/2629896796 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f772c004220 con 0x7f77340d33f0 2024-05-09T04:35:42.201 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.201+0000 7f772b400640 1 -- 172.21.15.145:0/2629896796 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f772c003610 con 0x7f77340d33f0 2024-05-09T04:35:42.201 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.201+0000 7f7738e00640 1 -- 172.21.15.145:0/2629896796 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7734154a20 con 0x7f77340d33f0 2024-05-09T04:35:42.201 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.201+0000 7f7738e00640 1 -- 172.21.15.145:0/2629896796 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77340cd640 con 0x7f77340d33f0 2024-05-09T04:35:42.201 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.202+0000 7f7738e00640 1 -- 172.21.15.145:0/2629896796 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7734076930 con 0x7f77340d33f0 2024-05-09T04:35:42.201 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.203+0000 7f772b400640 1 -- 172.21.15.145:0/2629896796 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f772c0037b0 con 0x7f77340d33f0 2024-05-09T04:35:42.203 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.204+0000 7f772b400640 1 --2- 172.21.15.145:0/2629896796 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f7710076300 0x7f77100787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:42.203 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.205+0000 7f772b400640 1 -- 172.21.15.145:0/2629896796 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6258+0+0 (secure 0 0 0) 0x7f772c0975b0 con 0x7f77340d33f0 2024-05-09T04:35:42.203 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.205+0000 7f7733400640 1 --2- 172.21.15.145:0/2629896796 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f7710076300 0x7f77100787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:42.204 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.205+0000 7f7733400640 1 --2- 172.21.15.145:0/2629896796 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f7710076300 0x7f77100787c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7724033000 tx=0x7f7724009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:42.208 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.209+0000 7f772b400640 1 -- 172.21.15.145:0/2629896796 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f7734154a20 con 0x7f77340d33f0 2024-05-09T04:35:42.401 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:42.402+0000 7f7738e00640 1 -- 172.21.15.145:0/2629896796 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f77340d5fd0 con 0x7f77340d33f0 2024-05-09T04:35:43.402 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.400+0000 7f772b400640 1 -- 172.21.15.145:0/2629896796 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v53) ==== 181+0+0 (secure 0 0 0) 0x7f77340cd0e0 con 0x7f77340d33f0 2024-05-09T04:35:43.402 INFO:teuthology.orchestra.run.smithi145.stderr:pool 'cephfs_metadata' removed 2024-05-09T04:35:43.404 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.405+0000 7f7738e00640 1 -- 172.21.15.145:0/2629896796 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f7710076300 msgr2=0x7f77100787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:43.405 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.405+0000 7f7738e00640 1 --2- 172.21.15.145:0/2629896796 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f7710076300 0x7f77100787c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7724033000 tx=0x7f7724009000 comp rx=0 tx=0).stop 2024-05-09T04:35:43.405 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.405+0000 7f7738e00640 1 -- 172.21.15.145:0/2629896796 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f77340d33f0 msgr2=0x7f77341544e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:43.405 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.405+0000 7f7738e00640 1 --2- 172.21.15.145:0/2629896796 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f77340d33f0 0x7f77341544e0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f772c00c4d0 tx=0x7f772c00c990 comp rx=0 tx=0).stop 2024-05-09T04:35:43.405 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.406+0000 7f7738e00640 1 -- 172.21.15.145:0/2629896796 shutdown_connections 2024-05-09T04:35:43.405 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.406+0000 7f7738e00640 1 --2- 172.21.15.145:0/2629896796 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f7710076300 0x7f77100787c0 secure :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7724033000 tx=0x7f7724009000 comp rx=0 tx=0).stop 2024-05-09T04:35:43.405 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.406+0000 7f7738e00640 1 --2- 172.21.15.145:0/2629896796 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f77340d33f0 0x7f77341544e0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:43.405 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.406+0000 7f7738e00640 1 --2- 172.21.15.145:0/2629896796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77340b40a0 0x7f77340ccba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:43.405 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.406+0000 7f7738e00640 1 --2- 172.21.15.145:0/2629896796 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f77340b2600 0x7f77340b9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:43.405 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.406+0000 7f7738e00640 1 -- 172.21.15.145:0/2629896796 >> 172.21.15.145:0/2629896796 conn(0x7f773409e400 msgr2=0x7f77340b2a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:43.406 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.406+0000 7f7738e00640 1 -- 172.21.15.145:0/2629896796 shutdown_connections 2024-05-09T04:35:43.406 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.407+0000 7f7738e00640 1 -- 172.21.15.145:0/2629896796 wait complete. 2024-05-09T04:35:43.431 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-09T04:35:43.606 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.607+0000 7f7e99000640 1 Processor -- start 2024-05-09T04:35:43.606 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.608+0000 7f7e99000640 1 -- start start 2024-05-09T04:35:43.611 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.609+0000 7f7e99000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f7e940d33f0 0x7f7e940d5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:43.611 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.609+0000 7f7e99000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7e940b40a0 0x7f7e940d37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:43.611 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.609+0000 7f7e99000640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e940d0fd0 con 0x7f7e940d16f0 2024-05-09T04:35:43.611 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.609+0000 7f7e99000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f7e940b2170 con 0x7f7e940d33f0 2024-05-09T04:35:43.611 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.609+0000 7f7e99000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f7e940d3d10 con 0x7f7e940b40a0 2024-05-09T04:35:43.611 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.610+0000 7f7e93400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f7e940d33f0 0x7f7e940d5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:43.611 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.610+0000 7f7e93400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f7e940d33f0 0x7f7e940d5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.145:46268/0 (socket says 172.21.15.145:46268) 2024-05-09T04:35:43.611 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.610+0000 7f7e93400640 1 -- 172.21.15.145:0/3968401098 learned_addr learned my addr 172.21.15.145:0/3968401098 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:43.612 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.610+0000 7f7e92a00640 1 --2- 172.21.15.145:0/3968401098 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7e940b40a0 0x7f7e940d37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:43.612 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.611+0000 7f7e93400640 1 -- 172.21.15.145:0/3968401098 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7e940b40a0 msgr2=0x7f7e940d37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:43.612 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.611+0000 7f7e93400640 1 --2- 172.21.15.145:0/3968401098 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7e940b40a0 0x7f7e940d37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:43.612 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.611+0000 7f7e93400640 1 -- 172.21.15.145:0/3968401098 >> v1:172.21.15.192:6789/0 conn(0x7f7e940d16f0 legacy=0x7f7e940d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:43.612 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.611+0000 7f7e93400640 1 -- 172.21.15.145:0/3968401098 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e940d0a10 con 0x7f7e940d33f0 2024-05-09T04:35:43.612 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.612+0000 7f7e93400640 1 --2- 172.21.15.145:0/3968401098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e940d33f0 0x7f7e940d5990 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7e84009870 tx=0x7f7e8402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b56cc7ed1bc7768 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:43.612 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.612+0000 7f7e92000640 1 -- 172.21.15.145:0/3968401098 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e84037070 con 0x7f7e940d33f0 2024-05-09T04:35:43.612 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.612+0000 7f7e92000640 1 -- 172.21.15.145:0/3968401098 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e84032440 con 0x7f7e940d33f0 2024-05-09T04:35:43.612 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.612+0000 7f7e92000640 1 -- 172.21.15.145:0/3968401098 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e84032810 con 0x7f7e940d33f0 2024-05-09T04:35:43.614 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.613+0000 7f7e99000640 1 -- 172.21.15.145:0/3968401098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e940d33f0 msgr2=0x7f7e940d5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:43.614 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.613+0000 7f7e99000640 1 --2- 172.21.15.145:0/3968401098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e940d33f0 0x7f7e940d5990 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7e84009870 tx=0x7f7e8402a1c0 comp rx=0 tx=0).stop 2024-05-09T04:35:43.614 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.613+0000 7f7e99000640 1 -- 172.21.15.145:0/3968401098 shutdown_connections 2024-05-09T04:35:43.614 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.613+0000 7f7e99000640 1 --2- 172.21.15.145:0/3968401098 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7e940b40a0 0x7f7e940d37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:43.614 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.613+0000 7f7e99000640 1 --2- 172.21.15.145:0/3968401098 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e940d33f0 0x7f7e940d5990 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:43.614 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.613+0000 7f7e99000640 1 -- 172.21.15.145:0/3968401098 >> 172.21.15.145:0/3968401098 conn(0x7f7e9409e400 msgr2=0x7f7e940d50c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:43.614 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.613+0000 7f7e99000640 1 -- 172.21.15.145:0/3968401098 shutdown_connections 2024-05-09T04:35:43.614 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.613+0000 7f7e99000640 1 -- 172.21.15.145:0/3968401098 wait complete. 2024-05-09T04:35:43.614 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.614+0000 7f7e99000640 1 Processor -- start 2024-05-09T04:35:43.615 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.615+0000 7f7e99000640 1 -- start start 2024-05-09T04:35:43.617 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.615+0000 7f7e99000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7e940b40a0 0x7f7e941f4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:43.617 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.615+0000 7f7e99000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7e940d16f0 0x7f7e941f8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:43.617 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.615+0000 7f7e99000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e940d33f0 0x7f7e94151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:43.617 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.616+0000 7f7e99000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7e940d06e0 con 0x7f7e940d16f0 2024-05-09T04:35:43.618 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.616+0000 7f7e99000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7e940b1e00 con 0x7f7e940d33f0 2024-05-09T04:35:43.618 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.616+0000 7f7e99000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f7e94162c10 con 0x7f7e940b40a0 2024-05-09T04:35:43.618 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.616+0000 7f7e92a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7e940d16f0 0x7f7e941f8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:43.618 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.616+0000 7f7e92a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7e940d16f0 0x7f7e941f8ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:42332/0 (socket says 172.21.15.145:42332) 2024-05-09T04:35:43.618 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.616+0000 7f7e92a00640 1 -- 172.21.15.145:0/2118702352 learned_addr learned my addr 172.21.15.145:0/2118702352 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:43.618 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.616+0000 7f7e92a00640 1 -- 172.21.15.145:0/2118702352 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7e940b40a0 msgr2=0x7f7e941f4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:43.618 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.616+0000 7f7e92a00640 1 --2- 172.21.15.145:0/2118702352 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7e940b40a0 0x7f7e941f4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:43.618 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.616+0000 7f7e93e00640 1 --2- 172.21.15.145:0/2118702352 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e940d33f0 0x7f7e94151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:43.618 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.616+0000 7f7e92a00640 1 -- 172.21.15.145:0/2118702352 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e940d33f0 msgr2=0x7f7e94151fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:43.618 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.616+0000 7f7e92a00640 1 --2- 172.21.15.145:0/2118702352 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e940d33f0 0x7f7e94151fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:43.619 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.616+0000 7f7e92a00640 1 -- 172.21.15.145:0/2118702352 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e941f55e0 con 0x7f7e940d16f0 2024-05-09T04:35:43.619 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.616+0000 7f7e93e00640 1 --2- 172.21.15.145:0/2118702352 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e940d33f0 0x7f7e94151fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:35:43.619 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.617+0000 7f7e92a00640 1 --2- 172.21.15.145:0/2118702352 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7e940d16f0 0x7f7e941f8ee0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c00efc0 tx=0x7f7e7c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:43.619 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.617+0000 7f7e8b400640 1 -- 172.21.15.145:0/2118702352 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c018020 con 0x7f7e940d16f0 2024-05-09T04:35:43.619 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.617+0000 7f7e8b400640 1 -- 172.21.15.145:0/2118702352 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e7c014480 con 0x7f7e940d16f0 2024-05-09T04:35:43.619 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.617+0000 7f7e99000640 1 -- 172.21.15.145:0/2118702352 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e941f58c0 con 0x7f7e940d16f0 2024-05-09T04:35:43.619 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.617+0000 7f7e8b400640 1 -- 172.21.15.145:0/2118702352 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c014850 con 0x7f7e940d16f0 2024-05-09T04:35:43.619 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.617+0000 7f7e99000640 1 -- 172.21.15.145:0/2118702352 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e94151ab0 con 0x7f7e940d16f0 2024-05-09T04:35:43.619 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.618+0000 7f7e99000640 1 -- 172.21.15.145:0/2118702352 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e941f5de0 con 0x7f7e940d16f0 2024-05-09T04:35:43.620 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.619+0000 7f7e8b400640 1 -- 172.21.15.145:0/2118702352 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7e7c005d10 con 0x7f7e940d16f0 2024-05-09T04:35:43.621 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.621+0000 7f7e8b400640 1 --2- 172.21.15.145:0/2118702352 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f7e700762d0 0x7f7e70078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:43.621 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.621+0000 7f7e8b400640 1 -- 172.21.15.145:0/2118702352 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5820+0+0 (secure 0 0 0) 0x7f7e7c098190 con 0x7f7e940d16f0 2024-05-09T04:35:43.621 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.621+0000 7f7e93400640 1 --2- 172.21.15.145:0/2118702352 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f7e700762d0 0x7f7e70078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:43.621 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.622+0000 7f7e93400640 1 --2- 172.21.15.145:0/2118702352 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f7e700762d0 0x7f7e70078790 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7e84034000 tx=0x7f7e8402c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:43.624 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.625+0000 7f7e8b400640 1 -- 172.21.15.145:0/2118702352 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f7e941f5de0 con 0x7f7e940d16f0 2024-05-09T04:35:43.813 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:43.814+0000 7f7e99000640 1 -- 172.21.15.145:0/2118702352 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7e941518c0 con 0x7f7e940d16f0 2024-05-09T04:35:44.403 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.404+0000 7f7e8b400640 1 -- 172.21.15.145:0/2118702352 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v54) ==== 169+0+0 (secure 0 0 0) 0x7f7e7c022070 con 0x7f7e940d16f0 2024-05-09T04:35:44.403 INFO:teuthology.orchestra.run.smithi145.stderr:pool 'cephfs_data' removed 2024-05-09T04:35:44.407 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.408+0000 7f7e99000640 1 -- 172.21.15.145:0/2118702352 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f7e700762d0 msgr2=0x7f7e70078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:44.407 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.408+0000 7f7e99000640 1 --2- 172.21.15.145:0/2118702352 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f7e700762d0 0x7f7e70078790 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7e84034000 tx=0x7f7e8402c000 comp rx=0 tx=0).stop 2024-05-09T04:35:44.407 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.408+0000 7f7e99000640 1 -- 172.21.15.145:0/2118702352 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7e940d16f0 msgr2=0x7f7e941f8ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:44.407 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.408+0000 7f7e99000640 1 --2- 172.21.15.145:0/2118702352 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7e940d16f0 0x7f7e941f8ee0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c00efc0 tx=0x7f7e7c00c490 comp rx=0 tx=0).stop 2024-05-09T04:35:44.407 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.408+0000 7f7e99000640 1 -- 172.21.15.145:0/2118702352 shutdown_connections 2024-05-09T04:35:44.407 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.408+0000 7f7e99000640 1 --2- 172.21.15.145:0/2118702352 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f7e700762d0 0x7f7e70078790 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:44.407 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.408+0000 7f7e99000640 1 --2- 172.21.15.145:0/2118702352 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e940d33f0 0x7f7e94151fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:44.407 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.408+0000 7f7e99000640 1 --2- 172.21.15.145:0/2118702352 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7e940d16f0 0x7f7e941f8ee0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:44.407 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.408+0000 7f7e99000640 1 --2- 172.21.15.145:0/2118702352 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7e940b40a0 0x7f7e941f4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:44.407 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.408+0000 7f7e99000640 1 -- 172.21.15.145:0/2118702352 >> 172.21.15.145:0/2118702352 conn(0x7f7e9409e400 msgr2=0x7f7e940b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:44.408 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.408+0000 7f7e99000640 1 -- 172.21.15.145:0/2118702352 shutdown_connections 2024-05-09T04:35:44.408 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.409+0000 7f7e99000640 1 -- 172.21.15.145:0/2118702352 wait complete. 2024-05-09T04:35:44.435 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_extra1 cephfs_data_extra1 --yes-i-really-really-mean-it 2024-05-09T04:35:44.635 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.636+0000 7f5bbc200640 1 Processor -- start 2024-05-09T04:35:44.635 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.636+0000 7f5bbc200640 1 -- start start 2024-05-09T04:35:44.636 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.637+0000 7f5bbc200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5bb415f2f0 0x7f5bb40d08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:44.636 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.637+0000 7f5bbc200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f5bb40d0e10 0x7f5bb40d1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:44.636 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.637+0000 7f5bbc200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bb40d30d0 con 0x7f5bb40d1fa0 2024-05-09T04:35:44.636 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.637+0000 7f5bbc200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f5bb40d2410 con 0x7f5bb40d0e10 2024-05-09T04:35:44.636 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.637+0000 7f5bbc200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f5bb40d33b0 con 0x7f5bb415f2f0 2024-05-09T04:35:44.637 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.638+0000 7f5bb8e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f5bb40d0e10 0x7f5bb40d1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:44.637 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.638+0000 7f5bb8e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f5bb40d0e10 0x7f5bb40d1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.145:46300/0 (socket says 172.21.15.145:46300) 2024-05-09T04:35:44.637 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.638+0000 7f5bb9800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5bb415f2f0 0x7f5bb40d08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:44.637 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.638+0000 7f5bb8e00640 1 -- 172.21.15.145:0/3644643478 learned_addr learned my addr 172.21.15.145:0/3644643478 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:44.638 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.639+0000 7f5bb8e00640 1 -- 172.21.15.145:0/3644643478 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5bb415f2f0 msgr2=0x7f5bb40d08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:44.638 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.639+0000 7f5bb8e00640 1 --2- 172.21.15.145:0/3644643478 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5bb415f2f0 0x7f5bb40d08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:44.638 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.639+0000 7f5bb8e00640 1 -- 172.21.15.145:0/3644643478 >> v1:172.21.15.192:6789/0 conn(0x7f5bb40d1fa0 legacy=0x7f5bb40ba290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:44.638 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.639+0000 7f5bb8e00640 1 -- 172.21.15.145:0/3644643478 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bb40d3690 con 0x7f5bb40d0e10 2024-05-09T04:35:44.639 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.640+0000 7f5bb8e00640 1 --2- 172.21.15.145:0/3644643478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5bb40d0e10 0x7f5bb40d1a60 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5ba0009870 tx=0x7f5ba002a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c15287da9d636b73 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:44.639 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.641+0000 7f5bb3e00640 1 -- 172.21.15.145:0/3644643478 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba0037070 con 0x7f5bb40d0e10 2024-05-09T04:35:44.639 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.641+0000 7f5bb3e00640 1 -- 172.21.15.145:0/3644643478 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ba0032440 con 0x7f5bb40d0e10 2024-05-09T04:35:44.639 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.641+0000 7f5bb3e00640 1 -- 172.21.15.145:0/3644643478 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba0032810 con 0x7f5bb40d0e10 2024-05-09T04:35:44.640 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.641+0000 7f5bbc200640 1 -- 172.21.15.145:0/3644643478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5bb40d0e10 msgr2=0x7f5bb40d1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:44.640 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.641+0000 7f5bbc200640 1 --2- 172.21.15.145:0/3644643478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5bb40d0e10 0x7f5bb40d1a60 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5ba0009870 tx=0x7f5ba002a270 comp rx=0 tx=0).stop 2024-05-09T04:35:44.640 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.641+0000 7f5bbc200640 1 -- 172.21.15.145:0/3644643478 shutdown_connections 2024-05-09T04:35:44.640 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.641+0000 7f5bbc200640 1 --2- 172.21.15.145:0/3644643478 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5bb40d0e10 0x7f5bb40d1a60 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:44.640 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.641+0000 7f5bbc200640 1 --2- 172.21.15.145:0/3644643478 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5bb415f2f0 0x7f5bb40d08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:44.640 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.641+0000 7f5bbc200640 1 -- 172.21.15.145:0/3644643478 >> 172.21.15.145:0/3644643478 conn(0x7f5bb409e4d0 msgr2=0x7f5bb40d5190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:44.640 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.642+0000 7f5bbc200640 1 -- 172.21.15.145:0/3644643478 shutdown_connections 2024-05-09T04:35:44.641 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.642+0000 7f5bbc200640 1 -- 172.21.15.145:0/3644643478 wait complete. 2024-05-09T04:35:44.642 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.643+0000 7f5bbc200640 1 Processor -- start 2024-05-09T04:35:44.642 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.643+0000 7f5bbc200640 1 -- start start 2024-05-09T04:35:44.644 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.644+0000 7f5bbc200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5bb40d0e10 0x7f5bb41540b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:44.644 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.644+0000 7f5bbc200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5bb40d1fa0 0x7f5bb41594e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:44.644 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.644+0000 7f5bbc200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5bb415f2f0 0x7f5bb4160720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:44.644 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.644+0000 7f5bbc200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb40d4150 con 0x7f5bb40d0e10 2024-05-09T04:35:44.644 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.644+0000 7f5bbc200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb40d3c50 con 0x7f5bb40d1fa0 2024-05-09T04:35:44.645 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.644+0000 7f5bbc200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f5bb40d2d60 con 0x7f5bb415f2f0 2024-05-09T04:35:44.645 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.645+0000 7f5bb9800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5bb40d0e10 0x7f5bb41540b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:44.645 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.645+0000 7f5bb9800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5bb40d0e10 0x7f5bb41540b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:42348/0 (socket says 172.21.15.145:42348) 2024-05-09T04:35:44.645 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.645+0000 7f5bb9800640 1 -- 172.21.15.145:0/1715125990 learned_addr learned my addr 172.21.15.145:0/1715125990 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:44.645 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.645+0000 7f5bb8e00640 1 --2- 172.21.15.145:0/1715125990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5bb40d1fa0 0x7f5bb41594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:44.645 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.645+0000 7f5bba200640 1 --2- 172.21.15.145:0/1715125990 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5bb415f2f0 0x7f5bb4160720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:44.645 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.645+0000 7f5bb9800640 1 -- 172.21.15.145:0/1715125990 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5bb415f2f0 msgr2=0x7f5bb4160720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:44.645 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.645+0000 7f5bb9800640 1 --2- 172.21.15.145:0/1715125990 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5bb415f2f0 0x7f5bb4160720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:44.645 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.645+0000 7f5bb9800640 1 -- 172.21.15.145:0/1715125990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5bb40d1fa0 msgr2=0x7f5bb41594e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:44.645 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.645+0000 7f5bb9800640 1 --2- 172.21.15.145:0/1715125990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5bb40d1fa0 0x7f5bb41594e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:44.645 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.645+0000 7f5bb9800640 1 -- 172.21.15.145:0/1715125990 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bb415b620 con 0x7f5bb40d0e10 2024-05-09T04:35:44.645 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.646+0000 7f5bb9800640 1 --2- 172.21.15.145:0/1715125990 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5bb40d0e10 0x7f5bb41540b0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f5bac009d10 tx=0x7f5bac00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:44.646 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.646+0000 7f5bb1600640 1 -- 172.21.15.145:0/1715125990 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bac018020 con 0x7f5bb40d0e10 2024-05-09T04:35:44.646 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.646+0000 7f5bb1600640 1 -- 172.21.15.145:0/1715125990 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bac014480 con 0x7f5bb40d0e10 2024-05-09T04:35:44.646 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.646+0000 7f5bb1600640 1 -- 172.21.15.145:0/1715125990 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bac014830 con 0x7f5bb40d0e10 2024-05-09T04:35:44.646 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.646+0000 7f5bbc200640 1 -- 172.21.15.145:0/1715125990 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bb415b900 con 0x7f5bb40d0e10 2024-05-09T04:35:44.647 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.646+0000 7f5bbc200640 1 -- 172.21.15.145:0/1715125990 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bb41545f0 con 0x7f5bb40d0e10 2024-05-09T04:35:44.647 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.647+0000 7f5bbc200640 1 -- 172.21.15.145:0/1715125990 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bb406a310 con 0x7f5bb40d0e10 2024-05-09T04:35:44.647 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.648+0000 7f5bb1600640 1 -- 172.21.15.145:0/1715125990 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f5bac005d10 con 0x7f5bb40d0e10 2024-05-09T04:35:44.648 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.649+0000 7f5bb1600640 1 --2- 172.21.15.145:0/1715125990 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f5b940762d0 0x7f5b94078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:44.648 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.649+0000 7f5bb1600640 1 -- 172.21.15.145:0/1715125990 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5348+0+0 (secure 0 0 0) 0x7f5bac097f80 con 0x7f5bb40d0e10 2024-05-09T04:35:44.648 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.650+0000 7f5bb8e00640 1 --2- 172.21.15.145:0/1715125990 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f5b940762d0 0x7f5b94078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:44.649 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.650+0000 7f5bb8e00640 1 --2- 172.21.15.145:0/1715125990 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f5b940762d0 0x7f5b94078790 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5ba0034000 tx=0x7f5ba002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:44.652 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.654+0000 7f5bb1600640 1 -- 172.21.15.145:0/1715125990 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5bac022070 con 0x7f5bb40d0e10 2024-05-09T04:35:44.862 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:44.863+0000 7f5bbc200640 1 -- 172.21.15.145:0/1715125990 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_extra1", "pool2": "cephfs_data_extra1", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5bb40d2810 con 0x7f5bb40d0e10 2024-05-09T04:35:45.425 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.426+0000 7f5bb1600640 1 -- 172.21.15.145:0/1715125990 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_extra1", "pool2": "cephfs_data_extra1", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_extra1' removed v55) ==== 190+0+0 (secure 0 0 0) 0x7f5bac010040 con 0x7f5bb40d0e10 2024-05-09T04:35:45.425 INFO:teuthology.orchestra.run.smithi145.stderr:pool 'cephfs_data_extra1' removed 2024-05-09T04:35:45.429 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.430+0000 7f5bbc200640 1 -- 172.21.15.145:0/1715125990 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f5b940762d0 msgr2=0x7f5b94078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:45.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.430+0000 7f5bbc200640 1 --2- 172.21.15.145:0/1715125990 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f5b940762d0 0x7f5b94078790 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5ba0034000 tx=0x7f5ba002e000 comp rx=0 tx=0).stop 2024-05-09T04:35:45.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.430+0000 7f5bbc200640 1 -- 172.21.15.145:0/1715125990 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5bb40d0e10 msgr2=0x7f5bb41540b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:45.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.430+0000 7f5bbc200640 1 --2- 172.21.15.145:0/1715125990 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5bb40d0e10 0x7f5bb41540b0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f5bac009d10 tx=0x7f5bac00c490 comp rx=0 tx=0).stop 2024-05-09T04:35:45.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.430+0000 7f5bbc200640 1 -- 172.21.15.145:0/1715125990 shutdown_connections 2024-05-09T04:35:45.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.430+0000 7f5bbc200640 1 --2- 172.21.15.145:0/1715125990 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f5b940762d0 0x7f5b94078790 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:45.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.430+0000 7f5bbc200640 1 --2- 172.21.15.145:0/1715125990 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5bb415f2f0 0x7f5bb4160720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:45.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.430+0000 7f5bbc200640 1 --2- 172.21.15.145:0/1715125990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5bb40d1fa0 0x7f5bb41594e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:45.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.430+0000 7f5bbc200640 1 --2- 172.21.15.145:0/1715125990 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5bb40d0e10 0x7f5bb41540b0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:45.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.430+0000 7f5bbc200640 1 -- 172.21.15.145:0/1715125990 >> 172.21.15.145:0/1715125990 conn(0x7f5bb409e4d0 msgr2=0x7f5bb40d1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:45.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.431+0000 7f5bbc200640 1 -- 172.21.15.145:0/1715125990 shutdown_connections 2024-05-09T04:35:45.430 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.431+0000 7f5bbc200640 1 -- 172.21.15.145:0/1715125990 wait complete. 2024-05-09T04:35:45.450 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-05-09T04:35:45.649 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.650+0000 7f99fd600640 1 Processor -- start 2024-05-09T04:35:45.649 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.651+0000 7f99fd600640 1 -- start start 2024-05-09T04:35:45.651 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.652+0000 7f99fd600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99f806fce0 0x7f99f8166fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:45.651 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.652+0000 7f99fd600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99f815ffa0 con 0x7f99f8144c00 2024-05-09T04:35:45.651 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.652+0000 7f99fd600640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99f8160280 con 0x7f99f80ce640 2024-05-09T04:35:45.651 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.652+0000 7f99fd600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f99f8160560 con 0x7f99f806fce0 2024-05-09T04:35:45.651 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.652+0000 7f99f6a00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f99f8144c00 0x7f99f8145440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:52048/0 (socket says 172.21.15.145:52048) 2024-05-09T04:35:45.651 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.652+0000 7f99f6a00640 1 -- 172.21.15.145:0/1164690317 learned_addr learned my addr 172.21.15.145:0/1164690317 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:35:45.653 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.653+0000 7f99f6000640 1 -- 172.21.15.145:0/1164690317 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3482420518 0 0) 0x7f99f815ffa0 con 0x7f99f8144c00 2024-05-09T04:35:45.653 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.653+0000 7f99f7400640 1 --2- 172.21.15.145:0/1164690317 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99f806fce0 0x7f99f8166fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:45.653 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.653+0000 7f99f6000640 1 -- 172.21.15.145:0/1164690317 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f99dc005730 con 0x7f99f8144c00 2024-05-09T04:35:45.653 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.653+0000 7f99f6000640 1 -- 172.21.15.145:0/1164690317 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 586792307 0 0) 0x7f99dc005730 con 0x7f99f8144c00 2024-05-09T04:35:45.653 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.653+0000 7f99f6000640 1 -- 172.21.15.145:0/1164690317 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99f806fce0 msgr2=0x7f99f8166fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:45.653 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.653+0000 7f99f6000640 1 --2- 172.21.15.145:0/1164690317 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99f806fce0 0x7f99f8166fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:45.654 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.653+0000 7f99f6000640 1 -- 172.21.15.145:0/1164690317 >> v1:172.21.15.192:6789/0 conn(0x7f99f80ce640 legacy=0x7f99f806f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:45.654 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.653+0000 7f99f6000640 1 -- 172.21.15.145:0/1164690317 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99f8160840 con 0x7f99f8144c00 2024-05-09T04:35:45.654 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.653+0000 7f99f6000640 1 -- 172.21.15.145:0/1164690317 <== mon.0 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3474413261 0 0) 0x7f99e80016f0 con 0x7f99f8144c00 2024-05-09T04:35:45.654 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.653+0000 7f99f6000640 1 -- 172.21.15.145:0/1164690317 >> v1:172.21.15.145:6789/0 conn(0x7f99f8144c00 legacy=0x7f99f8145440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:45.654 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.653+0000 7f99f6000640 1 --2- 172.21.15.145:0/1164690317 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99dc005c90 0x7f99dc0060b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:45.654 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.653+0000 7f99f6000640 1 --2- 172.21.15.145:0/1164690317 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99f806fce0 0x7f99dc006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:45.654 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.653+0000 7f99f6000640 1 --2- 172.21.15.145:0/1164690317 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99dc007070 0x7f99dc00dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:45.654 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.653+0000 7f99f6000640 1 -- 172.21.15.145:0/1164690317 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f99f8160560 con 0x7f99dc005c90 2024-05-09T04:35:45.654 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.653+0000 7f99f6000640 1 -- 172.21.15.145:0/1164690317 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f99dc007730 con 0x7f99dc007070 2024-05-09T04:35:45.654 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.653+0000 7f99f6000640 1 -- 172.21.15.145:0/1164690317 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f99dc00e110 con 0x7f99f806fce0 2024-05-09T04:35:45.654 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.654+0000 7f99f6a00640 1 --2- 172.21.15.145:0/1164690317 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99dc005c90 0x7f99dc0060b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:45.654 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.654+0000 7f99f7400640 1 --2- 172.21.15.145:0/1164690317 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99dc007070 0x7f99dc00dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:45.654 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.654+0000 7f99f6a00640 1 -- 172.21.15.145:0/1164690317 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99f806fce0 msgr2=0x7f99dc006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:45.655 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.654+0000 7f99ee000640 1 --2- 172.21.15.145:0/1164690317 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99f806fce0 0x7f99dc006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:45.655 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.654+0000 7f99f6a00640 1 --2- 172.21.15.145:0/1164690317 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99f806fce0 0x7f99dc006a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:45.655 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.654+0000 7f99f6a00640 1 -- 172.21.15.145:0/1164690317 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99dc007070 msgr2=0x7f99dc00dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:45.655 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.654+0000 7f99f6a00640 1 --2- 172.21.15.145:0/1164690317 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99dc007070 0x7f99dc00dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:45.655 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.654+0000 7f99f6a00640 1 -- 172.21.15.145:0/1164690317 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99dc005730 con 0x7f99dc005c90 2024-05-09T04:35:45.655 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.654+0000 7f99f7400640 1 --2- 172.21.15.145:0/1164690317 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99dc007070 0x7f99dc00dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:35:45.656 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.655+0000 7f99f6a00640 1 --2- 172.21.15.145:0/1164690317 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99dc005c90 0x7f99dc0060b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f99e800c160 tx=0x7f99dc007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ee1c20ef3912d42 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:45.656 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.655+0000 7f99f6000640 1 -- 172.21.15.145:0/1164690317 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e8007ca0 con 0x7f99dc005c90 2024-05-09T04:35:45.656 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.655+0000 7f99f6000640 1 -- 172.21.15.145:0/1164690317 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99e8005ad0 con 0x7f99dc005c90 2024-05-09T04:35:45.656 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.655+0000 7f99f6000640 1 -- 172.21.15.145:0/1164690317 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e8007070 con 0x7f99dc005c90 2024-05-09T04:35:45.656 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.656+0000 7f99fd600640 1 -- 172.21.15.145:0/1164690317 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99dc005c90 msgr2=0x7f99dc0060b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:45.656 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.656+0000 7f99fd600640 1 --2- 172.21.15.145:0/1164690317 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99dc005c90 0x7f99dc0060b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f99e800c160 tx=0x7f99dc007a40 comp rx=0 tx=0).stop 2024-05-09T04:35:45.656 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.656+0000 7f99f7400640 1 -- 172.21.15.145:0/1164690317 reap_dead start 2024-05-09T04:35:45.657 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.656+0000 7f99fd600640 1 -- 172.21.15.145:0/1164690317 shutdown_connections 2024-05-09T04:35:45.657 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.656+0000 7f99fd600640 1 -- 172.21.15.145:0/1164690317 >> 172.21.15.145:0/1164690317 conn(0x7f99f806cab0 msgr2=0x7f99f80d0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:45.657 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.656+0000 7f99fd600640 1 -- 172.21.15.145:0/1164690317 shutdown_connections 2024-05-09T04:35:45.657 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.656+0000 7f99fd600640 1 -- 172.21.15.145:0/1164690317 wait complete. 2024-05-09T04:35:45.658 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.658+0000 7f99fd600640 1 Processor -- start 2024-05-09T04:35:45.659 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.658+0000 7f99fd600640 1 -- start start 2024-05-09T04:35:45.659 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.659+0000 7f99fd600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99dc005c90 0x7f99f815f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:45.659 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.659+0000 7f99fd600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99f815d9d0 0x7f99f80ccf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:45.659 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.659+0000 7f99fd600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99f815b900 0x7f99f81f1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:45.659 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.659+0000 7f99fd600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f99f8145b50 con 0x7f99f815b900 2024-05-09T04:35:45.659 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.659+0000 7f99fd600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f99f815fc30 con 0x7f99f815d9d0 2024-05-09T04:35:45.659 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.659+0000 7f99fd600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f99f8145cd0 con 0x7f99dc005c90 2024-05-09T04:35:45.659 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.659+0000 7f99f7400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99f815b900 0x7f99f81f1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:45.659 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.659+0000 7f99f7400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99f815b900 0x7f99f81f1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:42362/0 (socket says 172.21.15.145:42362) 2024-05-09T04:35:45.659 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.659+0000 7f99f7400640 1 -- 172.21.15.145:0/1504502249 learned_addr learned my addr 172.21.15.145:0/1504502249 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:45.659 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.659+0000 7f99ee000640 1 --2- 172.21.15.145:0/1504502249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99f815d9d0 0x7f99f80ccf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:45.659 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.660+0000 7f99f6a00640 1 --2- 172.21.15.145:0/1504502249 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99dc005c90 0x7f99f815f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:45.659 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.660+0000 7f99f7400640 1 -- 172.21.15.145:0/1504502249 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99dc005c90 msgr2=0x7f99f815f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:45.659 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.660+0000 7f99f7400640 1 --2- 172.21.15.145:0/1504502249 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99dc005c90 0x7f99f815f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:45.659 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.660+0000 7f99f7400640 1 -- 172.21.15.145:0/1504502249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99f815d9d0 msgr2=0x7f99f80ccf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:45.660 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.660+0000 7f99f7400640 1 --2- 172.21.15.145:0/1504502249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99f815d9d0 0x7f99f80ccf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:45.660 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.660+0000 7f99f7400640 1 -- 172.21.15.145:0/1504502249 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99f80cd4c0 con 0x7f99f815b900 2024-05-09T04:35:45.660 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.660+0000 7f99f7400640 1 --2- 172.21.15.145:0/1504502249 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99f815b900 0x7f99f81f1630 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f99f80b5160 tx=0x7f99f0009d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:45.660 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.660+0000 7f99ef400640 1 -- 172.21.15.145:0/1504502249 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f0019070 con 0x7f99f815b900 2024-05-09T04:35:45.661 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.660+0000 7f99ef400640 1 -- 172.21.15.145:0/1504502249 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99f0007d40 con 0x7f99f815b900 2024-05-09T04:35:45.661 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.660+0000 7f99ef400640 1 -- 172.21.15.145:0/1504502249 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f0014640 con 0x7f99f815b900 2024-05-09T04:35:45.661 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.660+0000 7f99fd600640 1 -- 172.21.15.145:0/1504502249 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99f815e060 con 0x7f99f815b900 2024-05-09T04:35:45.661 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.661+0000 7f99fd600640 1 -- 172.21.15.145:0/1504502249 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99f815e620 con 0x7f99f815b900 2024-05-09T04:35:45.661 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.662+0000 7f99fd600640 1 -- 172.21.15.145:0/1504502249 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99f806a310 con 0x7f99f815b900 2024-05-09T04:35:45.661 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.663+0000 7f99ef400640 1 -- 172.21.15.145:0/1504502249 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f99f001c070 con 0x7f99f815b900 2024-05-09T04:35:45.663 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.664+0000 7f99ef400640 1 --2- 172.21.15.145:0/1504502249 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f99d40762d0 0x7f99d4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:45.663 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.664+0000 7f99ef400640 1 -- 172.21.15.145:0/1504502249 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7f99f0096d70 con 0x7f99f815b900 2024-05-09T04:35:45.663 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.664+0000 7f99f6a00640 1 --2- 172.21.15.145:0/1504502249 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f99d40762d0 0x7f99d4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:45.663 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.665+0000 7f99f6a00640 1 --2- 172.21.15.145:0/1504502249 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f99d40762d0 0x7f99d4078790 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f99e8032000 tx=0x7f99e8009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:45.667 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.668+0000 7f99ef400640 1 -- 172.21.15.145:0/1504502249 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f99f00609c0 con 0x7f99f815b900 2024-05-09T04:35:45.882 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.883+0000 7f99fd600640 1 -- 172.21.15.145:0/1504502249 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f99f80b5b10 con 0x7f99f815b900 2024-05-09T04:35:45.889 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.890+0000 7f99ef400640 1 -- 172.21.15.145:0/1504502249 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v16) ==== 104+0+0 (secure 0 0 0) 0x7f99f00602c0 con 0x7f99f815b900 2024-05-09T04:35:45.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.893+0000 7f99fd600640 1 -- 172.21.15.145:0/1504502249 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f99d40762d0 msgr2=0x7f99d4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:45.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.893+0000 7f99fd600640 1 --2- 172.21.15.145:0/1504502249 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f99d40762d0 0x7f99d4078790 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f99e8032000 tx=0x7f99e8009030 comp rx=0 tx=0).stop 2024-05-09T04:35:45.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.893+0000 7f99fd600640 1 -- 172.21.15.145:0/1504502249 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99f815b900 msgr2=0x7f99f81f1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:45.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.893+0000 7f99fd600640 1 --2- 172.21.15.145:0/1504502249 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99f815b900 0x7f99f81f1630 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f99f80b5160 tx=0x7f99f0009d30 comp rx=0 tx=0).stop 2024-05-09T04:35:45.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.893+0000 7f99fd600640 1 -- 172.21.15.145:0/1504502249 shutdown_connections 2024-05-09T04:35:45.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.893+0000 7f99fd600640 1 --2- 172.21.15.145:0/1504502249 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f99d40762d0 0x7f99d4078790 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:45.893 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.893+0000 7f99fd600640 1 --2- 172.21.15.145:0/1504502249 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f99f815b900 0x7f99f81f1630 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:45.893 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.893+0000 7f99fd600640 1 --2- 172.21.15.145:0/1504502249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99f815d9d0 0x7f99f80ccf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:45.893 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.893+0000 7f99fd600640 1 --2- 172.21.15.145:0/1504502249 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99dc005c90 0x7f99f815f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:45.893 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.893+0000 7f99fd600640 1 -- 172.21.15.145:0/1504502249 >> 172.21.15.145:0/1504502249 conn(0x7f99f806cab0 msgr2=0x7f99f8142a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:45.893 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.893+0000 7f99fd600640 1 -- 172.21.15.145:0/1504502249 shutdown_connections 2024-05-09T04:35:45.893 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:45.894+0000 7f99fd600640 1 -- 172.21.15.145:0/1504502249 wait complete. 2024-05-09T04:35:45.911 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_backtrace.TestBacktrace.test_backtrace_flush_on_deleted_data_pool' 2024-05-09T04:35:46.110 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.111+0000 7ff1cbe00640 1 Processor -- start 2024-05-09T04:35:46.111 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.112+0000 7ff1cbe00640 1 -- start start 2024-05-09T04:35:46.112 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.113+0000 7ff1cbe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff1cc0d3180 0x7ff1cc0d5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:46.112 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.113+0000 7ff1cbe00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff1cc0b4760 0x7ff1cc0b4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:46.112 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.113+0000 7ff1cbe00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1cc0d3e70 con 0x7ff1cc0d22f0 2024-05-09T04:35:46.112 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.113+0000 7ff1cbe00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7ff1cc0d4150 con 0x7ff1cc0b4760 2024-05-09T04:35:46.112 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.113+0000 7ff1cbe00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff1cc0b6200 con 0x7ff1cc0d3180 2024-05-09T04:35:46.112 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.113+0000 7ff1cb400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff1cc0b4760 0x7ff1cc0b4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:46.112 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.113+0000 7ff1cb400640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ff1cc0b4760 0x7ff1cc0b4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:42366/0 (socket says 172.21.15.145:42366) 2024-05-09T04:35:46.113 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.113+0000 7ff1cb400640 1 -- 172.21.15.145:0/2434335463 learned_addr learned my addr 172.21.15.145:0/2434335463 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:46.113 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.113+0000 7ff1cb400640 1 -- 172.21.15.145:0/2434335463 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff1cc0d3180 msgr2=0x7ff1cc0d5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:46.113 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.113+0000 7ff1cb400640 1 --2- 172.21.15.145:0/2434335463 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff1cc0d3180 0x7ff1cc0d5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:46.113 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.114+0000 7ff1cb400640 1 -- 172.21.15.145:0/2434335463 >> v1:172.21.15.192:6789/0 conn(0x7ff1cc0d22f0 legacy=0x7ff1cc0d3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:46.113 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.114+0000 7ff1cb400640 1 -- 172.21.15.145:0/2434335463 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1cc0b6450 con 0x7ff1cc0b4760 2024-05-09T04:35:46.113 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.115+0000 7ff1cb400640 1 --2- 172.21.15.145:0/2434335463 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff1cc0b4760 0x7ff1cc0b4bd0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7ff1c4009870 tx=0x7ff1c402a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2415c64b12c45afe server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:46.114 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.115+0000 7ff1c9600640 1 -- 172.21.15.145:0/2434335463 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c4037070 con 0x7ff1cc0b4760 2024-05-09T04:35:46.114 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.115+0000 7ff1c9600640 1 -- 172.21.15.145:0/2434335463 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1c4032440 con 0x7ff1cc0b4760 2024-05-09T04:35:46.114 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.115+0000 7ff1c9600640 1 -- 172.21.15.145:0/2434335463 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c4032810 con 0x7ff1cc0b4760 2024-05-09T04:35:46.115 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.116+0000 7ff1cbe00640 1 -- 172.21.15.145:0/2434335463 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff1cc0b4760 msgr2=0x7ff1cc0b4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:46.115 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.116+0000 7ff1cbe00640 1 --2- 172.21.15.145:0/2434335463 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff1cc0b4760 0x7ff1cc0b4bd0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7ff1c4009870 tx=0x7ff1c402a130 comp rx=0 tx=0).stop 2024-05-09T04:35:46.115 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.116+0000 7ff1cbe00640 1 -- 172.21.15.145:0/2434335463 shutdown_connections 2024-05-09T04:35:46.115 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.116+0000 7ff1cbe00640 1 --2- 172.21.15.145:0/2434335463 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff1cc0b4760 0x7ff1cc0b4bd0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:46.115 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.116+0000 7ff1cbe00640 1 --2- 172.21.15.145:0/2434335463 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff1cc0d3180 0x7ff1cc0d5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:46.115 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.116+0000 7ff1cbe00640 1 -- 172.21.15.145:0/2434335463 >> 172.21.15.145:0/2434335463 conn(0x7ff1cc0d4fb0 msgr2=0x7ff1cc0d53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:46.115 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.116+0000 7ff1cbe00640 1 -- 172.21.15.145:0/2434335463 shutdown_connections 2024-05-09T04:35:46.115 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.116+0000 7ff1cbe00640 1 -- 172.21.15.145:0/2434335463 wait complete. 2024-05-09T04:35:46.116 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.118+0000 7ff1cbe00640 1 Processor -- start 2024-05-09T04:35:46.116 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.118+0000 7ff1cbe00640 1 -- start start 2024-05-09T04:35:46.119 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.119+0000 7ff1cbe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff1cc0b4760 0x7ff1cc1582e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:46.119 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.119+0000 7ff1cbe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1cc0d22f0 0x7ff1cc15f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:46.119 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.119+0000 7ff1cbe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff1cc0d3180 0x7ff1cc15cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:46.119 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.119+0000 7ff1cbe00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ff1cc0d2750 con 0x7ff1cc0b4760 2024-05-09T04:35:46.119 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.119+0000 7ff1cbe00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ff1cc0d1e60 con 0x7ff1cc0d22f0 2024-05-09T04:35:46.119 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.119+0000 7ff1cbe00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ff1cc0b3fa0 con 0x7ff1cc0d3180 2024-05-09T04:35:46.119 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.119+0000 7ff1caa00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff1cc0b4760 0x7ff1cc1582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:46.119 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.119+0000 7ff1caa00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff1cc0b4760 0x7ff1cc1582e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:42370/0 (socket says 172.21.15.145:42370) 2024-05-09T04:35:46.119 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.119+0000 7ff1caa00640 1 -- 172.21.15.145:0/773911140 learned_addr learned my addr 172.21.15.145:0/773911140 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:46.120 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.119+0000 7ff1caa00640 1 -- 172.21.15.145:0/773911140 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff1cc0d3180 msgr2=0x7ff1cc15cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:46.120 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.119+0000 7ff1ca000640 1 --2- 172.21.15.145:0/773911140 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1cc0d22f0 0x7ff1cc15f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:46.120 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.119+0000 7ff1caa00640 1 --2- 172.21.15.145:0/773911140 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff1cc0d3180 0x7ff1cc15cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:46.120 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.119+0000 7ff1caa00640 1 -- 172.21.15.145:0/773911140 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1cc0d22f0 msgr2=0x7ff1cc15f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:46.120 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.119+0000 7ff1caa00640 1 --2- 172.21.15.145:0/773911140 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1cc0d22f0 0x7ff1cc15f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:46.120 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.119+0000 7ff1caa00640 1 -- 172.21.15.145:0/773911140 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1cc0ce680 con 0x7ff1cc0b4760 2024-05-09T04:35:46.120 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.120+0000 7ff1caa00640 1 --2- 172.21.15.145:0/773911140 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff1cc0b4760 0x7ff1cc1582e0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc00b440 tx=0x7ff1bc00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:46.120 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.120+0000 7ff1c2a00640 1 -- 172.21.15.145:0/773911140 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc015020 con 0x7ff1cc0b4760 2024-05-09T04:35:46.120 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.120+0000 7ff1c2a00640 1 -- 172.21.15.145:0/773911140 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1bc011890 con 0x7ff1cc0b4760 2024-05-09T04:35:46.120 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.120+0000 7ff1cbe00640 1 -- 172.21.15.145:0/773911140 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1cc0ce960 con 0x7ff1cc0b4760 2024-05-09T04:35:46.120 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.120+0000 7ff1c2a00640 1 -- 172.21.15.145:0/773911140 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc011c80 con 0x7ff1cc0b4760 2024-05-09T04:35:46.121 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.120+0000 7ff1cbe00640 1 -- 172.21.15.145:0/773911140 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1cc159440 con 0x7ff1cc0b4760 2024-05-09T04:35:46.121 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.121+0000 7ff1cbe00640 1 -- 172.21.15.145:0/773911140 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1cc06a310 con 0x7ff1cc0b4760 2024-05-09T04:35:46.121 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.122+0000 7ff1c2a00640 1 -- 172.21.15.145:0/773911140 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ff1bc018020 con 0x7ff1cc0b4760 2024-05-09T04:35:46.122 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.123+0000 7ff1c2a00640 1 --2- 172.21.15.145:0/773911140 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff1a80762d0 0x7ff1a8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:46.122 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.123+0000 7ff1c2a00640 1 -- 172.21.15.145:0/773911140 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7ff1bc097400 con 0x7ff1cc0b4760 2024-05-09T04:35:46.122 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.124+0000 7ff1ca000640 1 --2- 172.21.15.145:0/773911140 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff1a80762d0 0x7ff1a8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:46.123 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.124+0000 7ff1ca000640 1 --2- 172.21.15.145:0/773911140 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff1a80762d0 0x7ff1a8078790 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff1b4007fd0 tx=0x7ff1b4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:46.126 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.128+0000 7ff1c2a00640 1 -- 172.21.15.145:0/773911140 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff1bc01f020 con 0x7ff1cc0b4760 2024-05-09T04:35:46.413 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.414+0000 7ff1cbe00640 1 -- 172.21.15.145:0/773911140 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_backtrace.TestBacktrace.test_backtrace_flush_on_deleted_data_pool"]} v 0) -- 0x7ff1cc0d09b0 con 0x7ff1cc0b4760 2024-05-09T04:35:46.443 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.444+0000 7ff1c2a00640 1 -- 172.21.15.145:0/773911140 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_backtrace.TestBacktrace.test_backtrace_flush_on_deleted_data_pool"]}]=0 v129) ==== 162+0+0 (secure 0 0 0) 0x7ff1bc00d040 con 0x7ff1cc0b4760 2024-05-09T04:35:46.446 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.447+0000 7ff1cbe00640 1 -- 172.21.15.145:0/773911140 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff1a80762d0 msgr2=0x7ff1a8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:46.446 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.447+0000 7ff1cbe00640 1 --2- 172.21.15.145:0/773911140 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff1a80762d0 0x7ff1a8078790 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff1b4007fd0 tx=0x7ff1b4009040 comp rx=0 tx=0).stop 2024-05-09T04:35:46.446 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.447+0000 7ff1cbe00640 1 -- 172.21.15.145:0/773911140 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff1cc0b4760 msgr2=0x7ff1cc1582e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:46.447 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.447+0000 7ff1cbe00640 1 --2- 172.21.15.145:0/773911140 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff1cc0b4760 0x7ff1cc1582e0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc00b440 tx=0x7ff1bc00b900 comp rx=0 tx=0).stop 2024-05-09T04:35:46.447 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.447+0000 7ff1cbe00640 1 -- 172.21.15.145:0/773911140 shutdown_connections 2024-05-09T04:35:46.447 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.447+0000 7ff1cbe00640 1 --2- 172.21.15.145:0/773911140 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ff1a80762d0 0x7ff1a8078790 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:46.447 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.447+0000 7ff1cbe00640 1 --2- 172.21.15.145:0/773911140 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff1cc0d3180 0x7ff1cc15cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:46.447 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.447+0000 7ff1cbe00640 1 --2- 172.21.15.145:0/773911140 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1cc0d22f0 0x7ff1cc15f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:46.447 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.447+0000 7ff1cbe00640 1 --2- 172.21.15.145:0/773911140 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ff1cc0b4760 0x7ff1cc1582e0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:46.447 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.447+0000 7ff1cbe00640 1 -- 172.21.15.145:0/773911140 >> 172.21.15.145:0/773911140 conn(0x7ff1cc0d4fb0 msgr2=0x7ff1cc0cfdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:46.447 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.447+0000 7ff1cbe00640 1 -- 172.21.15.145:0/773911140 shutdown_connections 2024-05-09T04:35:46.447 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:46.447+0000 7ff1cbe00640 1 -- 172.21.15.145:0/773911140 wait complete. 2024-05-09T04:35:46.468 INFO:tasks.cephfs_test_runner:that the MDS does not go read-only when handling backtrace update errors ... FAIL 2024-05-09T04:35:46.469 INFO:tasks.cephfs_test_runner: 2024-05-09T04:35:46.469 INFO:tasks.cephfs_test_runner:====================================================================== 2024-05-09T04:35:46.469 INFO:tasks.cephfs_test_runner:FAIL: test_backtrace_flush_on_deleted_data_pool (tasks.cephfs.test_backtrace.TestBacktrace) 2024-05-09T04:35:46.469 INFO:tasks.cephfs_test_runner:that the MDS does not go read-only when handling backtrace update errors 2024-05-09T04:35:46.469 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-05-09T04:35:46.469 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-05-09T04:35:46.469 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_165a15e51d08073b5317c443415b0d8a7141b1ab/qa/tasks/cephfs/test_backtrace.py", line 117, in test_backtrace_flush_on_deleted_data_pool 2024-05-09T04:35:46.470 INFO:tasks.cephfs_test_runner: extra_data_pool_name_2 = self.fs.get_data_pool_name() + '_extra2' 2024-05-09T04:35:46.470 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_165a15e51d08073b5317c443415b0d8a7141b1ab/qa/tasks/cephfs/filesystem.py", line 1024, in get_data_pool_name 2024-05-09T04:35:46.470 INFO:tasks.cephfs_test_runner: assert(len(self.data_pools) == 1) 2024-05-09T04:35:46.470 INFO:tasks.cephfs_test_runner:AssertionError 2024-05-09T04:35:46.470 INFO:tasks.cephfs_test_runner: 2024-05-09T04:35:46.470 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-05-09T04:35:46.470 INFO:tasks.cephfs_test_runner:Ran 2 tests in 85.407s 2024-05-09T04:35:46.470 INFO:tasks.cephfs_test_runner: 2024-05-09T04:35:46.473 INFO:tasks.cephfs_test_runner:FAILED (failures=1) 2024-05-09T04:35:46.473 INFO:tasks.cephfs_test_runner: 2024-05-09T04:35:46.473 INFO:tasks.cephfs_test_runner:====================================================================== 2024-05-09T04:35:46.473 INFO:tasks.cephfs_test_runner:FAIL: test_backtrace_flush_on_deleted_data_pool (tasks.cephfs.test_backtrace.TestBacktrace) 2024-05-09T04:35:46.473 INFO:tasks.cephfs_test_runner:that the MDS does not go read-only when handling backtrace update errors 2024-05-09T04:35:46.473 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-05-09T04:35:46.474 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-05-09T04:35:46.474 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_165a15e51d08073b5317c443415b0d8a7141b1ab/qa/tasks/cephfs/test_backtrace.py", line 117, in test_backtrace_flush_on_deleted_data_pool 2024-05-09T04:35:46.474 INFO:tasks.cephfs_test_runner: extra_data_pool_name_2 = self.fs.get_data_pool_name() + '_extra2' 2024-05-09T04:35:46.474 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_165a15e51d08073b5317c443415b0d8a7141b1ab/qa/tasks/cephfs/filesystem.py", line 1024, in get_data_pool_name 2024-05-09T04:35:46.474 INFO:tasks.cephfs_test_runner: assert(len(self.data_pools) == 1) 2024-05-09T04:35:46.474 INFO:tasks.cephfs_test_runner:AssertionError 2024-05-09T04:35:46.474 INFO:tasks.cephfs_test_runner: 2024-05-09T04:35:46.475 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_165a15e51d08073b5317c443415b0d8a7141b1ab/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_backtrace_flush_on_deleted_data_pool (tasks.cephfs.test_backtrace.TestBacktrace) 2024-05-09T04:35:46.722 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=4cd0f71babcb4eb7a6fae31d35118bbb Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_165a15e51d08073b5317c443415b0d8a7141b1ab/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_backtrace_flush_on_deleted_data_pool (tasks.cephfs.test_backtrace.TestBacktrace) 2024-05-09T04:35:46.724 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-05-09T04:35:46.733 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-05-09T04:35:46.772 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-05-09T04:35:46.772 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-05-09T04:35:46.772 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T04:35:46.831 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-09T04:35:46.831 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2024-05-09T04:35:46.831 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T04:35:46.883 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-05-09T04:35:46.883 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-09T04:35:46.883 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T04:35:46.940 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-09T04:35:46.940 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-05-09T04:35:46.940 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T04:35:46.988 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-05-09T04:35:46.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:35:46.989 DEBUG:teuthology.orchestra.run.smithi192:> ip netns list 2024-05-09T04:35:47.082 INFO:teuthology.orchestra.run.smithi192.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-09T04:35:47.083 INFO:teuthology.orchestra.run.smithi192.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-05-09T04:35:47.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:35:47.084 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-09T04:35:47.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:35:47.174 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-05-09T04:35:47.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:35:47.252 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link delete ceph-brx 2024-05-09T04:35:47.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:35:47.374 DEBUG:teuthology.orchestra.run.smithi145:> ip netns list 2024-05-09T04:35:47.433 INFO:teuthology.orchestra.run.smithi145.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-09T04:35:47.433 INFO:teuthology.orchestra.run.smithi145.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-05-09T04:35:47.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:35:47.434 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T04:35:47.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:35:47.515 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-05-09T04:35:47.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:35:47.591 DEBUG:teuthology.orchestra.run.smithi145:> sudo ip link delete ceph-brx 2024-05-09T04:35:47.733 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-05-09T04:35:47.743 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-09T04:35:47.743 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T04:35:47.922 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.922+0000 7fba44400640 1 Processor -- start 2024-05-09T04:35:47.923 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.922+0000 7fba44400640 1 -- start start 2024-05-09T04:35:47.923 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.923+0000 7fba44400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fba3c15d780 0x7fba3c160c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:47.923 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.923+0000 7fba44400640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba3c1615c0 con 0x7fba3c0b4910 2024-05-09T04:35:47.924 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.923+0000 7fba44400640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba3c1618a0 con 0x7fba3c06e860 2024-05-09T04:35:47.924 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.923+0000 7fba44400640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fba3c0b4110 con 0x7fba3c15d780 2024-05-09T04:35:47.924 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.924+0000 7fba42400640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7fba3c0b4910 0x7fba3c0b4d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:52052/0 (socket says 172.21.15.145:52052) 2024-05-09T04:35:47.924 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.924+0000 7fba42400640 1 -- 172.21.15.145:0/2725154272 learned_addr learned my addr 172.21.15.145:0/2725154272 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:35:47.928 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.924+0000 7fba41000640 1 --2- 172.21.15.145:0/2725154272 >> v2:172.21.15.192:3300/0 conn(0x7fba3c15d780 0x7fba3c160c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:47.928 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.924+0000 7fba3be00640 1 -- 172.21.15.145:0/2725154272 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1046874788 0 0) 0x7fba3c1615c0 con 0x7fba3c0b4910 2024-05-09T04:35:47.928 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.924+0000 7fba3be00640 1 -- 172.21.15.145:0/2725154272 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fba24003520 con 0x7fba3c0b4910 2024-05-09T04:35:47.928 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.925+0000 7fba3be00640 1 -- 172.21.15.145:0/2725154272 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 130213805 0 0) 0x7fba24003520 con 0x7fba3c0b4910 2024-05-09T04:35:47.928 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.925+0000 7fba3be00640 1 -- 172.21.15.145:0/2725154272 >> v2:172.21.15.192:3300/0 conn(0x7fba3c15d780 msgr2=0x7fba3c160c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:47.929 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.925+0000 7fba3be00640 1 --2- 172.21.15.145:0/2725154272 >> v2:172.21.15.192:3300/0 conn(0x7fba3c15d780 0x7fba3c160c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:47.929 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.925+0000 7fba3be00640 1 -- 172.21.15.145:0/2725154272 >> v1:172.21.15.192:6789/0 conn(0x7fba3c06e860 legacy=0x7fba3c06ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:47.929 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.925+0000 7fba3be00640 1 -- 172.21.15.145:0/2725154272 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba3c0b43f0 con 0x7fba3c0b4910 2024-05-09T04:35:47.929 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.925+0000 7fba3be00640 1 -- 172.21.15.145:0/2725154272 <== mon.0 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3474413261 0 0) 0x7fba34001a70 con 0x7fba3c0b4910 2024-05-09T04:35:47.929 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.925+0000 7fba3be00640 1 -- 172.21.15.145:0/2725154272 >> v1:172.21.15.145:6789/0 conn(0x7fba3c0b4910 legacy=0x7fba3c0b4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:47.930 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.925+0000 7fba3be00640 1 --2- 172.21.15.145:0/2725154272 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fba24003aa0 0x7fba24003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:47.930 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.925+0000 7fba3be00640 1 --2- 172.21.15.145:0/2725154272 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fba24004800 0x7fba24004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:47.930 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.925+0000 7fba3be00640 1 --2- 172.21.15.145:0/2725154272 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba24005210 0x7fba2400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:47.930 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.925+0000 7fba3be00640 1 -- 172.21.15.145:0/2725154272 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fba3c0b4110 con 0x7fba24004800 2024-05-09T04:35:47.930 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.925+0000 7fba3be00640 1 -- 172.21.15.145:0/2725154272 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fba2400c160 con 0x7fba24005210 2024-05-09T04:35:47.931 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.925+0000 7fba3be00640 1 -- 172.21.15.145:0/2725154272 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fba2400c360 con 0x7fba24003aa0 2024-05-09T04:35:47.931 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.925+0000 7fba41000640 1 --2- 172.21.15.145:0/2725154272 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fba24004800 0x7fba24004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:47.931 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.925+0000 7fba41000640 1 -- 172.21.15.145:0/2725154272 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fba24003aa0 msgr2=0x7fba24003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:47.931 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.925+0000 7fba41000640 1 --2- 172.21.15.145:0/2725154272 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fba24003aa0 0x7fba24003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:47.931 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.925+0000 7fba41000640 1 -- 172.21.15.145:0/2725154272 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba24005210 msgr2=0x7fba2400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:47.931 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.925+0000 7fba41000640 1 --2- 172.21.15.145:0/2725154272 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba24005210 0x7fba2400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:47.931 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.925+0000 7fba41000640 1 -- 172.21.15.145:0/2725154272 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba24003520 con 0x7fba24004800 2024-05-09T04:35:47.931 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.925+0000 7fba42400640 1 -- 172.21.15.145:0/2725154272 reap_dead start 2024-05-09T04:35:47.931 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.926+0000 7fba41000640 1 --2- 172.21.15.145:0/2725154272 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fba24004800 0x7fba24004c80 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fba2c009a90 tx=0x7fba24005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19b4e74100a177cf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:47.932 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.927+0000 7fba3be00640 1 -- 172.21.15.145:0/2725154272 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2c036070 con 0x7fba24004800 2024-05-09T04:35:47.932 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.927+0000 7fba3be00640 1 -- 172.21.15.145:0/2725154272 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba2c031400 con 0x7fba24004800 2024-05-09T04:35:47.932 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.927+0000 7fba3be00640 1 -- 172.21.15.145:0/2725154272 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2c009690 con 0x7fba24004800 2024-05-09T04:35:47.932 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.927+0000 7fba44400640 1 -- 172.21.15.145:0/2725154272 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fba24004800 msgr2=0x7fba24004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:47.932 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.927+0000 7fba44400640 1 --2- 172.21.15.145:0/2725154272 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fba24004800 0x7fba24004c80 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fba2c009a90 tx=0x7fba24005830 comp rx=0 tx=0).stop 2024-05-09T04:35:47.932 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.927+0000 7fba44400640 1 -- 172.21.15.145:0/2725154272 shutdown_connections 2024-05-09T04:35:47.932 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.927+0000 7fba44400640 1 --2- 172.21.15.145:0/2725154272 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fba24004800 0x7fba24004c80 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:47.932 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.927+0000 7fba44400640 1 -- 172.21.15.145:0/2725154272 >> 172.21.15.145:0/2725154272 conn(0x7fba3c06c870 msgr2=0x7fba3c06cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:47.932 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.928+0000 7fba44400640 1 -- 172.21.15.145:0/2725154272 shutdown_connections 2024-05-09T04:35:47.933 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.928+0000 7fba44400640 1 -- 172.21.15.145:0/2725154272 wait complete. 2024-05-09T04:35:47.935 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.929+0000 7fba44400640 1 Processor -- start 2024-05-09T04:35:47.936 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.929+0000 7fba44400640 1 -- start start 2024-05-09T04:35:47.936 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.930+0000 7fba44400640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba24004800 0x7fba3c0afbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:47.936 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.930+0000 7fba44400640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fba3c159220 0x7fba3c1f7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:47.936 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.930+0000 7fba44400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fba3c072d50 0x7fba3c0731b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:47.936 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.930+0000 7fba44400640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fba3c06ec40 con 0x7fba3c072d50 2024-05-09T04:35:47.937 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.931+0000 7fba44400640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fba3c146e10 con 0x7fba24004800 2024-05-09T04:35:47.937 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.931+0000 7fba44400640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fba3c06edc0 con 0x7fba3c159220 2024-05-09T04:35:47.937 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.931+0000 7fba42400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fba3c072d50 0x7fba3c0731b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:47.937 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.931+0000 7fba42400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fba3c072d50 0x7fba3c0731b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:42378/0 (socket says 172.21.15.145:42378) 2024-05-09T04:35:47.937 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.931+0000 7fba42400640 1 -- 172.21.15.145:0/889214293 learned_addr learned my addr 172.21.15.145:0/889214293 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:47.938 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.931+0000 7fba41a00640 1 --2- 172.21.15.145:0/889214293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba24004800 0x7fba3c0afbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:47.938 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.931+0000 7fba42400640 1 -- 172.21.15.145:0/889214293 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fba3c159220 msgr2=0x7fba3c1f7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:47.938 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.931+0000 7fba42400640 1 --2- 172.21.15.145:0/889214293 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fba3c159220 0x7fba3c1f7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:47.938 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.931+0000 7fba42400640 1 -- 172.21.15.145:0/889214293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba24004800 msgr2=0x7fba3c0afbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:47.938 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.931+0000 7fba42400640 1 --2- 172.21.15.145:0/889214293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba24004800 0x7fba3c0afbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:47.938 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.931+0000 7fba42400640 1 -- 172.21.15.145:0/889214293 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba3c159770 con 0x7fba3c072d50 2024-05-09T04:35:47.939 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.932+0000 7fba42400640 1 --2- 172.21.15.145:0/889214293 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fba3c072d50 0x7fba3c0731b0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fba3c06fe60 tx=0x7fba3400cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:47.939 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.932+0000 7fba39600640 1 -- 172.21.15.145:0/889214293 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba34004d60 con 0x7fba3c072d50 2024-05-09T04:35:47.939 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.932+0000 7fba39600640 1 -- 172.21.15.145:0/889214293 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba3400a610 con 0x7fba3c072d50 2024-05-09T04:35:47.939 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.932+0000 7fba39600640 1 -- 172.21.15.145:0/889214293 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba3400aa00 con 0x7fba3c072d50 2024-05-09T04:35:47.939 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.932+0000 7fba44400640 1 -- 172.21.15.145:0/889214293 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba3c159a50 con 0x7fba3c072d50 2024-05-09T04:35:47.939 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.932+0000 7fba44400640 1 -- 172.21.15.145:0/889214293 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba3c15a010 con 0x7fba3c072d50 2024-05-09T04:35:47.940 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.933+0000 7fba44400640 1 -- 172.21.15.145:0/889214293 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba3c06a310 con 0x7fba3c072d50 2024-05-09T04:35:47.940 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.934+0000 7fba39600640 1 -- 172.21.15.145:0/889214293 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fba34011070 con 0x7fba3c072d50 2024-05-09T04:35:47.940 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.936+0000 7fba39600640 1 --2- 172.21.15.145:0/889214293 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fba200762d0 0x7fba20078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:47.940 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.936+0000 7fba39600640 1 -- 172.21.15.145:0/889214293 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7fba340968a0 con 0x7fba3c072d50 2024-05-09T04:35:47.940 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.936+0000 7fba41a00640 1 --2- 172.21.15.145:0/889214293 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fba200762d0 0x7fba20078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:47.941 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.937+0000 7fba41a00640 1 --2- 172.21.15.145:0/889214293 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fba200762d0 0x7fba20078790 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fba30001590 tx=0x7fba30009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:47.941 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:47.941+0000 7fba39600640 1 -- 172.21.15.145:0/889214293 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fba34060460 con 0x7fba3c072d50 2024-05-09T04:35:48.143 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.144+0000 7fba44400640 1 -- 172.21.15.145:0/889214293 --> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fba3c13d130 con 0x7fba200762d0 2024-05-09T04:35:48.145 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.145+0000 7fba39600640 1 -- 172.21.15.145:0/889214293 <== mgr.4104 v2:172.21.15.145:6832/2725855396 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13478 (secure 0 0 0) 0x7fba3c13d130 con 0x7fba200762d0 2024-05-09T04:35:48.145 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:35:48.146 INFO:teuthology.orchestra.run.smithi145.stderr:dumped all 2024-05-09T04:35:48.148 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.148+0000 7fba44400640 1 -- 172.21.15.145:0/889214293 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fba200762d0 msgr2=0x7fba20078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:48.148 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.148+0000 7fba44400640 1 --2- 172.21.15.145:0/889214293 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fba200762d0 0x7fba20078790 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fba30001590 tx=0x7fba30009040 comp rx=0 tx=0).stop 2024-05-09T04:35:48.148 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.148+0000 7fba44400640 1 -- 172.21.15.145:0/889214293 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fba3c072d50 msgr2=0x7fba3c0731b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:48.148 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.148+0000 7fba44400640 1 --2- 172.21.15.145:0/889214293 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fba3c072d50 0x7fba3c0731b0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fba3c06fe60 tx=0x7fba3400cb40 comp rx=0 tx=0).stop 2024-05-09T04:35:48.149 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.148+0000 7fba44400640 1 -- 172.21.15.145:0/889214293 shutdown_connections 2024-05-09T04:35:48.149 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.148+0000 7fba44400640 1 --2- 172.21.15.145:0/889214293 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fba200762d0 0x7fba20078790 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:48.149 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.148+0000 7fba44400640 1 --2- 172.21.15.145:0/889214293 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fba3c072d50 0x7fba3c0731b0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:48.149 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.148+0000 7fba44400640 1 --2- 172.21.15.145:0/889214293 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fba3c159220 0x7fba3c1f7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:48.149 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.148+0000 7fba44400640 1 --2- 172.21.15.145:0/889214293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba24004800 0x7fba3c0afbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:48.149 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.148+0000 7fba44400640 1 -- 172.21.15.145:0/889214293 >> 172.21.15.145:0/889214293 conn(0x7fba3c06c870 msgr2=0x7fba3c160c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:48.150 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.148+0000 7fba44400640 1 -- 172.21.15.145:0/889214293 shutdown_connections 2024-05-09T04:35:48.150 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.148+0000 7fba44400640 1 -- 172.21.15.145:0/889214293 wait complete. 2024-05-09T04:35:48.166 INFO:teuthology.orchestra.run.smithi145.stdout:{"pg_ready":true,"pg_map":{"version":112,"stamp":"2024-05-09T04:35:47.790261+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":235,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238988,"kb_used_data":6172,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735732,"statfs":{"total":773094113280,"available":772849389568,"internally_reserved":0,"allocated":6320128,"data_stored":3359375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14117,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-25,"num_whiteouts":0,"num_read":-1,"num_read_kb":0,"num_write":-76,"num_write_kb":-39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001801"},"pg_stats":[{"pgid":"1.0","version":"16'72","reported_seq":174,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-09T04:35:44.565986+0000","last_change":"2024-05-09T04:33:41.092063+0000","last_active":"2024-05-09T04:35:44.565986+0000","last_peered":"2024-05-09T04:35:44.565986+0000","last_clean":"2024-05-09T04:35:44.565986+0000","last_became_active":"2024-05-09T04:33:41.091194+0000","last_became_peered":"2024-05-09T04:33:41.091194+0000","last_unstale":"2024-05-09T04:35:44.565986+0000","last_undegraded":"2024-05-09T04:35:44.565986+0000","last_fullsized":"2024-05-09T04:35:44.565986+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:40.045339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:40.045339+0000","last_clean_scrub_stamp":"2024-05-09T04:33:40.045339+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:04.908456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574878,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28236,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343604,"statfs":{"total":96636764160,"available":96607850496,"internally_reserved":0,"allocated":1245184,"data_stored":866367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574877,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27664,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":651264,"data_stored":275999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574875,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27660,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344180,"statfs":{"total":96636764160,"available":96608440320,"internally_reserved":0,"allocated":651264,"data_stored":275999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31788,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340052,"statfs":{"total":96636764160,"available":96604213248,"internally_reserved":0,"allocated":622592,"data_stored":264111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574876,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31804,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340036,"statfs":{"total":96636764160,"available":96604196864,"internally_reserved":0,"allocated":634880,"data_stored":269995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574877,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27648,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344192,"statfs":{"total":96636764160,"available":96608452608,"internally_reserved":0,"allocated":643072,"data_stored":276848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31792,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340048,"statfs":{"total":96636764160,"available":96604209152,"internally_reserved":0,"allocated":626688,"data_stored":263689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574876,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32396,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339444,"statfs":{"total":96636764160,"available":96603590656,"internally_reserved":0,"allocated":1245184,"data_stored":866367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T04:35:48.167 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T04:35:48.367 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.367+0000 7f00d7200640 1 Processor -- start 2024-05-09T04:35:48.367 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.367+0000 7f00d7200640 1 -- start start 2024-05-09T04:35:48.369 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.369+0000 7f00d7200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f00d8163270 0x7f00d806e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:48.370 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.369+0000 7f00d7200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f00d8073c70 0x7f00d8166fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:48.370 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.369+0000 7f00d7200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00d80af5e0 con 0x7f00d80af130 2024-05-09T04:35:48.370 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.369+0000 7f00d7200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f00d80af890 con 0x7f00d8073c70 2024-05-09T04:35:48.370 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.369+0000 7f00d7200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f00d80afb70 con 0x7f00d8163270 2024-05-09T04:35:48.370 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.369+0000 7f00d6800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f00d8073c70 0x7f00d8166fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:48.370 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.369+0000 7f00d6800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f00d8073c70 0x7f00d8166fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:42386/0 (socket says 172.21.15.145:42386) 2024-05-09T04:35:48.371 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.369+0000 7f00d6800640 1 -- 172.21.15.145:0/1526524599 learned_addr learned my addr 172.21.15.145:0/1526524599 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:48.371 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.369+0000 7f00d5e00640 1 --2- 172.21.15.145:0/1526524599 >> v2:172.21.15.192:3300/0 conn(0x7f00d8163270 0x7f00d806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:48.371 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.370+0000 7f00d6800640 1 -- 172.21.15.145:0/1526524599 >> v2:172.21.15.192:3300/0 conn(0x7f00d8163270 msgr2=0x7f00d806e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:48.371 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.370+0000 7f00d6800640 1 --2- 172.21.15.145:0/1526524599 >> v2:172.21.15.192:3300/0 conn(0x7f00d8163270 0x7f00d806e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:48.371 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.370+0000 7f00d6800640 1 -- 172.21.15.145:0/1526524599 >> v1:172.21.15.192:6789/0 conn(0x7f00d80af130 legacy=0x7f00d8073560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:48.372 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.370+0000 7f00d6800640 1 -- 172.21.15.145:0/1526524599 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00d8072a80 con 0x7f00d8073c70 2024-05-09T04:35:48.373 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.371+0000 7f00d6800640 1 --2- 172.21.15.145:0/1526524599 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f00d8073c70 0x7f00d8166fa0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f00d8070050 tx=0x7f00d002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2dc758fc3e61468 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:48.373 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.371+0000 7f00d4a00640 1 -- 172.21.15.145:0/1526524599 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d0036070 con 0x7f00d8073c70 2024-05-09T04:35:48.374 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.371+0000 7f00d4a00640 1 -- 172.21.15.145:0/1526524599 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00d0031600 con 0x7f00d8073c70 2024-05-09T04:35:48.374 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.371+0000 7f00d4a00640 1 -- 172.21.15.145:0/1526524599 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d00319d0 con 0x7f00d8073c70 2024-05-09T04:35:48.374 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.372+0000 7f00d7200640 1 -- 172.21.15.145:0/1526524599 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f00d8073c70 msgr2=0x7f00d8166fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:48.374 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.372+0000 7f00d7200640 1 --2- 172.21.15.145:0/1526524599 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f00d8073c70 0x7f00d8166fa0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f00d8070050 tx=0x7f00d002a610 comp rx=0 tx=0).stop 2024-05-09T04:35:48.374 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.372+0000 7f00d7200640 1 -- 172.21.15.145:0/1526524599 shutdown_connections 2024-05-09T04:35:48.375 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.372+0000 7f00d7200640 1 --2- 172.21.15.145:0/1526524599 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f00d8073c70 0x7f00d8166fa0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:48.375 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.372+0000 7f00d7200640 1 --2- 172.21.15.145:0/1526524599 >> v2:172.21.15.192:3300/0 conn(0x7f00d8163270 0x7f00d806e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:48.375 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.372+0000 7f00d7200640 1 -- 172.21.15.145:0/1526524599 >> 172.21.15.145:0/1526524599 conn(0x7f00d806c8a0 msgr2=0x7f00d806ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:48.375 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.372+0000 7f00d7200640 1 -- 172.21.15.145:0/1526524599 shutdown_connections 2024-05-09T04:35:48.376 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.372+0000 7f00d7200640 1 -- 172.21.15.145:0/1526524599 wait complete. 2024-05-09T04:35:48.376 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.373+0000 7f00d7200640 1 Processor -- start 2024-05-09T04:35:48.376 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.374+0000 7f00d7200640 1 -- start start 2024-05-09T04:35:48.379 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.374+0000 7f00d7200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00d8073c70 0x7f00d815f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:48.379 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.374+0000 7f00d7200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f00d80af130 0x7f00d81fbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:48.379 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.374+0000 7f00d7200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f00d8163270 0x7f00d81fc510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:48.380 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.374+0000 7f00d7200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f00d80740e0 con 0x7f00d80af130 2024-05-09T04:35:48.380 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.374+0000 7f00d7200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f00d8073040 con 0x7f00d8073c70 2024-05-09T04:35:48.380 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.374+0000 7f00d7200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f00d8074260 con 0x7f00d8163270 2024-05-09T04:35:48.380 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.375+0000 7f00d5400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f00d80af130 0x7f00d81fbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:48.380 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.375+0000 7f00d5400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f00d80af130 0x7f00d81fbfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:42398/0 (socket says 172.21.15.145:42398) 2024-05-09T04:35:48.380 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.375+0000 7f00d5400640 1 -- 172.21.15.145:0/4197961717 learned_addr learned my addr 172.21.15.145:0/4197961717 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:48.381 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.375+0000 7f00d5e00640 1 --2- 172.21.15.145:0/4197961717 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00d8073c70 0x7f00d815f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:48.381 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.375+0000 7f00d5400640 1 -- 172.21.15.145:0/4197961717 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f00d8163270 msgr2=0x7f00d81fc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:48.381 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.375+0000 7f00d5400640 1 --2- 172.21.15.145:0/4197961717 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f00d8163270 0x7f00d81fc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:48.381 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.375+0000 7f00d5400640 1 -- 172.21.15.145:0/4197961717 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00d8073c70 msgr2=0x7f00d815f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:48.381 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.375+0000 7f00d5400640 1 --2- 172.21.15.145:0/4197961717 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00d8073c70 0x7f00d815f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:48.381 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.375+0000 7f00d5400640 1 -- 172.21.15.145:0/4197961717 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00d80d02c0 con 0x7f00d80af130 2024-05-09T04:35:48.381 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.375+0000 7f00d5400640 1 --2- 172.21.15.145:0/4197961717 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f00d80af130 0x7f00d81fbfd0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f00c400c430 tx=0x7f00c400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:48.381 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.376+0000 7f00ce000640 1 -- 172.21.15.145:0/4197961717 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c4011790 con 0x7f00d80af130 2024-05-09T04:35:48.382 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.376+0000 7f00ce000640 1 -- 172.21.15.145:0/4197961717 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00c4017070 con 0x7f00d80af130 2024-05-09T04:35:48.382 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.376+0000 7f00ce000640 1 -- 172.21.15.145:0/4197961717 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c4011bb0 con 0x7f00d80af130 2024-05-09T04:35:48.382 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.376+0000 7f00d7200640 1 -- 172.21.15.145:0/4197961717 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00d80cf500 con 0x7f00d80af130 2024-05-09T04:35:48.382 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.376+0000 7f00d7200640 1 -- 172.21.15.145:0/4197961717 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00d80cfac0 con 0x7f00d80af130 2024-05-09T04:35:48.382 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.377+0000 7f00d7200640 1 -- 172.21.15.145:0/4197961717 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00d806a310 con 0x7f00d80af130 2024-05-09T04:35:48.382 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.378+0000 7f00ce000640 1 -- 172.21.15.145:0/4197961717 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f00c401c460 con 0x7f00d80af130 2024-05-09T04:35:48.382 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.379+0000 7f00ce000640 1 --2- 172.21.15.145:0/4197961717 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f00b80762d0 0x7f00b8078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:48.382 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.379+0000 7f00ce000640 1 -- 172.21.15.145:0/4197961717 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7f00c40971c0 con 0x7f00d80af130 2024-05-09T04:35:48.383 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.379+0000 7f00d5e00640 1 --2- 172.21.15.145:0/4197961717 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f00b80762d0 0x7f00b8078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:48.383 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.380+0000 7f00d5e00640 1 --2- 172.21.15.145:0/4197961717 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f00b80762d0 0x7f00b8078790 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f00c8006fd0 tx=0x7f00c8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:48.384 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.383+0000 7f00ce000640 1 -- 172.21.15.145:0/4197961717 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f00c4060e10 con 0x7f00d80af130 2024-05-09T04:35:48.586 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.586+0000 7f00d7200640 1 -- 172.21.15.145:0/4197961717 --> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f00d8075a70 con 0x7f00b80762d0 2024-05-09T04:35:48.587 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.587+0000 7f00ce000640 1 -- 172.21.15.145:0/4197961717 <== mgr.4104 v2:172.21.15.145:6832/2725855396 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13478 (secure 0 0 0) 0x7f00d8075a70 con 0x7f00b80762d0 2024-05-09T04:35:48.588 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:35:48.588 INFO:teuthology.orchestra.run.smithi145.stderr:dumped all 2024-05-09T04:35:48.590 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.590+0000 7f00d7200640 1 -- 172.21.15.145:0/4197961717 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f00b80762d0 msgr2=0x7f00b8078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:48.591 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.590+0000 7f00d7200640 1 --2- 172.21.15.145:0/4197961717 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f00b80762d0 0x7f00b8078790 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f00c8006fd0 tx=0x7f00c8008040 comp rx=0 tx=0).stop 2024-05-09T04:35:48.591 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.590+0000 7f00d7200640 1 -- 172.21.15.145:0/4197961717 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f00d80af130 msgr2=0x7f00d81fbfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:48.591 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.590+0000 7f00d7200640 1 --2- 172.21.15.145:0/4197961717 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f00d80af130 0x7f00d81fbfd0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f00c400c430 tx=0x7f00c400c8f0 comp rx=0 tx=0).stop 2024-05-09T04:35:48.591 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.590+0000 7f00d7200640 1 -- 172.21.15.145:0/4197961717 shutdown_connections 2024-05-09T04:35:48.591 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.590+0000 7f00d7200640 1 --2- 172.21.15.145:0/4197961717 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f00b80762d0 0x7f00b8078790 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:48.591 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.590+0000 7f00d7200640 1 --2- 172.21.15.145:0/4197961717 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f00d8163270 0x7f00d81fc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:48.592 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.590+0000 7f00d7200640 1 --2- 172.21.15.145:0/4197961717 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f00d80af130 0x7f00d81fbfd0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:48.592 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.590+0000 7f00d7200640 1 --2- 172.21.15.145:0/4197961717 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00d8073c70 0x7f00d815f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:48.592 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.590+0000 7f00d7200640 1 -- 172.21.15.145:0/4197961717 >> 172.21.15.145:0/4197961717 conn(0x7f00d806c8a0 msgr2=0x7f00d806cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:48.592 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.591+0000 7f00d7200640 1 -- 172.21.15.145:0/4197961717 shutdown_connections 2024-05-09T04:35:48.592 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.591+0000 7f00d7200640 1 -- 172.21.15.145:0/4197961717 wait complete. 2024-05-09T04:35:48.609 INFO:teuthology.orchestra.run.smithi145.stdout:{"pg_ready":true,"pg_map":{"version":112,"stamp":"2024-05-09T04:35:47.790261+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":235,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238988,"kb_used_data":6172,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735732,"statfs":{"total":773094113280,"available":772849389568,"internally_reserved":0,"allocated":6320128,"data_stored":3359375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14117,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-25,"num_whiteouts":0,"num_read":-1,"num_read_kb":0,"num_write":-76,"num_write_kb":-39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001801"},"pg_stats":[{"pgid":"1.0","version":"16'72","reported_seq":174,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-09T04:35:44.565986+0000","last_change":"2024-05-09T04:33:41.092063+0000","last_active":"2024-05-09T04:35:44.565986+0000","last_peered":"2024-05-09T04:35:44.565986+0000","last_clean":"2024-05-09T04:35:44.565986+0000","last_became_active":"2024-05-09T04:33:41.091194+0000","last_became_peered":"2024-05-09T04:33:41.091194+0000","last_unstale":"2024-05-09T04:35:44.565986+0000","last_undegraded":"2024-05-09T04:35:44.565986+0000","last_fullsized":"2024-05-09T04:35:44.565986+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:40.045339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:40.045339+0000","last_clean_scrub_stamp":"2024-05-09T04:33:40.045339+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:04.908456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574878,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28236,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343604,"statfs":{"total":96636764160,"available":96607850496,"internally_reserved":0,"allocated":1245184,"data_stored":866367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574877,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27664,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":651264,"data_stored":275999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574875,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27660,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344180,"statfs":{"total":96636764160,"available":96608440320,"internally_reserved":0,"allocated":651264,"data_stored":275999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31788,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340052,"statfs":{"total":96636764160,"available":96604213248,"internally_reserved":0,"allocated":622592,"data_stored":264111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574876,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31804,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340036,"statfs":{"total":96636764160,"available":96604196864,"internally_reserved":0,"allocated":634880,"data_stored":269995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574877,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27648,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344192,"statfs":{"total":96636764160,"available":96608452608,"internally_reserved":0,"allocated":643072,"data_stored":276848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31792,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340048,"statfs":{"total":96636764160,"available":96604209152,"internally_reserved":0,"allocated":626688,"data_stored":263689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574876,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32396,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339444,"statfs":{"total":96636764160,"available":96603590656,"internally_reserved":0,"allocated":1245184,"data_stored":866367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T04:35:48.609 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-09T04:35:48.610 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T04:35:48.803 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.804+0000 7f36d6c00640 1 Processor -- start 2024-05-09T04:35:48.804 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.804+0000 7f36d6c00640 1 -- start start 2024-05-09T04:35:48.806 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.805+0000 7f36d6c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36d00b4700 0x7f36d0073c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:48.806 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.805+0000 7f36d6c00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f36d00af610 0x7f36d00afa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:48.806 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.805+0000 7f36d6c00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36d0072c00 con 0x7f36d00b4ae0 2024-05-09T04:35:48.806 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.805+0000 7f36d6c00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f36d00aea70 con 0x7f36d00af610 2024-05-09T04:35:48.806 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.805+0000 7f36d6c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f36d00737f0 con 0x7f36d00b4700 2024-05-09T04:35:48.807 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.806+0000 7f36d4e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f36d00af610 0x7f36d00afa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:48.807 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.806+0000 7f36d6200640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f36d00b4ae0 0x7f36d00b4f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:52054/0 (socket says 172.21.15.145:52054) 2024-05-09T04:35:48.807 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.806+0000 7f36d6200640 1 -- 172.21.15.145:0/3308030194 learned_addr learned my addr 172.21.15.145:0/3308030194 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:35:48.807 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.806+0000 7f36d4e00640 1 --2- 172.21.15.145:0/3308030194 >> v2:172.21.15.145:3300/0 conn(0x7f36d00af610 0x7f36d00afa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:42400/0 (socket says 172.21.15.145:42400) 2024-05-09T04:35:48.807 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.806+0000 7f36d5800640 1 --2- 172.21.15.145:0/3308030194 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36d00b4700 0x7f36d0073c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:48.807 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.806+0000 7f36d4e00640 1 -- 172.21.15.145:0/3308030194 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36d00b4700 msgr2=0x7f36d0073c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:48.807 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.806+0000 7f36d4e00640 1 --2- 172.21.15.145:0/3308030194 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36d00b4700 0x7f36d0073c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:48.807 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.806+0000 7f36d4e00640 1 -- 172.21.15.145:0/3308030194 >> v1:172.21.15.145:6789/0 conn(0x7f36d00b4ae0 legacy=0x7f36d00b4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:48.807 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.806+0000 7f36d4e00640 1 -- 172.21.15.145:0/3308030194 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36d0072ec0 con 0x7f36d00af610 2024-05-09T04:35:48.808 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.806+0000 7f36cfe00640 1 -- 172.21.15.145:0/3308030194 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2443199821 0 0) 0x7f36d0072c00 con 0x7f36d00b4ae0 2024-05-09T04:35:48.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.807+0000 7f36d4e00640 1 --2- 172.21.15.145:0/3308030194 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36d00af610 0x7f36d00afa10 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f36bc0097a0 tx=0x7f36bc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e50bf5c954655d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:48.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.808+0000 7f36cfe00640 1 -- 172.21.15.145:0/3308030194 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36bc037070 con 0x7f36d00af610 2024-05-09T04:35:48.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.808+0000 7f36cfe00640 1 -- 172.21.15.145:0/3308030194 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36bc032440 con 0x7f36d00af610 2024-05-09T04:35:48.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.808+0000 7f36cfe00640 1 -- 172.21.15.145:0/3308030194 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36bc032810 con 0x7f36d00af610 2024-05-09T04:35:48.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.808+0000 7f36d6c00640 1 -- 172.21.15.145:0/3308030194 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36d00af610 msgr2=0x7f36d00afa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:48.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.808+0000 7f36d6c00640 1 --2- 172.21.15.145:0/3308030194 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36d00af610 0x7f36d00afa10 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f36bc0097a0 tx=0x7f36bc02a180 comp rx=0 tx=0).stop 2024-05-09T04:35:48.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.808+0000 7f36d6c00640 1 -- 172.21.15.145:0/3308030194 shutdown_connections 2024-05-09T04:35:48.809 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.808+0000 7f36d6c00640 1 --2- 172.21.15.145:0/3308030194 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36d00af610 0x7f36d00afa10 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:48.810 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.808+0000 7f36d6c00640 1 --2- 172.21.15.145:0/3308030194 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36d00b4700 0x7f36d0073c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:48.810 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.808+0000 7f36d6c00640 1 -- 172.21.15.145:0/3308030194 >> 172.21.15.145:0/3308030194 conn(0x7f36d006c8a0 msgr2=0x7f36d006ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:48.810 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.809+0000 7f36d6c00640 1 -- 172.21.15.145:0/3308030194 shutdown_connections 2024-05-09T04:35:48.810 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.809+0000 7f36d6c00640 1 -- 172.21.15.145:0/3308030194 wait complete. 2024-05-09T04:35:48.811 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.810+0000 7f36d6c00640 1 Processor -- start 2024-05-09T04:35:48.811 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.811+0000 7f36d6c00640 1 -- start start 2024-05-09T04:35:48.812 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.812+0000 7f36d6c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36d00af610 0x7f36d01e7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:48.812 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.812+0000 7f36d6c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36d00b4700 0x7f36d015b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:48.812 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.812+0000 7f36d6c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36d00b4ae0 0x7f36d00b11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:48.812 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.812+0000 7f36d6c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f36d00741c0 con 0x7f36d00b4ae0 2024-05-09T04:35:48.812 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.812+0000 7f36d6c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f36d0073480 con 0x7f36d00af610 2024-05-09T04:35:48.812 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.812+0000 7f36d6c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f36d006e320 con 0x7f36d00b4700 2024-05-09T04:35:48.812 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.812+0000 7f36d6200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36d00b4ae0 0x7f36d00b11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:48.812 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.812+0000 7f36d6200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36d00b4ae0 0x7f36d00b11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:42412/0 (socket says 172.21.15.145:42412) 2024-05-09T04:35:48.813 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.812+0000 7f36d6200640 1 -- 172.21.15.145:0/26231634 learned_addr learned my addr 172.21.15.145:0/26231634 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:48.814 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.812+0000 7f36d5800640 1 --2- 172.21.15.145:0/26231634 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36d00af610 0x7f36d01e7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:48.814 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.813+0000 7f36d6200640 1 -- 172.21.15.145:0/26231634 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36d00b4700 msgr2=0x7f36d015b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:48.814 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.813+0000 7f36d6200640 1 --2- 172.21.15.145:0/26231634 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36d00b4700 0x7f36d015b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:48.814 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.813+0000 7f36d6200640 1 -- 172.21.15.145:0/26231634 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36d00af610 msgr2=0x7f36d01e7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:48.814 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.813+0000 7f36d6200640 1 --2- 172.21.15.145:0/26231634 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36d00af610 0x7f36d01e7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:48.814 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.813+0000 7f36d6200640 1 -- 172.21.15.145:0/26231634 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36d01e81e0 con 0x7f36d00b4ae0 2024-05-09T04:35:48.814 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.813+0000 7f36d6200640 1 --2- 172.21.15.145:0/26231634 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36d00b4ae0 0x7f36d00b11f0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f36c800c6e0 tx=0x7f36c800cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:48.815 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.813+0000 7f36cd600640 1 -- 172.21.15.145:0/26231634 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c8018070 con 0x7f36d00b4ae0 2024-05-09T04:35:48.815 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.813+0000 7f36cd600640 1 -- 172.21.15.145:0/26231634 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36c8004d60 con 0x7f36d00b4ae0 2024-05-09T04:35:48.815 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.813+0000 7f36d6c00640 1 -- 172.21.15.145:0/26231634 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36d01e84c0 con 0x7f36d00b4ae0 2024-05-09T04:35:48.815 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.813+0000 7f36cd600640 1 -- 172.21.15.145:0/26231634 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c800a880 con 0x7f36d00b4ae0 2024-05-09T04:35:48.815 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.813+0000 7f36d6c00640 1 -- 172.21.15.145:0/26231634 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36d01e8a80 con 0x7f36d00b4ae0 2024-05-09T04:35:48.816 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.814+0000 7f36d6c00640 1 -- 172.21.15.145:0/26231634 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36d006a310 con 0x7f36d00b4ae0 2024-05-09T04:35:48.816 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.816+0000 7f36cd600640 1 -- 172.21.15.145:0/26231634 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f36c801b070 con 0x7f36d00b4ae0 2024-05-09T04:35:48.816 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.816+0000 7f36cd600640 1 --2- 172.21.15.145:0/26231634 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f36b0076300 0x7f36b00787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:48.816 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.817+0000 7f36cd600640 1 -- 172.21.15.145:0/26231634 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7f36c8096870 con 0x7f36d00b4ae0 2024-05-09T04:35:48.817 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.817+0000 7f36d5800640 1 --2- 172.21.15.145:0/26231634 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f36b0076300 0x7f36b00787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:48.817 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.817+0000 7f36d5800640 1 --2- 172.21.15.145:0/26231634 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f36b0076300 0x7f36b00787c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f36c4006fd0 tx=0x7f36c4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:48.820 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:48.821+0000 7f36cd600640 1 -- 172.21.15.145:0/26231634 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f36c8060430 con 0x7f36d00b4ae0 2024-05-09T04:35:49.024 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.024+0000 7f36d6c00640 1 -- 172.21.15.145:0/26231634 --> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f36d009e870 con 0x7f36b0076300 2024-05-09T04:35:49.025 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.025+0000 7f36cd600640 1 -- 172.21.15.145:0/26231634 <== mgr.4104 v2:172.21.15.145:6832/2725855396 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13478 (secure 0 0 0) 0x7f36d009e870 con 0x7f36b0076300 2024-05-09T04:35:49.026 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:35:49.026 INFO:teuthology.orchestra.run.smithi145.stderr:dumped all 2024-05-09T04:35:49.028 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.028+0000 7f36d6c00640 1 -- 172.21.15.145:0/26231634 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f36b0076300 msgr2=0x7f36b00787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:49.028 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.028+0000 7f36d6c00640 1 --2- 172.21.15.145:0/26231634 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f36b0076300 0x7f36b00787c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f36c4006fd0 tx=0x7f36c4008040 comp rx=0 tx=0).stop 2024-05-09T04:35:49.028 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.028+0000 7f36d6c00640 1 -- 172.21.15.145:0/26231634 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36d00b4ae0 msgr2=0x7f36d00b11f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:49.028 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.028+0000 7f36d6c00640 1 --2- 172.21.15.145:0/26231634 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36d00b4ae0 0x7f36d00b11f0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f36c800c6e0 tx=0x7f36c800cba0 comp rx=0 tx=0).stop 2024-05-09T04:35:49.029 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.028+0000 7f36d6c00640 1 -- 172.21.15.145:0/26231634 shutdown_connections 2024-05-09T04:35:49.029 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.028+0000 7f36d6c00640 1 --2- 172.21.15.145:0/26231634 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f36b0076300 0x7f36b00787c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:49.029 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.029+0000 7f36d6c00640 1 --2- 172.21.15.145:0/26231634 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f36d00b4ae0 0x7f36d00b11f0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:49.029 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.029+0000 7f36d6c00640 1 --2- 172.21.15.145:0/26231634 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36d00b4700 0x7f36d015b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:49.029 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.029+0000 7f36d6c00640 1 --2- 172.21.15.145:0/26231634 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36d00af610 0x7f36d01e7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:49.029 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.029+0000 7f36d6c00640 1 -- 172.21.15.145:0/26231634 >> 172.21.15.145:0/26231634 conn(0x7f36d006c8a0 msgr2=0x7f36d015c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:49.029 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.029+0000 7f36d6c00640 1 -- 172.21.15.145:0/26231634 shutdown_connections 2024-05-09T04:35:49.030 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.029+0000 7f36d6c00640 1 -- 172.21.15.145:0/26231634 wait complete. 2024-05-09T04:35:49.048 INFO:teuthology.orchestra.run.smithi145.stdout:{"pg_ready":true,"pg_map":{"version":112,"stamp":"2024-05-09T04:35:47.790261+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":235,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238988,"kb_used_data":6172,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735732,"statfs":{"total":773094113280,"available":772849389568,"internally_reserved":0,"allocated":6320128,"data_stored":3359375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14117,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-25,"num_whiteouts":0,"num_read":-1,"num_read_kb":0,"num_write":-76,"num_write_kb":-39,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001801"},"pg_stats":[{"pgid":"1.0","version":"16'72","reported_seq":174,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-05-09T04:35:44.565986+0000","last_change":"2024-05-09T04:33:41.092063+0000","last_active":"2024-05-09T04:35:44.565986+0000","last_peered":"2024-05-09T04:35:44.565986+0000","last_clean":"2024-05-09T04:35:44.565986+0000","last_became_active":"2024-05-09T04:33:41.091194+0000","last_became_peered":"2024-05-09T04:33:41.091194+0000","last_unstale":"2024-05-09T04:35:44.565986+0000","last_undegraded":"2024-05-09T04:35:44.565986+0000","last_fullsized":"2024-05-09T04:35:44.565986+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:40.045339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:40.045339+0000","last_clean_scrub_stamp":"2024-05-09T04:33:40.045339+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:04.908456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574878,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28236,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343604,"statfs":{"total":96636764160,"available":96607850496,"internally_reserved":0,"allocated":1245184,"data_stored":866367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574877,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27664,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":651264,"data_stored":275999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574875,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27660,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344180,"statfs":{"total":96636764160,"available":96608440320,"internally_reserved":0,"allocated":651264,"data_stored":275999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31788,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340052,"statfs":{"total":96636764160,"available":96604213248,"internally_reserved":0,"allocated":622592,"data_stored":264111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574876,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31804,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340036,"statfs":{"total":96636764160,"available":96604196864,"internally_reserved":0,"allocated":634880,"data_stored":269995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574877,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27648,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344192,"statfs":{"total":96636764160,"available":96608452608,"internally_reserved":0,"allocated":643072,"data_stored":276848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31792,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340048,"statfs":{"total":96636764160,"available":96604209152,"internally_reserved":0,"allocated":626688,"data_stored":263689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574876,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32396,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339444,"statfs":{"total":96636764160,"available":96603590656,"internally_reserved":0,"allocated":1245184,"data_stored":866367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T04:35:49.049 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T04:35:49.249 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.249+0000 7f1ed7200640 1 Processor -- start 2024-05-09T04:35:49.250 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.250+0000 7f1ed7200640 1 -- start start 2024-05-09T04:35:49.251 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.251+0000 7f1ed7200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f1ed815bec0 0x7f1ed815f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:49.251 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.251+0000 7f1ed7200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ed815fde0 0x7f1ed80cfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:49.251 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.251+0000 7f1ed7200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ed8160280 con 0x7f1ed815bae0 2024-05-09T04:35:49.251 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.251+0000 7f1ed7200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f1ed8160840 con 0x7f1ed815bec0 2024-05-09T04:35:49.252 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.251+0000 7f1ed7200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f1ed80b4190 con 0x7f1ed815fde0 2024-05-09T04:35:49.252 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.251+0000 7f1ed5e00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f1ed815bae0 0x7f1ed80721b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:52062/0 (socket says 172.21.15.145:52062) 2024-05-09T04:35:49.252 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.252+0000 7f1ed5e00640 1 -- 172.21.15.145:0/1994085632 learned_addr learned my addr 172.21.15.145:0/1994085632 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:35:49.252 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.252+0000 7f1ecd400640 1 --2- 172.21.15.145:0/1994085632 >> v2:172.21.15.145:3300/0 conn(0x7f1ed815bec0 0x7f1ed815f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:49.253 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.252+0000 7f1ed6800640 1 --2- 172.21.15.145:0/1994085632 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ed815fde0 0x7f1ed80cfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:49.253 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.252+0000 7f1ed5400640 1 -- 172.21.15.145:0/1994085632 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 481650004 0 0) 0x7f1ed8160280 con 0x7f1ed815bae0 2024-05-09T04:35:49.254 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.252+0000 7f1ed5400640 1 -- 172.21.15.145:0/1994085632 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ebc003520 con 0x7f1ed815bae0 2024-05-09T04:35:49.254 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.252+0000 7f1ecd400640 1 -- 172.21.15.145:0/1994085632 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ed815fde0 msgr2=0x7f1ed80cfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:49.254 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.252+0000 7f1ecd400640 1 --2- 172.21.15.145:0/1994085632 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ed815fde0 0x7f1ed80cfe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:49.254 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.252+0000 7f1ecd400640 1 -- 172.21.15.145:0/1994085632 >> v1:172.21.15.145:6789/0 conn(0x7f1ed815bae0 legacy=0x7f1ed80721b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:49.254 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.252+0000 7f1ecd400640 1 -- 172.21.15.145:0/1994085632 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ed8160560 con 0x7f1ed815bec0 2024-05-09T04:35:49.254 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.253+0000 7f1ecd400640 1 --2- 172.21.15.145:0/1994085632 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1ed815bec0 0x7f1ed815f8a0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f1ec00097a0 tx=0x7f1ec002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cb1c7e3a4af8e1f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:49.254 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.254+0000 7f1ed5400640 1 -- 172.21.15.145:0/1994085632 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec0037070 con 0x7f1ed815bec0 2024-05-09T04:35:49.254 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.254+0000 7f1ed5400640 1 -- 172.21.15.145:0/1994085632 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ec002ae50 con 0x7f1ed815bec0 2024-05-09T04:35:49.255 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.254+0000 7f1ed5400640 1 -- 172.21.15.145:0/1994085632 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec0032640 con 0x7f1ed815bec0 2024-05-09T04:35:49.256 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.254+0000 7f1ed7200640 1 -- 172.21.15.145:0/1994085632 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1ed815bec0 msgr2=0x7f1ed815f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:49.256 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.254+0000 7f1ed7200640 1 --2- 172.21.15.145:0/1994085632 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1ed815bec0 0x7f1ed815f8a0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f1ec00097a0 tx=0x7f1ec002a100 comp rx=0 tx=0).stop 2024-05-09T04:35:49.256 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.254+0000 7f1ed7200640 1 -- 172.21.15.145:0/1994085632 shutdown_connections 2024-05-09T04:35:49.256 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.254+0000 7f1ed7200640 1 --2- 172.21.15.145:0/1994085632 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ed815fde0 0x7f1ed80cfe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:49.256 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.254+0000 7f1ed7200640 1 --2- 172.21.15.145:0/1994085632 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1ed815bec0 0x7f1ed815f8a0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:49.256 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.254+0000 7f1ed7200640 1 -- 172.21.15.145:0/1994085632 >> 172.21.15.145:0/1994085632 conn(0x7f1ed806c8a0 msgr2=0x7f1ed806ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:49.257 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.254+0000 7f1ed7200640 1 -- 172.21.15.145:0/1994085632 shutdown_connections 2024-05-09T04:35:49.257 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.254+0000 7f1ed7200640 1 -- 172.21.15.145:0/1994085632 wait complete. 2024-05-09T04:35:49.257 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.256+0000 7f1ed7200640 1 Processor -- start 2024-05-09T04:35:49.257 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.256+0000 7f1ed7200640 1 -- start start 2024-05-09T04:35:49.259 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.257+0000 7f1ed7200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ed815bae0 0x7f1ed814f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:49.259 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.257+0000 7f1ed7200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ed815bec0 0x7f1ed8159e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:49.259 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.257+0000 7f1ed7200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1ed815fde0 0x7f1ed815a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:49.259 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.257+0000 7f1ed7200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed80ae050 con 0x7f1ed815fde0 2024-05-09T04:35:49.259 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.257+0000 7f1ed7200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed80aded0 con 0x7f1ed815bae0 2024-05-09T04:35:49.260 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.257+0000 7f1ed7200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f1ed80d0370 con 0x7f1ed815bec0 2024-05-09T04:35:49.260 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.258+0000 7f1ed6800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1ed815fde0 0x7f1ed815a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:49.260 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.258+0000 7f1ed6800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1ed815fde0 0x7f1ed815a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:42444/0 (socket says 172.21.15.145:42444) 2024-05-09T04:35:49.260 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.258+0000 7f1ed6800640 1 -- 172.21.15.145:0/252901908 learned_addr learned my addr 172.21.15.145:0/252901908 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:49.260 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.258+0000 7f1ed5e00640 1 --2- 172.21.15.145:0/252901908 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ed815bae0 0x7f1ed814f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:49.260 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.258+0000 7f1ecd400640 1 --2- 172.21.15.145:0/252901908 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ed815bec0 0x7f1ed8159e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:49.260 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.258+0000 7f1ed6800640 1 -- 172.21.15.145:0/252901908 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ed815bec0 msgr2=0x7f1ed8159e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:49.260 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.258+0000 7f1ed6800640 1 --2- 172.21.15.145:0/252901908 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ed815bec0 0x7f1ed8159e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:49.261 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.258+0000 7f1ed6800640 1 -- 172.21.15.145:0/252901908 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ed815bae0 msgr2=0x7f1ed814f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:49.261 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.258+0000 7f1ed6800640 1 --2- 172.21.15.145:0/252901908 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ed815bae0 0x7f1ed814f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:49.261 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.258+0000 7f1ed6800640 1 -- 172.21.15.145:0/252901908 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ed8157d90 con 0x7f1ed815fde0 2024-05-09T04:35:49.261 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.258+0000 7f1ed5e00640 1 --2- 172.21.15.145:0/252901908 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ed815bae0 0x7f1ed814f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:35:49.261 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.258+0000 7f1ecd400640 1 --2- 172.21.15.145:0/252901908 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ed815bec0 0x7f1ed8159e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:35:49.261 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.258+0000 7f1ed6800640 1 --2- 172.21.15.145:0/252901908 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1ed815fde0 0x7f1ed815a370 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0007e60 tx=0x7f1ed000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:49.261 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.259+0000 7f1ecea00640 1 -- 172.21.15.145:0/252901908 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed0014420 con 0x7f1ed815fde0 2024-05-09T04:35:49.261 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.259+0000 7f1ecea00640 1 -- 172.21.15.145:0/252901908 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ed0019070 con 0x7f1ed815fde0 2024-05-09T04:35:49.261 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.259+0000 7f1ecea00640 1 -- 172.21.15.145:0/252901908 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed0014820 con 0x7f1ed815fde0 2024-05-09T04:35:49.262 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.259+0000 7f1ed7200640 1 -- 172.21.15.145:0/252901908 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ed815dcb0 con 0x7f1ed815fde0 2024-05-09T04:35:49.262 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.259+0000 7f1ed7200640 1 -- 172.21.15.145:0/252901908 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ed8158310 con 0x7f1ed815fde0 2024-05-09T04:35:49.263 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.260+0000 7f1ed7200640 1 -- 172.21.15.145:0/252901908 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ed806a310 con 0x7f1ed815fde0 2024-05-09T04:35:49.263 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.261+0000 7f1ecea00640 1 -- 172.21.15.145:0/252901908 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1ed00149c0 con 0x7f1ed815fde0 2024-05-09T04:35:49.263 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.262+0000 7f1ecea00640 1 --2- 172.21.15.145:0/252901908 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f1eb40762d0 0x7f1eb4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:49.263 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.262+0000 7f1ecea00640 1 -- 172.21.15.145:0/252901908 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7f1ed0097bf0 con 0x7f1ed815fde0 2024-05-09T04:35:49.263 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.262+0000 7f1ed5e00640 1 --2- 172.21.15.145:0/252901908 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f1eb40762d0 0x7f1eb4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:49.263 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.263+0000 7f1ed5e00640 1 --2- 172.21.15.145:0/252901908 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f1eb40762d0 0x7f1eb4078790 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8001220 tx=0x7f1ec8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:49.266 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.266+0000 7f1ecea00640 1 -- 172.21.15.145:0/252901908 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f1ed00617b0 con 0x7f1ed815fde0 2024-05-09T04:35:49.475 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.475+0000 7f1ed7200640 1 -- 172.21.15.145:0/252901908 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1ed806faa0 con 0x7f1ed815fde0 2024-05-09T04:35:49.476 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.477+0000 7f1ecea00640 1 -- 172.21.15.145:0/252901908 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) ==== 74+0+13070 (secure 0 0 0) 0x7f1ed00610b0 con 0x7f1ed815fde0 2024-05-09T04:35:49.477 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:35:49.477 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":55,"fsid":"957ee6c1-073b-44cf-9b93-9491b3fdfc1d","created":"2024-05-09T04:33:26.544367+0000","modified":"2024-05-09T04:35:45.422652+0000","last_up_change":"2024-05-09T04:33:39.039318+0000","last_in_change":"2024-05-09T04:33:29.902361+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T04:33:39.795344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22e550ad-847b-4952-80e6-63d859ed1a0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6801","nonce":3182572912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6803","nonce":3182572912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6807","nonce":3182572912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":3182572912},{"type":"v1","addr":"172.21.15.145:6805","nonce":3182572912}]},"public_addr":"172.21.15.145:6801/3182572912","cluster_addr":"172.21.15.145:6803/3182572912","heartbeat_back_addr":"172.21.15.145:6807/3182572912","heartbeat_front_addr":"172.21.15.145:6805/3182572912","state":["exists","up"]},{"osd":1,"uuid":"0f6b9fca-0c41-4145-8d74-598ddb42f2a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6825","nonce":1845225555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6827","nonce":1845225555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6831","nonce":1845225555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":1845225555},{"type":"v1","addr":"172.21.15.145:6829","nonce":1845225555}]},"public_addr":"172.21.15.145:6825/1845225555","cluster_addr":"172.21.15.145:6827/1845225555","heartbeat_back_addr":"172.21.15.145:6831/1845225555","heartbeat_front_addr":"172.21.15.145:6829/1845225555","state":["exists","up"]},{"osd":2,"uuid":"b3b1f67e-5790-483f-8610-e05a5bce4c30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6816","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6817","nonce":2168958993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6819","nonce":2168958993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6823","nonce":2168958993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":2168958993},{"type":"v1","addr":"172.21.15.145:6821","nonce":2168958993}]},"public_addr":"172.21.15.145:6817/2168958993","cluster_addr":"172.21.15.145:6819/2168958993","heartbeat_back_addr":"172.21.15.145:6823/2168958993","heartbeat_front_addr":"172.21.15.145:6821/2168958993","state":["exists","up"]},{"osd":3,"uuid":"884787a5-e037-4d31-9aae-743030eba4ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6809","nonce":337009826}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6811","nonce":337009826}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6815","nonce":337009826}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":337009826},{"type":"v1","addr":"172.21.15.145:6813","nonce":337009826}]},"public_addr":"172.21.15.145:6809/337009826","cluster_addr":"172.21.15.145:6811/337009826","heartbeat_back_addr":"172.21.15.145:6815/337009826","heartbeat_front_addr":"172.21.15.145:6813/337009826","state":["exists","up"]},{"osd":4,"uuid":"0759fc1e-c5fc-4a2f-86bb-e8eccd83f833","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6818","nonce":3117700995}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6819","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6821","nonce":3117700995}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6827","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6829","nonce":3117700995}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6823","nonce":3117700995},{"type":"v1","addr":"172.21.15.192:6825","nonce":3117700995}]},"public_addr":"172.21.15.192:6818/3117700995","cluster_addr":"172.21.15.192:6821/3117700995","heartbeat_back_addr":"172.21.15.192:6829/3117700995","heartbeat_front_addr":"172.21.15.192:6825/3117700995","state":["exists","up"]},{"osd":5,"uuid":"9443cc22-9379-46e3-9173-98b592bf2c16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6801","nonce":1328373268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6803","nonce":1328373268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6807","nonce":1328373268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1328373268},{"type":"v1","addr":"172.21.15.192:6805","nonce":1328373268}]},"public_addr":"172.21.15.192:6801/1328373268","cluster_addr":"172.21.15.192:6803/1328373268","heartbeat_back_addr":"172.21.15.192:6807/1328373268","heartbeat_front_addr":"172.21.15.192:6805/1328373268","state":["exists","up"]},{"osd":6,"uuid":"d05a3311-505a-45b5-85b9-aa3113e96987","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6817","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6820","nonce":1868011057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6824","nonce":1868011057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6831","nonce":1868011057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1868011057},{"type":"v1","addr":"172.21.15.192:6828","nonce":1868011057}]},"public_addr":"172.21.15.192:6820/1868011057","cluster_addr":"172.21.15.192:6824/1868011057","heartbeat_back_addr":"172.21.15.192:6831/1868011057","heartbeat_front_addr":"172.21.15.192:6828/1868011057","state":["exists","up"]},{"osd":7,"uuid":"c7c23dbc-2029-43ec-9f53-25f3d58019a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6809","nonce":3368512157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6811","nonce":3368512157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6815","nonce":3368512157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3368512157},{"type":"v1","addr":"172.21.15.192:6813","nonce":3368512157}]},"public_addr":"172.21.15.192:6809/3368512157","cluster_addr":"172.21.15.192:6811/3368512157","heartbeat_back_addr":"172.21.15.192:6815/3368512157","heartbeat_front_addr":"172.21.15.192:6813/3368512157","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.091188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.032567+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:34.797468+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.228839+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.622400+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.473103+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.566092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:33:35.349395+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.145:6837/2593908438":"2024-05-10T04:35:40.966317+0000","172.21.15.145:6835/2593908438":"2024-05-10T04:35:40.966317+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T04:35:49.479 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.479+0000 7f1ed7200640 1 -- 172.21.15.145:0/252901908 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f1eb40762d0 msgr2=0x7f1eb4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:49.479 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.479+0000 7f1ed7200640 1 --2- 172.21.15.145:0/252901908 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f1eb40762d0 0x7f1eb4078790 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8001220 tx=0x7f1ec8009040 comp rx=0 tx=0).stop 2024-05-09T04:35:49.480 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.479+0000 7f1ed7200640 1 -- 172.21.15.145:0/252901908 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1ed815fde0 msgr2=0x7f1ed815a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:49.480 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.480+0000 7f1ed7200640 1 --2- 172.21.15.145:0/252901908 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1ed815fde0 0x7f1ed815a370 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0007e60 tx=0x7f1ed000e4b0 comp rx=0 tx=0).stop 2024-05-09T04:35:49.480 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.480+0000 7f1ed7200640 1 -- 172.21.15.145:0/252901908 shutdown_connections 2024-05-09T04:35:49.480 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.480+0000 7f1ed7200640 1 --2- 172.21.15.145:0/252901908 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f1eb40762d0 0x7f1eb4078790 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:49.481 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.480+0000 7f1ed7200640 1 --2- 172.21.15.145:0/252901908 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1ed815fde0 0x7f1ed815a370 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:49.481 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.480+0000 7f1ed7200640 1 --2- 172.21.15.145:0/252901908 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ed815bec0 0x7f1ed8159e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:49.481 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.480+0000 7f1ed7200640 1 --2- 172.21.15.145:0/252901908 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ed815bae0 0x7f1ed814f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:49.481 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.480+0000 7f1ed7200640 1 -- 172.21.15.145:0/252901908 >> 172.21.15.145:0/252901908 conn(0x7f1ed806c8a0 msgr2=0x7f1ed8072930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:49.481 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.480+0000 7f1ed7200640 1 -- 172.21.15.145:0/252901908 shutdown_connections 2024-05-09T04:35:49.481 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:49.480+0000 7f1ed7200640 1 -- 172.21.15.145:0/252901908 wait complete. 2024-05-09T04:35:50.500 INFO:tasks.ceph:Scrubbing osd.0 2024-05-09T04:35:50.500 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-05-09T04:35:50.714 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.714+0000 7fe71ec00640 1 Processor -- start 2024-05-09T04:35:50.714 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.714+0000 7fe71ec00640 1 -- start start 2024-05-09T04:35:50.716 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.716+0000 7fe71ec00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe718146970 0x7fe7180cea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:50.716 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.716+0000 7fe71ec00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe71815dc10 con 0x7fe7180cdc60 2024-05-09T04:35:50.716 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.716+0000 7fe71ec00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe71815def0 con 0x7fe71806e060 2024-05-09T04:35:50.716 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.716+0000 7fe71ec00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe718072ee0 con 0x7fe718146970 2024-05-09T04:35:50.718 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.716+0000 7fe71e200640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7fe7180cdc60 0x7fe718073370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:52068/0 (socket says 172.21.15.145:52068) 2024-05-09T04:35:50.718 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.716+0000 7fe71e200640 1 -- 172.21.15.145:0/2873417580 learned_addr learned my addr 172.21.15.145:0/2873417580 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:35:50.718 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.717+0000 7fe71d800640 1 --2- 172.21.15.145:0/2873417580 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe718146970 0x7fe7180cea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:50.719 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.717+0000 7fe717e00640 1 -- 172.21.15.145:0/2873417580 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3607508223 0 0) 0x7fe71815dc10 con 0x7fe7180cdc60 2024-05-09T04:35:50.719 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.717+0000 7fe717e00640 1 -- 172.21.15.145:0/2873417580 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe700003520 con 0x7fe7180cdc60 2024-05-09T04:35:50.719 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.717+0000 7fe717e00640 1 -- 172.21.15.145:0/2873417580 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3860452116 0 0) 0x7fe700003520 con 0x7fe7180cdc60 2024-05-09T04:35:50.719 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.717+0000 7fe717e00640 1 -- 172.21.15.145:0/2873417580 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe718146970 msgr2=0x7fe7180cea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:50.719 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.717+0000 7fe717e00640 1 --2- 172.21.15.145:0/2873417580 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe718146970 0x7fe7180cea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:50.719 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.717+0000 7fe717e00640 1 -- 172.21.15.145:0/2873417580 >> v1:172.21.15.192:6789/0 conn(0x7fe71806e060 legacy=0x7fe7180cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:50.719 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.717+0000 7fe717e00640 1 -- 172.21.15.145:0/2873417580 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe71815e1d0 con 0x7fe7180cdc60 2024-05-09T04:35:50.719 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.717+0000 7fe717e00640 1 -- 172.21.15.145:0/2873417580 <== mon.0 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3474413261 0 0) 0x7fe710001a70 con 0x7fe7180cdc60 2024-05-09T04:35:50.720 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.717+0000 7fe717e00640 1 -- 172.21.15.145:0/2873417580 >> v1:172.21.15.145:6789/0 conn(0x7fe7180cdc60 legacy=0x7fe718073370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:50.720 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.717+0000 7fe717e00640 1 --2- 172.21.15.145:0/2873417580 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe718146970 0x7fe700003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:50.720 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.718+0000 7fe717e00640 1 --2- 172.21.15.145:0/2873417580 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe700004000 0x7fe700004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:50.720 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.718+0000 7fe717e00640 1 --2- 172.21.15.145:0/2873417580 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe700004d40 0x7fe7000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:50.720 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.718+0000 7fe717e00640 1 -- 172.21.15.145:0/2873417580 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe718072ee0 con 0x7fe700004000 2024-05-09T04:35:50.720 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.718+0000 7fe717e00640 1 -- 172.21.15.145:0/2873417580 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe70000bb50 con 0x7fe700004d40 2024-05-09T04:35:50.720 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.718+0000 7fe717e00640 1 -- 172.21.15.145:0/2873417580 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe70000bd50 con 0x7fe718146970 2024-05-09T04:35:50.720 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.718+0000 7fe71ce00640 1 --2- 172.21.15.145:0/2873417580 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe700004000 0x7fe700004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:50.721 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.718+0000 7fe71d800640 1 --2- 172.21.15.145:0/2873417580 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe718146970 0x7fe700003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:50.721 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.718+0000 7fe71e200640 1 --2- 172.21.15.145:0/2873417580 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe700004d40 0x7fe7000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:50.721 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.718+0000 7fe71ce00640 1 -- 172.21.15.145:0/2873417580 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe718146970 msgr2=0x7fe700003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:50.721 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.718+0000 7fe71ce00640 1 --2- 172.21.15.145:0/2873417580 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe718146970 0x7fe700003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:50.721 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.718+0000 7fe71ce00640 1 -- 172.21.15.145:0/2873417580 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe700004d40 msgr2=0x7fe7000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:50.721 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.718+0000 7fe71ce00640 1 --2- 172.21.15.145:0/2873417580 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe700004d40 0x7fe7000051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:50.721 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.718+0000 7fe71ce00640 1 -- 172.21.15.145:0/2873417580 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe700003520 con 0x7fe700004000 2024-05-09T04:35:50.721 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.718+0000 7fe71d800640 1 --2- 172.21.15.145:0/2873417580 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe718146970 0x7fe700003a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:35:50.721 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.718+0000 7fe71e200640 1 --2- 172.21.15.145:0/2873417580 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe700004d40 0x7fe7000051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:35:50.723 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.719+0000 7fe71ce00640 1 --2- 172.21.15.145:0/2873417580 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe700004000 0x7fe700004800 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fe704001790 tx=0x7fe700005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c775037c88b3da2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:50.724 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.720+0000 7fe717e00640 1 -- 172.21.15.145:0/2873417580 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7040013b0 con 0x7fe700004000 2024-05-09T04:35:50.724 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.720+0000 7fe717e00640 1 -- 172.21.15.145:0/2873417580 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7040052d0 con 0x7fe700004000 2024-05-09T04:35:50.724 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.720+0000 7fe717e00640 1 -- 172.21.15.145:0/2873417580 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70402aae0 con 0x7fe700004000 2024-05-09T04:35:50.724 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.720+0000 7fe71ec00640 1 -- 172.21.15.145:0/2873417580 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe700004000 msgr2=0x7fe700004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:50.724 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.720+0000 7fe71ec00640 1 --2- 172.21.15.145:0/2873417580 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe700004000 0x7fe700004800 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fe704001790 tx=0x7fe700005830 comp rx=0 tx=0).stop 2024-05-09T04:35:50.724 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.720+0000 7fe71e200640 1 -- 172.21.15.145:0/2873417580 reap_dead start 2024-05-09T04:35:50.724 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.720+0000 7fe71ec00640 1 -- 172.21.15.145:0/2873417580 shutdown_connections 2024-05-09T04:35:50.724 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.720+0000 7fe71ec00640 1 -- 172.21.15.145:0/2873417580 >> 172.21.15.145:0/2873417580 conn(0x7fe7180b2850 msgr2=0x7fe7180b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:50.725 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.720+0000 7fe71ec00640 1 -- 172.21.15.145:0/2873417580 shutdown_connections 2024-05-09T04:35:50.725 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.720+0000 7fe71ec00640 1 -- 172.21.15.145:0/2873417580 wait complete. 2024-05-09T04:35:50.725 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.722+0000 7fe71ec00640 1 Processor -- start 2024-05-09T04:35:50.725 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.722+0000 7fe71ec00640 1 -- start start 2024-05-09T04:35:50.725 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.723+0000 7fe71ec00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe700004000 0x7fe718139300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:50.725 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.723+0000 7fe71ec00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe71815a720 0x7fe7180b3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:50.725 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.723+0000 7fe71ec00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7180b4190 0x7fe71813bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:50.725 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.723+0000 7fe71ec00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe7180ce7c0 con 0x7fe71815a720 2024-05-09T04:35:50.725 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.723+0000 7fe71ec00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe7180ce640 con 0x7fe7180b4190 2024-05-09T04:35:50.726 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.723+0000 7fe71ec00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe7180cea40 con 0x7fe700004000 2024-05-09T04:35:50.726 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.724+0000 7fe71ce00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe71815a720 0x7fe7180b3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:50.726 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.724+0000 7fe71ce00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe71815a720 0x7fe7180b3c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:42458/0 (socket says 172.21.15.145:42458) 2024-05-09T04:35:50.726 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.724+0000 7fe71ce00640 1 -- 172.21.15.145:0/53932163 learned_addr learned my addr 172.21.15.145:0/53932163 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:50.726 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.724+0000 7fe71d800640 1 --2- 172.21.15.145:0/53932163 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe700004000 0x7fe718139300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:50.726 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.724+0000 7fe71e200640 1 --2- 172.21.15.145:0/53932163 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7180b4190 0x7fe71813bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:50.727 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.724+0000 7fe71ce00640 1 -- 172.21.15.145:0/53932163 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe700004000 msgr2=0x7fe718139300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:50.728 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.724+0000 7fe71ce00640 1 --2- 172.21.15.145:0/53932163 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe700004000 0x7fe718139300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:50.728 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.724+0000 7fe71ce00640 1 -- 172.21.15.145:0/53932163 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7180b4190 msgr2=0x7fe71813bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:50.728 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.724+0000 7fe71ce00640 1 --2- 172.21.15.145:0/53932163 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7180b4190 0x7fe71813bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:50.728 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.724+0000 7fe71ce00640 1 -- 172.21.15.145:0/53932163 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe71813ad90 con 0x7fe71815a720 2024-05-09T04:35:50.728 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.725+0000 7fe71ce00640 1 --2- 172.21.15.145:0/53932163 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe71815a720 0x7fe7180b3c50 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fe704005040 tx=0x7fe704001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:50.728 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.725+0000 7fe715600640 1 -- 172.21.15.145:0/53932163 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe704037040 con 0x7fe71815a720 2024-05-09T04:35:50.729 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.725+0000 7fe715600640 1 -- 172.21.15.145:0/53932163 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe704005480 con 0x7fe71815a720 2024-05-09T04:35:50.729 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.725+0000 7fe71ec00640 1 -- 172.21.15.145:0/53932163 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7181f93d0 con 0x7fe71815a720 2024-05-09T04:35:50.729 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.725+0000 7fe715600640 1 -- 172.21.15.145:0/53932163 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7040013b0 con 0x7fe71815a720 2024-05-09T04:35:50.729 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.725+0000 7fe71ec00640 1 -- 172.21.15.145:0/53932163 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe71813b2e0 con 0x7fe71815a720 2024-05-09T04:35:50.729 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.726+0000 7fe71ec00640 1 -- 172.21.15.145:0/53932163 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe718076930 con 0x7fe71815a720 2024-05-09T04:35:50.729 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.728+0000 7fe715600640 1 -- 172.21.15.145:0/53932163 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe704044020 con 0x7fe71815a720 2024-05-09T04:35:50.730 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.728+0000 7fe715600640 1 --2- 172.21.15.145:0/53932163 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fe6f8076300 0x7fe6f80787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:50.731 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.729+0000 7fe715600640 1 -- 172.21.15.145:0/53932163 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7fe7040b6bf0 con 0x7fe71815a720 2024-05-09T04:35:50.731 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.729+0000 7fe71d800640 1 --2- 172.21.15.145:0/53932163 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fe6f8076300 0x7fe6f80787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:50.731 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.729+0000 7fe715600640 1 --2- 172.21.15.145:0/53932163 >> [v2:172.21.15.145:6800/3182572912,v1:172.21.15.145:6801/3182572912] conn(0x7fe6f807c1c0 0x7fe6f807e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:50.731 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.729+0000 7fe715600640 1 -- 172.21.15.145:0/53932163 --> [v2:172.21.15.145:6800/3182572912,v1:172.21.15.145:6801/3182572912] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe7040052d0 con 0x7fe6f807c1c0 2024-05-09T04:35:50.731 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.729+0000 7fe715600640 1 -- 172.21.15.145:0/53932163 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fe7040b9270 con 0x7fe71815a720 2024-05-09T04:35:50.731 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.729+0000 7fe71e200640 1 --2- 172.21.15.145:0/53932163 >> [v2:172.21.15.145:6800/3182572912,v1:172.21.15.145:6801/3182572912] conn(0x7fe6f807c1c0 0x7fe6f807e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:50.731 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.729+0000 7fe71d800640 1 --2- 172.21.15.145:0/53932163 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fe6f8076300 0x7fe6f80787c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe718145f10 tx=0x7fe70c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:50.732 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.730+0000 7fe71e200640 1 --2- 172.21.15.145:0/53932163 >> [v2:172.21.15.145:6800/3182572912,v1:172.21.15.145:6801/3182572912] conn(0x7fe6f807c1c0 0x7fe6f807e5e0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:50.732 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.730+0000 7fe715600640 1 -- 172.21.15.145:0/53932163 <== osd.0 v2:172.21.15.145:6800/3182572912 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe7040052d0 con 0x7fe6f807c1c0 2024-05-09T04:35:50.745 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.745+0000 7fe71ec00640 1 -- 172.21.15.145:0/53932163 --> [v2:172.21.15.145:6800/3182572912,v1:172.21.15.145:6801/3182572912] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fe7180d0820 con 0x7fe6f807c1c0 2024-05-09T04:35:50.745 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.745+0000 7fe715600640 1 -- 172.21.15.145:0/53932163 <== osd.0 v2:172.21.15.145:6800/3182572912 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fe7180d0820 con 0x7fe6f807c1c0 2024-05-09T04:35:50.746 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-05-09T04:35:50.746 INFO:teuthology.orchestra.run.smithi145.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-09T04:35:50.746 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-05-09T04:35:50.747 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.746+0000 7fe71ec00640 1 -- 172.21.15.145:0/53932163 >> [v2:172.21.15.145:6800/3182572912,v1:172.21.15.145:6801/3182572912] conn(0x7fe6f807c1c0 msgr2=0x7fe6f807e5e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:50.747 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.746+0000 7fe71ec00640 1 --2- 172.21.15.145:0/53932163 >> [v2:172.21.15.145:6800/3182572912,v1:172.21.15.145:6801/3182572912] conn(0x7fe6f807c1c0 0x7fe6f807e5e0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:50.747 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.746+0000 7fe71ec00640 1 -- 172.21.15.145:0/53932163 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fe6f8076300 msgr2=0x7fe6f80787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:50.747 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.746+0000 7fe71ec00640 1 --2- 172.21.15.145:0/53932163 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fe6f8076300 0x7fe6f80787c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe718145f10 tx=0x7fe70c008040 comp rx=0 tx=0).stop 2024-05-09T04:35:50.747 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.746+0000 7fe71ec00640 1 -- 172.21.15.145:0/53932163 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe71815a720 msgr2=0x7fe7180b3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:50.748 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.746+0000 7fe71ec00640 1 --2- 172.21.15.145:0/53932163 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe71815a720 0x7fe7180b3c50 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fe704005040 tx=0x7fe704001120 comp rx=0 tx=0).stop 2024-05-09T04:35:50.748 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.746+0000 7fe71e200640 1 -- 172.21.15.145:0/53932163 reap_dead start 2024-05-09T04:35:50.748 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.746+0000 7fe71ec00640 1 -- 172.21.15.145:0/53932163 shutdown_connections 2024-05-09T04:35:50.748 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.746+0000 7fe71ec00640 1 -- 172.21.15.145:0/53932163 >> 172.21.15.145:0/53932163 conn(0x7fe7180b2850 msgr2=0x7fe7180b2010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:50.748 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.746+0000 7fe71ec00640 1 -- 172.21.15.145:0/53932163 shutdown_connections 2024-05-09T04:35:50.748 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.747+0000 7fe71ec00640 1 -- 172.21.15.145:0/53932163 wait complete. 2024-05-09T04:35:50.763 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-05-09T04:35:50.956 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.956+0000 7fc04fe00640 1 Processor -- start 2024-05-09T04:35:50.956 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.956+0000 7fc04fe00640 1 -- start start 2024-05-09T04:35:50.957 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.958+0000 7fc04fe00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc05006c8a0 0x7fc05015f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:50.957 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.958+0000 7fc04fe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc050160a00 0x7fc05006cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:50.958 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.958+0000 7fc04fe00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc05006fe10 con 0x7fc0500d03c0 2024-05-09T04:35:50.958 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.958+0000 7fc04fe00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fc050160e80 con 0x7fc05006c8a0 2024-05-09T04:35:50.958 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.958+0000 7fc04fe00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc05006d1c0 con 0x7fc050160a00 2024-05-09T04:35:50.959 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.958+0000 7fc04ea00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc05006c8a0 0x7fc05015f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:50.959 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.959+0000 7fc04ea00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc05006c8a0 0x7fc05015f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.145:33322/0 (socket says 172.21.15.145:33322) 2024-05-09T04:35:50.959 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.959+0000 7fc04ea00640 1 -- 172.21.15.145:0/1040911826 learned_addr learned my addr 172.21.15.145:0/1040911826 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:50.959 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.959+0000 7fc04e000640 1 --2- 172.21.15.145:0/1040911826 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc050160a00 0x7fc05006cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:50.959 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.959+0000 7fc04ea00640 1 -- 172.21.15.145:0/1040911826 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc050160a00 msgr2=0x7fc05006cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:50.960 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.959+0000 7fc04ea00640 1 --2- 172.21.15.145:0/1040911826 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc050160a00 0x7fc05006cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:50.960 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.959+0000 7fc04ea00640 1 -- 172.21.15.145:0/1040911826 >> v1:172.21.15.192:6789/0 conn(0x7fc0500d03c0 legacy=0x7fc050073760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:50.960 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.959+0000 7fc04ea00640 1 -- 172.21.15.145:0/1040911826 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc05015fe20 con 0x7fc05006c8a0 2024-05-09T04:35:50.960 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.959+0000 7fc04e000640 1 --2- 172.21.15.145:0/1040911826 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc050160a00 0x7fc05006cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:35:50.960 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.961+0000 7fc04ea00640 1 --2- 172.21.15.145:0/1040911826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc05006c8a0 0x7fc05015f7e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc040009870 tx=0x7fc04002a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc87b417f0662c1b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:50.961 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.961+0000 7fc04d600640 1 -- 172.21.15.145:0/1040911826 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc040037070 con 0x7fc05006c8a0 2024-05-09T04:35:50.961 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.961+0000 7fc04d600640 1 -- 172.21.15.145:0/1040911826 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc040032440 con 0x7fc05006c8a0 2024-05-09T04:35:50.961 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.961+0000 7fc04d600640 1 -- 172.21.15.145:0/1040911826 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc040032810 con 0x7fc05006c8a0 2024-05-09T04:35:50.962 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.962+0000 7fc04fe00640 1 -- 172.21.15.145:0/1040911826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc05006c8a0 msgr2=0x7fc05015f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:50.962 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.962+0000 7fc04fe00640 1 --2- 172.21.15.145:0/1040911826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc05006c8a0 0x7fc05015f7e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc040009870 tx=0x7fc04002a250 comp rx=0 tx=0).stop 2024-05-09T04:35:50.962 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.962+0000 7fc04fe00640 1 -- 172.21.15.145:0/1040911826 shutdown_connections 2024-05-09T04:35:50.962 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.962+0000 7fc04fe00640 1 --2- 172.21.15.145:0/1040911826 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc050160a00 0x7fc05006cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:50.962 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.962+0000 7fc04fe00640 1 --2- 172.21.15.145:0/1040911826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc05006c8a0 0x7fc05015f7e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:50.962 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.962+0000 7fc04fe00640 1 -- 172.21.15.145:0/1040911826 >> 172.21.15.145:0/1040911826 conn(0x7fc0500b2d80 msgr2=0x7fc0500b3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:50.962 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.962+0000 7fc04fe00640 1 -- 172.21.15.145:0/1040911826 shutdown_connections 2024-05-09T04:35:50.962 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.962+0000 7fc04fe00640 1 -- 172.21.15.145:0/1040911826 wait complete. 2024-05-09T04:35:50.963 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.964+0000 7fc04fe00640 1 Processor -- start 2024-05-09T04:35:50.963 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.964+0000 7fc04fe00640 1 -- start start 2024-05-09T04:35:50.965 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.965+0000 7fc04fe00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc05006c8a0 0x7fc05009c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:50.965 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.965+0000 7fc04fe00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0500d03c0 0x7fc05014ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:50.966 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.965+0000 7fc04fe00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc050160a00 0x7fc0501e7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:50.966 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.965+0000 7fc04fe00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fc0501606c0 con 0x7fc0500d03c0 2024-05-09T04:35:50.966 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.965+0000 7fc04fe00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fc0501603e0 con 0x7fc05006c8a0 2024-05-09T04:35:50.966 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.965+0000 7fc04fe00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fc0500d0850 con 0x7fc050160a00 2024-05-09T04:35:50.966 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.965+0000 7fc04e000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0500d03c0 0x7fc05014ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:50.966 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.965+0000 7fc04e000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0500d03c0 0x7fc05014ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:52928/0 (socket says 172.21.15.145:52928) 2024-05-09T04:35:50.966 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.965+0000 7fc04e000640 1 -- 172.21.15.145:0/702808117 learned_addr learned my addr 172.21.15.145:0/702808117 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:50.966 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.965+0000 7fc04ea00640 1 --2- 172.21.15.145:0/702808117 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc05006c8a0 0x7fc05009c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:50.967 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.965+0000 7fc04e000640 1 -- 172.21.15.145:0/702808117 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc050160a00 msgr2=0x7fc0501e7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:50.967 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.966+0000 7fc04f400640 1 --2- 172.21.15.145:0/702808117 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc050160a00 0x7fc0501e7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:50.967 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.966+0000 7fc04e000640 1 --2- 172.21.15.145:0/702808117 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc050160a00 0x7fc0501e7ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:50.967 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.966+0000 7fc04e000640 1 -- 172.21.15.145:0/702808117 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc05006c8a0 msgr2=0x7fc05009c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:50.967 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.966+0000 7fc04e000640 1 --2- 172.21.15.145:0/702808117 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc05006c8a0 0x7fc05009c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:50.967 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.966+0000 7fc04e000640 1 -- 172.21.15.145:0/702808117 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0501e8010 con 0x7fc0500d03c0 2024-05-09T04:35:50.967 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.966+0000 7fc04e000640 1 --2- 172.21.15.145:0/702808117 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0500d03c0 0x7fc05014ca90 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fc03c00ec10 tx=0x7fc03c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:50.968 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.966+0000 7fc047400640 1 -- 172.21.15.145:0/702808117 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc03c017070 con 0x7fc0500d03c0 2024-05-09T04:35:50.968 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.966+0000 7fc047400640 1 -- 172.21.15.145:0/702808117 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc03c00cc30 con 0x7fc0500d03c0 2024-05-09T04:35:50.969 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.966+0000 7fc04fe00640 1 -- 172.21.15.145:0/702808117 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc050149e70 con 0x7fc0500d03c0 2024-05-09T04:35:50.969 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.966+0000 7fc047400640 1 -- 172.21.15.145:0/702808117 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc03c013470 con 0x7fc0500d03c0 2024-05-09T04:35:50.969 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.966+0000 7fc04fe00640 1 -- 172.21.15.145:0/702808117 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc05014a430 con 0x7fc0500d03c0 2024-05-09T04:35:50.969 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.967+0000 7fc04fe00640 1 -- 172.21.15.145:0/702808117 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc050076780 con 0x7fc0500d03c0 2024-05-09T04:35:50.969 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.969+0000 7fc047400640 1 -- 172.21.15.145:0/702808117 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fc03c01a070 con 0x7fc0500d03c0 2024-05-09T04:35:50.970 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.970+0000 7fc047400640 1 --2- 172.21.15.145:0/702808117 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fc0300762d0 0x7fc030078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:50.970 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.970+0000 7fc047400640 1 -- 172.21.15.145:0/702808117 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7fc03c0973a0 con 0x7fc0500d03c0 2024-05-09T04:35:50.970 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.970+0000 7fc04ea00640 1 --2- 172.21.15.145:0/702808117 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fc0300762d0 0x7fc030078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:50.970 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.971+0000 7fc04ea00640 1 --2- 172.21.15.145:0/702808117 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fc0300762d0 0x7fc030078790 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc040034000 tx=0x7fc04002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:50.974 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:50.975+0000 7fc047400640 1 -- 172.21.15.145:0/702808117 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc03c060e50 con 0x7fc0500d03c0 2024-05-09T04:35:51.183 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.183+0000 7fc04fe00640 1 -- 172.21.15.145:0/702808117 --> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7fc0500b3300 con 0x7fc0300762d0 2024-05-09T04:35:51.183 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.184+0000 7fc047400640 1 -- 172.21.15.145:0/702808117 <== mgr.4104 v2:172.21.15.145:6832/2725855396 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fc0500b3300 con 0x7fc0300762d0 2024-05-09T04:35:51.184 INFO:teuthology.orchestra.run.smithi145.stderr:instructed osd(s) 0 to deep-scrub 2024-05-09T04:35:51.187 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.186+0000 7fc04fe00640 1 -- 172.21.15.145:0/702808117 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fc0300762d0 msgr2=0x7fc030078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:51.187 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.186+0000 7fc04fe00640 1 --2- 172.21.15.145:0/702808117 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fc0300762d0 0x7fc030078790 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc040034000 tx=0x7fc04002e000 comp rx=0 tx=0).stop 2024-05-09T04:35:51.187 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.186+0000 7fc04fe00640 1 -- 172.21.15.145:0/702808117 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0500d03c0 msgr2=0x7fc05014ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:51.187 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.186+0000 7fc04fe00640 1 --2- 172.21.15.145:0/702808117 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0500d03c0 0x7fc05014ca90 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fc03c00ec10 tx=0x7fc03c00c520 comp rx=0 tx=0).stop 2024-05-09T04:35:51.187 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.187+0000 7fc04fe00640 1 -- 172.21.15.145:0/702808117 shutdown_connections 2024-05-09T04:35:51.187 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.187+0000 7fc04fe00640 1 --2- 172.21.15.145:0/702808117 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fc0300762d0 0x7fc030078790 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:51.187 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.187+0000 7fc04fe00640 1 --2- 172.21.15.145:0/702808117 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc050160a00 0x7fc0501e7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:51.188 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.187+0000 7fc04fe00640 1 --2- 172.21.15.145:0/702808117 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc0500d03c0 0x7fc05014ca90 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:51.188 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.187+0000 7fc04fe00640 1 --2- 172.21.15.145:0/702808117 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc05006c8a0 0x7fc05009c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:51.188 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.187+0000 7fc04fe00640 1 -- 172.21.15.145:0/702808117 >> 172.21.15.145:0/702808117 conn(0x7fc0500b2d80 msgr2=0x7fc0500b3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:51.188 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.187+0000 7fc04fe00640 1 -- 172.21.15.145:0/702808117 shutdown_connections 2024-05-09T04:35:51.188 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.187+0000 7fc04fe00640 1 -- 172.21.15.145:0/702808117 wait complete. 2024-05-09T04:35:51.205 INFO:tasks.ceph:Scrubbing osd.1 2024-05-09T04:35:51.205 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-05-09T04:35:51.409 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.409+0000 7f032f800640 1 Processor -- start 2024-05-09T04:35:51.409 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.409+0000 7f032f800640 1 -- start start 2024-05-09T04:35:51.410 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.410+0000 7f032f800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0328146a90 0x7f03280729b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:51.410 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.411+0000 7f032f800640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0328146e70 con 0x7f03280ce480 2024-05-09T04:35:51.411 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.411+0000 7f032f800640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0328160f80 con 0x7f0328072ef0 2024-05-09T04:35:51.411 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.411+0000 7f032f800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0328160d30 con 0x7f0328146a90 2024-05-09T04:35:51.411 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.411+0000 7f032d800640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f03280ce480 0x7f03280b46f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:60516/0 (socket says 172.21.15.145:60516) 2024-05-09T04:35:51.411 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.411+0000 7f032d800640 1 -- 172.21.15.145:0/3048590385 learned_addr learned my addr 172.21.15.145:0/3048590385 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:35:51.413 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.411+0000 7f032ce00640 1 --2- 172.21.15.145:0/3048590385 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0328146a90 0x7f03280729b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:51.413 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.411+0000 7f0327400640 1 -- 172.21.15.145:0/3048590385 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3785331394 0 0) 0x7f0328146e70 con 0x7f03280ce480 2024-05-09T04:35:51.414 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.412+0000 7f0327400640 1 -- 172.21.15.145:0/3048590385 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0310003520 con 0x7f03280ce480 2024-05-09T04:35:51.414 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.412+0000 7f0327400640 1 -- 172.21.15.145:0/3048590385 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3511688036 0 0) 0x7f0310003520 con 0x7f03280ce480 2024-05-09T04:35:51.414 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.412+0000 7f0327400640 1 -- 172.21.15.145:0/3048590385 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0328146a90 msgr2=0x7f03280729b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:51.414 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.412+0000 7f0327400640 1 --2- 172.21.15.145:0/3048590385 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0328146a90 0x7f03280729b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:51.414 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.412+0000 7f0327400640 1 -- 172.21.15.145:0/3048590385 >> v1:172.21.15.192:6789/0 conn(0x7f0328072ef0 legacy=0x7f03280cdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:51.414 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.412+0000 7f0327400640 1 -- 172.21.15.145:0/3048590385 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0328161260 con 0x7f03280ce480 2024-05-09T04:35:51.414 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.412+0000 7f0327400640 1 -- 172.21.15.145:0/3048590385 <== mon.0 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3474413261 0 0) 0x7f0320001a70 con 0x7f03280ce480 2024-05-09T04:35:51.414 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.412+0000 7f0327400640 1 -- 172.21.15.145:0/3048590385 >> v1:172.21.15.145:6789/0 conn(0x7f03280ce480 legacy=0x7f03280b46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:51.415 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.412+0000 7f0327400640 1 --2- 172.21.15.145:0/3048590385 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0310003a30 0x7f0310003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:51.415 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.412+0000 7f0327400640 1 --2- 172.21.15.145:0/3048590385 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0310004800 0x7f0310004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:51.415 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.412+0000 7f0327400640 1 --2- 172.21.15.145:0/3048590385 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0328146a90 0x7f0310005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:51.415 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.412+0000 7f0327400640 1 -- 172.21.15.145:0/3048590385 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0328160d30 con 0x7f0310004800 2024-05-09T04:35:51.415 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.412+0000 7f0327400640 1 -- 172.21.15.145:0/3048590385 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f031000baf0 con 0x7f0310003a30 2024-05-09T04:35:51.415 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.412+0000 7f0327400640 1 -- 172.21.15.145:0/3048590385 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f031000bcf0 con 0x7f0328146a90 2024-05-09T04:35:51.415 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.412+0000 7f0327e00640 1 --2- 172.21.15.145:0/3048590385 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0310004800 0x7f0310004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:51.415 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.413+0000 7f032ce00640 1 --2- 172.21.15.145:0/3048590385 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0310003a30 0x7f0310003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:51.415 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.413+0000 7f0327e00640 1 -- 172.21.15.145:0/3048590385 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0328146a90 msgr2=0x7f0310005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:51.415 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.413+0000 7f032d800640 1 --2- 172.21.15.145:0/3048590385 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0328146a90 0x7f0310005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:51.416 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.413+0000 7f0327e00640 1 --2- 172.21.15.145:0/3048590385 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0328146a90 0x7f0310005200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:51.416 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.413+0000 7f0327e00640 1 -- 172.21.15.145:0/3048590385 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0310003a30 msgr2=0x7f0310003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:51.416 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.413+0000 7f0327e00640 1 --2- 172.21.15.145:0/3048590385 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0310003a30 0x7f0310003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:51.416 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.413+0000 7f0327e00640 1 -- 172.21.15.145:0/3048590385 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0310003520 con 0x7f0310004800 2024-05-09T04:35:51.416 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.414+0000 7f0327e00640 1 --2- 172.21.15.145:0/3048590385 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0310004800 0x7f0310004c80 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f0314001790 tx=0x7f0310005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80665575fb5223b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:51.419 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.414+0000 7f0327400640 1 -- 172.21.15.145:0/3048590385 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03140013b0 con 0x7f0310004800 2024-05-09T04:35:51.419 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.414+0000 7f0327400640 1 -- 172.21.15.145:0/3048590385 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03140052d0 con 0x7f0310004800 2024-05-09T04:35:51.419 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.414+0000 7f0327400640 1 -- 172.21.15.145:0/3048590385 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031402aae0 con 0x7f0310004800 2024-05-09T04:35:51.419 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.415+0000 7f032f800640 1 -- 172.21.15.145:0/3048590385 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0310004800 msgr2=0x7f0310004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:51.419 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.415+0000 7f032f800640 1 --2- 172.21.15.145:0/3048590385 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0310004800 0x7f0310004c80 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f0314001790 tx=0x7f0310005830 comp rx=0 tx=0).stop 2024-05-09T04:35:51.420 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.415+0000 7f032d800640 1 -- 172.21.15.145:0/3048590385 reap_dead start 2024-05-09T04:35:51.420 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.415+0000 7f032f800640 1 -- 172.21.15.145:0/3048590385 shutdown_connections 2024-05-09T04:35:51.420 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.415+0000 7f032f800640 1 -- 172.21.15.145:0/3048590385 >> 172.21.15.145:0/3048590385 conn(0x7f032806c8d0 msgr2=0x7f032806cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:51.420 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.415+0000 7f032f800640 1 -- 172.21.15.145:0/3048590385 shutdown_connections 2024-05-09T04:35:51.420 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.415+0000 7f032f800640 1 -- 172.21.15.145:0/3048590385 wait complete. 2024-05-09T04:35:51.420 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.417+0000 7f032f800640 1 Processor -- start 2024-05-09T04:35:51.420 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.417+0000 7f032f800640 1 -- start start 2024-05-09T04:35:51.420 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.418+0000 7f032f800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0310004800 0x7f0328159a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:51.421 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.418+0000 7f032f800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03280b0320 0x7f032813c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:51.421 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.418+0000 7f032f800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03281450c0 0x7f0328145520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:51.421 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.418+0000 7f032f800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f0328161b00 con 0x7f03280b0320 2024-05-09T04:35:51.421 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.418+0000 7f032f800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f0328161820 con 0x7f03281450c0 2024-05-09T04:35:51.421 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.418+0000 7f032f800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f0328072610 con 0x7f0310004800 2024-05-09T04:35:51.421 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.418+0000 7f0327e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03280b0320 0x7f032813c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:51.421 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.418+0000 7f0327e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03280b0320 0x7f032813c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:52956/0 (socket says 172.21.15.145:52956) 2024-05-09T04:35:51.421 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.418+0000 7f0327e00640 1 -- 172.21.15.145:0/1759754736 learned_addr learned my addr 172.21.15.145:0/1759754736 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:51.421 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.419+0000 7f032ce00640 1 --2- 172.21.15.145:0/1759754736 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0310004800 0x7f0328159a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:51.422 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.419+0000 7f0327e00640 1 -- 172.21.15.145:0/1759754736 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0310004800 msgr2=0x7f0328159a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:51.422 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.419+0000 7f0327e00640 1 --2- 172.21.15.145:0/1759754736 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0310004800 0x7f0328159a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:51.422 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.419+0000 7f0327e00640 1 -- 172.21.15.145:0/1759754736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03281450c0 msgr2=0x7f0328145520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:51.422 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.419+0000 7f0327e00640 1 --2- 172.21.15.145:0/1759754736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03281450c0 0x7f0328145520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:51.422 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.419+0000 7f0327e00640 1 -- 172.21.15.145:0/1759754736 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f032815a090 con 0x7f03280b0320 2024-05-09T04:35:51.422 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.419+0000 7f0327e00640 1 --2- 172.21.15.145:0/1759754736 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03280b0320 0x7f032813c430 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f0314001790 tx=0x7f0314001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:51.422 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.419+0000 7f0324c00640 1 -- 172.21.15.145:0/1759754736 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0314037040 con 0x7f03280b0320 2024-05-09T04:35:51.422 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.419+0000 7f0324c00640 1 -- 172.21.15.145:0/1759754736 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0314005480 con 0x7f03280b0320 2024-05-09T04:35:51.422 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.419+0000 7f032f800640 1 -- 172.21.15.145:0/1759754736 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f032815a370 con 0x7f03280b0320 2024-05-09T04:35:51.423 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.419+0000 7f0324c00640 1 -- 172.21.15.145:0/1759754736 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03140013b0 con 0x7f03280b0320 2024-05-09T04:35:51.423 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.419+0000 7f032f800640 1 -- 172.21.15.145:0/1759754736 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f032813c970 con 0x7f03280b0320 2024-05-09T04:35:51.424 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.420+0000 7f032f800640 1 -- 172.21.15.145:0/1759754736 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f032815a5b0 con 0x7f03280b0320 2024-05-09T04:35:51.424 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.422+0000 7f0324c00640 1 -- 172.21.15.145:0/1759754736 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0314044020 con 0x7f03280b0320 2024-05-09T04:35:51.424 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.423+0000 7f0324c00640 1 --2- 172.21.15.145:0/1759754736 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f03080762d0 0x7f0308078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:51.425 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.423+0000 7f0324c00640 1 -- 172.21.15.145:0/1759754736 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7f03140b6d60 con 0x7f03280b0320 2024-05-09T04:35:51.425 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.423+0000 7f0324c00640 1 --2- 172.21.15.145:0/1759754736 >> [v2:172.21.15.145:6824/1845225555,v1:172.21.15.145:6825/1845225555] conn(0x7f030807c190 0x7f030807e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:51.425 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.423+0000 7f0324c00640 1 -- 172.21.15.145:0/1759754736 --> [v2:172.21.15.145:6824/1845225555,v1:172.21.15.145:6825/1845225555] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f03140052d0 con 0x7f030807c190 2024-05-09T04:35:51.425 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.423+0000 7f0324c00640 1 -- 172.21.15.145:0/1759754736 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f03140b9380 con 0x7f03280b0320 2024-05-09T04:35:51.425 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.423+0000 7f032ce00640 1 --2- 172.21.15.145:0/1759754736 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f03080762d0 0x7f0308078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:51.425 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.423+0000 7f032d800640 1 --2- 172.21.15.145:0/1759754736 >> [v2:172.21.15.145:6824/1845225555,v1:172.21.15.145:6825/1845225555] conn(0x7f030807c190 0x7f030807e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:51.425 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.424+0000 7f032ce00640 1 --2- 172.21.15.145:0/1759754736 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f03080762d0 0x7f0308078790 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f031c009950 tx=0x7f031c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:51.425 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.424+0000 7f032d800640 1 --2- 172.21.15.145:0/1759754736 >> [v2:172.21.15.145:6824/1845225555,v1:172.21.15.145:6825/1845225555] conn(0x7f030807c190 0x7f030807e5b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:51.426 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.425+0000 7f0324c00640 1 -- 172.21.15.145:0/1759754736 <== osd.1 v2:172.21.15.145:6824/1845225555 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f03140052d0 con 0x7f030807c190 2024-05-09T04:35:51.439 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.439+0000 7f032f800640 1 -- 172.21.15.145:0/1759754736 --> [v2:172.21.15.145:6824/1845225555,v1:172.21.15.145:6825/1845225555] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f03281f3060 con 0x7f030807c190 2024-05-09T04:35:51.439 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.440+0000 7f0324c00640 1 -- 172.21.15.145:0/1759754736 <== osd.1 v2:172.21.15.145:6824/1845225555 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f03281f3060 con 0x7f030807c190 2024-05-09T04:35:51.440 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-05-09T04:35:51.440 INFO:teuthology.orchestra.run.smithi145.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-09T04:35:51.440 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-05-09T04:35:51.441 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.440+0000 7f032f800640 1 -- 172.21.15.145:0/1759754736 >> [v2:172.21.15.145:6824/1845225555,v1:172.21.15.145:6825/1845225555] conn(0x7f030807c190 msgr2=0x7f030807e5b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:51.441 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.440+0000 7f032f800640 1 --2- 172.21.15.145:0/1759754736 >> [v2:172.21.15.145:6824/1845225555,v1:172.21.15.145:6825/1845225555] conn(0x7f030807c190 0x7f030807e5b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:51.441 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.440+0000 7f032f800640 1 -- 172.21.15.145:0/1759754736 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f03080762d0 msgr2=0x7f0308078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:51.441 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.440+0000 7f032f800640 1 --2- 172.21.15.145:0/1759754736 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f03080762d0 0x7f0308078790 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f031c009950 tx=0x7f031c008040 comp rx=0 tx=0).stop 2024-05-09T04:35:51.441 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.440+0000 7f032f800640 1 -- 172.21.15.145:0/1759754736 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03280b0320 msgr2=0x7f032813c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:51.442 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.440+0000 7f032f800640 1 --2- 172.21.15.145:0/1759754736 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f03280b0320 0x7f032813c430 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f0314001790 tx=0x7f0314001120 comp rx=0 tx=0).stop 2024-05-09T04:35:51.442 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.440+0000 7f032d800640 1 -- 172.21.15.145:0/1759754736 reap_dead start 2024-05-09T04:35:51.442 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.441+0000 7f032f800640 1 -- 172.21.15.145:0/1759754736 shutdown_connections 2024-05-09T04:35:51.442 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.441+0000 7f032f800640 1 -- 172.21.15.145:0/1759754736 >> 172.21.15.145:0/1759754736 conn(0x7f032806c8d0 msgr2=0x7f03281443b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:51.442 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.441+0000 7f032f800640 1 -- 172.21.15.145:0/1759754736 shutdown_connections 2024-05-09T04:35:51.442 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.441+0000 7f032f800640 1 -- 172.21.15.145:0/1759754736 wait complete. 2024-05-09T04:35:51.457 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-05-09T04:35:51.651 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.651+0000 7fcb17200640 1 Processor -- start 2024-05-09T04:35:51.651 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.651+0000 7fcb17200640 1 -- start start 2024-05-09T04:35:51.653 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.653+0000 7fcb17200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcb1806cfb0 0x7fcb180add40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:51.653 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.653+0000 7fcb17200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fcb180d06f0 0x7fcb180ae280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:51.653 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.653+0000 7fcb17200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb180b4ef0 con 0x7fcb1815d190 2024-05-09T04:35:51.653 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.653+0000 7fcb17200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fcb180b51d0 con 0x7fcb180d06f0 2024-05-09T04:35:51.653 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.653+0000 7fcb17200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fcb180b54b0 con 0x7fcb1806cfb0 2024-05-09T04:35:51.653 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.653+0000 7fcb15e00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7fcb1815d190 0x7fcb1806c8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:60526/0 (socket says 172.21.15.145:60526) 2024-05-09T04:35:51.653 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.653+0000 7fcb15e00640 1 -- 172.21.15.145:0/1252380060 learned_addr learned my addr 172.21.15.145:0/1252380060 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:35:51.654 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.653+0000 7fcb16800640 1 --2- 172.21.15.145:0/1252380060 >> v2:172.21.15.145:3300/0 conn(0x7fcb180d06f0 0x7fcb180ae280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:51.655 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.653+0000 7fcb0fe00640 1 --2- 172.21.15.145:0/1252380060 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcb1806cfb0 0x7fcb180add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:51.655 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.654+0000 7fcb16800640 1 -- 172.21.15.145:0/1252380060 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcb1806cfb0 msgr2=0x7fcb180add40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:51.655 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.654+0000 7fcb16800640 1 --2- 172.21.15.145:0/1252380060 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcb1806cfb0 0x7fcb180add40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:51.655 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.654+0000 7fcb16800640 1 -- 172.21.15.145:0/1252380060 >> v1:172.21.15.145:6789/0 conn(0x7fcb1815d190 legacy=0x7fcb1806c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:51.655 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.654+0000 7fcb16800640 1 -- 172.21.15.145:0/1252380060 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb180b5790 con 0x7fcb180d06f0 2024-05-09T04:35:51.655 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.654+0000 7fcb15400640 1 -- 172.21.15.145:0/1252380060 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2464274371 0 0) 0x7fcb180b4ef0 con 0x7fcb1815d190 2024-05-09T04:35:51.656 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.654+0000 7fcb0fe00640 1 --2- 172.21.15.145:0/1252380060 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcb1806cfb0 0x7fcb180add40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:35:51.656 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.655+0000 7fcb16800640 1 --2- 172.21.15.145:0/1252380060 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb180d06f0 0x7fcb180ae280 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fcb10009d90 tx=0x7fcb1002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfca75788aa5b1e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:51.656 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.655+0000 7fcb15400640 1 -- 172.21.15.145:0/1252380060 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb10007740 con 0x7fcb180d06f0 2024-05-09T04:35:51.656 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.655+0000 7fcb15400640 1 -- 172.21.15.145:0/1252380060 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb100078e0 con 0x7fcb180d06f0 2024-05-09T04:35:51.656 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.655+0000 7fcb15400640 1 -- 172.21.15.145:0/1252380060 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb10007cb0 con 0x7fcb180d06f0 2024-05-09T04:35:51.656 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.656+0000 7fcb17200640 1 -- 172.21.15.145:0/1252380060 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb180d06f0 msgr2=0x7fcb180ae280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:51.656 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.656+0000 7fcb17200640 1 --2- 172.21.15.145:0/1252380060 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb180d06f0 0x7fcb180ae280 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fcb10009d90 tx=0x7fcb1002a660 comp rx=0 tx=0).stop 2024-05-09T04:35:51.657 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.656+0000 7fcb17200640 1 -- 172.21.15.145:0/1252380060 shutdown_connections 2024-05-09T04:35:51.658 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.656+0000 7fcb17200640 1 --2- 172.21.15.145:0/1252380060 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb180d06f0 0x7fcb180ae280 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:51.658 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.656+0000 7fcb17200640 1 --2- 172.21.15.145:0/1252380060 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcb1806cfb0 0x7fcb180add40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:51.658 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.656+0000 7fcb17200640 1 -- 172.21.15.145:0/1252380060 >> 172.21.15.145:0/1252380060 conn(0x7fcb180b2d80 msgr2=0x7fcb180b3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:51.658 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.656+0000 7fcb17200640 1 -- 172.21.15.145:0/1252380060 shutdown_connections 2024-05-09T04:35:51.658 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.656+0000 7fcb17200640 1 -- 172.21.15.145:0/1252380060 wait complete. 2024-05-09T04:35:51.658 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.658+0000 7fcb17200640 1 Processor -- start 2024-05-09T04:35:51.658 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.658+0000 7fcb17200640 1 -- start start 2024-05-09T04:35:51.660 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.659+0000 7fcb17200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb1806cfb0 0x7fcb18158730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:51.660 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.659+0000 7fcb17200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb180d06f0 0x7fcb181f4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:51.661 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.659+0000 7fcb17200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcb1815d190 0x7fcb18155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:51.661 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.659+0000 7fcb17200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fcb180cf210 con 0x7fcb1806cfb0 2024-05-09T04:35:51.661 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.659+0000 7fcb17200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fcb180b4cd0 con 0x7fcb180d06f0 2024-05-09T04:35:51.661 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.659+0000 7fcb17200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fcb180cf390 con 0x7fcb1815d190 2024-05-09T04:35:51.661 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.659+0000 7fcb15e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb1806cfb0 0x7fcb18158730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:51.661 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.659+0000 7fcb15e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb1806cfb0 0x7fcb18158730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:52972/0 (socket says 172.21.15.145:52972) 2024-05-09T04:35:51.661 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.659+0000 7fcb15e00640 1 -- 172.21.15.145:0/4131899973 learned_addr learned my addr 172.21.15.145:0/4131899973 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:51.661 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.659+0000 7fcb15e00640 1 -- 172.21.15.145:0/4131899973 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcb1815d190 msgr2=0x7fcb18155bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:51.662 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.659+0000 7fcb15e00640 1 --2- 172.21.15.145:0/4131899973 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcb1815d190 0x7fcb18155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:51.662 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.659+0000 7fcb15e00640 1 -- 172.21.15.145:0/4131899973 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb180d06f0 msgr2=0x7fcb181f4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:51.662 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.659+0000 7fcb0fe00640 1 --2- 172.21.15.145:0/4131899973 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb180d06f0 0x7fcb181f4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:51.662 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.659+0000 7fcb15e00640 1 --2- 172.21.15.145:0/4131899973 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb180d06f0 0x7fcb181f4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:51.662 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.659+0000 7fcb15e00640 1 -- 172.21.15.145:0/4131899973 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb181577c0 con 0x7fcb1806cfb0 2024-05-09T04:35:51.662 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.660+0000 7fcb15e00640 1 --2- 172.21.15.145:0/4131899973 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb1806cfb0 0x7fcb18158730 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fcb0800c3e0 tx=0x7fcb0800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:51.662 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.660+0000 7fcb0e000640 1 -- 172.21.15.145:0/4131899973 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb080074e0 con 0x7fcb1806cfb0 2024-05-09T04:35:51.662 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.660+0000 7fcb0e000640 1 -- 172.21.15.145:0/4131899973 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb080047e0 con 0x7fcb1806cfb0 2024-05-09T04:35:51.663 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.660+0000 7fcb17200640 1 -- 172.21.15.145:0/4131899973 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb18157a60 con 0x7fcb1806cfb0 2024-05-09T04:35:51.663 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.660+0000 7fcb0e000640 1 -- 172.21.15.145:0/4131899973 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb08004bd0 con 0x7fcb1806cfb0 2024-05-09T04:35:51.663 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.660+0000 7fcb17200640 1 -- 172.21.15.145:0/4131899973 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb181515e0 con 0x7fcb1806cfb0 2024-05-09T04:35:51.664 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.661+0000 7fcb17200640 1 -- 172.21.15.145:0/4131899973 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb180b0640 con 0x7fcb1806cfb0 2024-05-09T04:35:51.664 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.662+0000 7fcb0e000640 1 -- 172.21.15.145:0/4131899973 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fcb08011070 con 0x7fcb1806cfb0 2024-05-09T04:35:51.664 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.663+0000 7fcb0e000640 1 --2- 172.21.15.145:0/4131899973 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fcaf40762d0 0x7fcaf4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:51.664 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.664+0000 7fcb0e000640 1 -- 172.21.15.145:0/4131899973 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7fcb08096fa0 con 0x7fcb1806cfb0 2024-05-09T04:35:51.664 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.664+0000 7fcb0fe00640 1 --2- 172.21.15.145:0/4131899973 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fcaf40762d0 0x7fcaf4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:51.665 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.664+0000 7fcb0fe00640 1 --2- 172.21.15.145:0/4131899973 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fcaf40762d0 0x7fcaf4078790 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fcb180b1dc0 tx=0x7fcb00008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:51.668 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.668+0000 7fcb0e000640 1 -- 172.21.15.145:0/4131899973 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fcb180b0640 con 0x7fcb1806cfb0 2024-05-09T04:35:51.881 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.881+0000 7fcb17200640 1 -- 172.21.15.145:0/4131899973 --> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7fcb18076780 con 0x7fcaf40762d0 2024-05-09T04:35:51.881 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.881+0000 7fcb0e000640 1 -- 172.21.15.145:0/4131899973 <== mgr.4104 v2:172.21.15.145:6832/2725855396 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fcb18076780 con 0x7fcaf40762d0 2024-05-09T04:35:51.882 INFO:teuthology.orchestra.run.smithi145.stderr:instructed osd(s) 1 to deep-scrub 2024-05-09T04:35:51.885 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.884+0000 7fcb17200640 1 -- 172.21.15.145:0/4131899973 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fcaf40762d0 msgr2=0x7fcaf4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:51.885 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.884+0000 7fcb17200640 1 --2- 172.21.15.145:0/4131899973 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fcaf40762d0 0x7fcaf4078790 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fcb180b1dc0 tx=0x7fcb00008040 comp rx=0 tx=0).stop 2024-05-09T04:35:51.885 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.884+0000 7fcb17200640 1 -- 172.21.15.145:0/4131899973 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb1806cfb0 msgr2=0x7fcb18158730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:51.885 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.884+0000 7fcb17200640 1 --2- 172.21.15.145:0/4131899973 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb1806cfb0 0x7fcb18158730 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fcb0800c3e0 tx=0x7fcb0800c8a0 comp rx=0 tx=0).stop 2024-05-09T04:35:51.885 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.885+0000 7fcb17200640 1 -- 172.21.15.145:0/4131899973 shutdown_connections 2024-05-09T04:35:51.885 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.885+0000 7fcb17200640 1 --2- 172.21.15.145:0/4131899973 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fcaf40762d0 0x7fcaf4078790 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:51.886 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.885+0000 7fcb17200640 1 --2- 172.21.15.145:0/4131899973 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcb1815d190 0x7fcb18155bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:51.886 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.885+0000 7fcb17200640 1 --2- 172.21.15.145:0/4131899973 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb180d06f0 0x7fcb181f4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:51.886 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.885+0000 7fcb17200640 1 --2- 172.21.15.145:0/4131899973 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcb1806cfb0 0x7fcb18158730 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:51.886 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.885+0000 7fcb17200640 1 -- 172.21.15.145:0/4131899973 >> 172.21.15.145:0/4131899973 conn(0x7fcb180b2d80 msgr2=0x7fcb1806c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:51.886 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.885+0000 7fcb17200640 1 -- 172.21.15.145:0/4131899973 shutdown_connections 2024-05-09T04:35:51.886 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:51.885+0000 7fcb17200640 1 -- 172.21.15.145:0/4131899973 wait complete. 2024-05-09T04:35:51.904 INFO:tasks.ceph:Scrubbing osd.2 2024-05-09T04:35:51.904 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-05-09T04:35:52.108 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.108+0000 7f7ff8a00640 1 Processor -- start 2024-05-09T04:35:52.108 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.108+0000 7f7ff8a00640 1 -- start start 2024-05-09T04:35:52.110 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.110+0000 7f7ff8a00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f7ff40cdc60 0x7f7ff40cea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:52.110 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.110+0000 7f7ff8a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7ff406e060 0x7f7ff415d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:52.110 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.110+0000 7f7ff8a00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ff415dc10 con 0x7f7ff4146970 2024-05-09T04:35:52.110 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.110+0000 7f7ff8a00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f7ff4072ee0 con 0x7f7ff40cdc60 2024-05-09T04:35:52.110 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.110+0000 7f7ff8a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f7ff415def0 con 0x7f7ff406e060 2024-05-09T04:35:52.110 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.110+0000 7f7ff3400640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f7ff4146970 0x7f7ff40cd550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:60532/0 (socket says 172.21.15.145:60532) 2024-05-09T04:35:52.110 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.110+0000 7f7ff3400640 1 -- 172.21.15.145:0/4000495601 learned_addr learned my addr 172.21.15.145:0/4000495601 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:35:52.112 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.111+0000 7f7ff2a00640 1 --2- 172.21.15.145:0/4000495601 >> v2:172.21.15.192:3300/0 conn(0x7f7ff40cdc60 0x7f7ff40cea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:52.113 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.111+0000 7f7ff3e00640 1 --2- 172.21.15.145:0/4000495601 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7ff406e060 0x7f7ff415d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:52.113 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.111+0000 7f7ff2000640 1 -- 172.21.15.145:0/4000495601 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4019632803 0 0) 0x7f7ff415dc10 con 0x7f7ff4146970 2024-05-09T04:35:52.113 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.111+0000 7f7ff2000640 1 -- 172.21.15.145:0/4000495601 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7fd8003520 con 0x7f7ff4146970 2024-05-09T04:35:52.113 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.111+0000 7f7ff2000640 1 -- 172.21.15.145:0/4000495601 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2390008263 0 0) 0x7f7fd8003520 con 0x7f7ff4146970 2024-05-09T04:35:52.113 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.111+0000 7f7ff2000640 1 -- 172.21.15.145:0/4000495601 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7ff406e060 msgr2=0x7f7ff415d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:52.113 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.111+0000 7f7ff2000640 1 --2- 172.21.15.145:0/4000495601 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7ff406e060 0x7f7ff415d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:52.113 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.111+0000 7f7ff2000640 1 -- 172.21.15.145:0/4000495601 >> v2:172.21.15.192:3300/0 conn(0x7f7ff40cdc60 msgr2=0x7f7ff40cea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:52.113 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.111+0000 7f7ff2000640 1 --2- 172.21.15.145:0/4000495601 >> v2:172.21.15.192:3300/0 conn(0x7f7ff40cdc60 0x7f7ff40cea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:52.114 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.111+0000 7f7ff2000640 1 -- 172.21.15.145:0/4000495601 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ff415e1d0 con 0x7f7ff4146970 2024-05-09T04:35:52.114 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.111+0000 7f7ff2000640 1 -- 172.21.15.145:0/4000495601 <== mon.0 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3474413261 0 0) 0x7f7fe40016f0 con 0x7f7ff4146970 2024-05-09T04:35:52.114 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.111+0000 7f7ff2000640 1 -- 172.21.15.145:0/4000495601 >> v1:172.21.15.145:6789/0 conn(0x7f7ff4146970 legacy=0x7f7ff40cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:52.114 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.111+0000 7f7ff2000640 1 --2- 172.21.15.145:0/4000495601 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7ff406e060 0x7f7fd8003a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:52.114 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.111+0000 7f7ff2000640 1 --2- 172.21.15.145:0/4000495601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fd8004020 0x7f7fd8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:52.114 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.111+0000 7f7ff2000640 1 --2- 172.21.15.145:0/4000495601 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7fd8004d40 0x7f7fd80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:52.114 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.111+0000 7f7ff2000640 1 -- 172.21.15.145:0/4000495601 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff4072ee0 con 0x7f7fd8004d40 2024-05-09T04:35:52.114 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.111+0000 7f7ff2000640 1 -- 172.21.15.145:0/4000495601 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff415def0 con 0x7f7fd8004020 2024-05-09T04:35:52.115 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.111+0000 7f7ff2000640 1 -- 172.21.15.145:0/4000495601 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f7fd800ba70 con 0x7f7ff406e060 2024-05-09T04:35:52.115 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.112+0000 7f7ff3e00640 1 --2- 172.21.15.145:0/4000495601 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7fd8004d40 0x7f7fd80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:52.115 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.112+0000 7f7ff3400640 1 --2- 172.21.15.145:0/4000495601 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7ff406e060 0x7f7fd8003a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:52.115 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.112+0000 7f7ff2a00640 1 --2- 172.21.15.145:0/4000495601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fd8004020 0x7f7fd8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:52.115 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.112+0000 7f7ff3e00640 1 -- 172.21.15.145:0/4000495601 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7ff406e060 msgr2=0x7f7fd8003a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:52.115 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.112+0000 7f7ff3e00640 1 --2- 172.21.15.145:0/4000495601 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7ff406e060 0x7f7fd8003a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:52.115 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.112+0000 7f7ff3e00640 1 -- 172.21.15.145:0/4000495601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fd8004020 msgr2=0x7f7fd8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:52.115 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.112+0000 7f7ff3e00640 1 --2- 172.21.15.145:0/4000495601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7fd8004020 0x7f7fd8004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:52.115 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.112+0000 7f7ff3e00640 1 -- 172.21.15.145:0/4000495601 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fd8003520 con 0x7f7fd8004d40 2024-05-09T04:35:52.118 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.113+0000 7f7ff3e00640 1 --2- 172.21.15.145:0/4000495601 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7fd8004d40 0x7f7fd80051c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7fec009f90 tx=0x7f7fd8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b99e6fd7bb244ce server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:52.118 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.114+0000 7f7ff2000640 1 -- 172.21.15.145:0/4000495601 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fec009c10 con 0x7f7fd8004d40 2024-05-09T04:35:52.119 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.114+0000 7f7ff2000640 1 -- 172.21.15.145:0/4000495601 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fec036070 con 0x7f7fd8004d40 2024-05-09T04:35:52.119 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.114+0000 7f7ff2000640 1 -- 172.21.15.145:0/4000495601 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fec031810 con 0x7f7fd8004d40 2024-05-09T04:35:52.119 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.114+0000 7f7ff8a00640 1 -- 172.21.15.145:0/4000495601 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7fd8004d40 msgr2=0x7f7fd80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:52.119 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.114+0000 7f7ff8a00640 1 --2- 172.21.15.145:0/4000495601 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7fd8004d40 0x7f7fd80051c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7fec009f90 tx=0x7f7fd8005830 comp rx=0 tx=0).stop 2024-05-09T04:35:52.119 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.114+0000 7f7ff3e00640 1 -- 172.21.15.145:0/4000495601 reap_dead start 2024-05-09T04:35:52.119 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.114+0000 7f7ff8a00640 1 -- 172.21.15.145:0/4000495601 shutdown_connections 2024-05-09T04:35:52.119 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.114+0000 7f7ff8a00640 1 -- 172.21.15.145:0/4000495601 >> 172.21.15.145:0/4000495601 conn(0x7f7ff40b2850 msgr2=0x7f7ff40b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:52.119 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.115+0000 7f7ff8a00640 1 -- 172.21.15.145:0/4000495601 shutdown_connections 2024-05-09T04:35:52.120 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.115+0000 7f7ff8a00640 1 -- 172.21.15.145:0/4000495601 wait complete. 2024-05-09T04:35:52.120 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.116+0000 7f7ff8a00640 1 Processor -- start 2024-05-09T04:35:52.120 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.117+0000 7f7ff8a00640 1 -- start start 2024-05-09T04:35:52.120 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.117+0000 7f7ff8a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7fd8004d40 0x7f7ff413fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:52.120 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.117+0000 7f7ff8a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7ff4145a00 0x7f7ff4142970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:52.120 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.117+0000 7f7ff8a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ff4159ec0 0x7f7ff415a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:52.120 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.118+0000 7f7ff8a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff40ce7c0 con 0x7f7fd8004d40 2024-05-09T04:35:52.120 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.118+0000 7f7ff8a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff40ce640 con 0x7f7ff4159ec0 2024-05-09T04:35:52.120 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.118+0000 7f7ff8a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f7ff415d5e0 con 0x7f7ff4145a00 2024-05-09T04:35:52.121 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.118+0000 7f7ff3400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7fd8004d40 0x7f7ff413fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:52.121 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.118+0000 7f7ff3400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7fd8004d40 0x7f7ff413fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:52996/0 (socket says 172.21.15.145:52996) 2024-05-09T04:35:52.121 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.118+0000 7f7ff3400640 1 -- 172.21.15.145:0/476225011 learned_addr learned my addr 172.21.15.145:0/476225011 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:52.121 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.118+0000 7f7ff3400640 1 -- 172.21.15.145:0/476225011 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7ff4145a00 msgr2=0x7f7ff4142970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:52.121 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.118+0000 7f7ff3400640 1 --2- 172.21.15.145:0/476225011 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7ff4145a00 0x7f7ff4142970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:52.121 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.118+0000 7f7ff3400640 1 -- 172.21.15.145:0/476225011 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ff4159ec0 msgr2=0x7f7ff415a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:52.121 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.118+0000 7f7ff3e00640 1 --2- 172.21.15.145:0/476225011 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ff4159ec0 0x7f7ff415a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:52.121 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.118+0000 7f7ff3400640 1 --2- 172.21.15.145:0/476225011 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ff4159ec0 0x7f7ff415a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:52.121 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.118+0000 7f7ff3400640 1 -- 172.21.15.145:0/476225011 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ff40ca820 con 0x7f7fd8004d40 2024-05-09T04:35:52.122 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.119+0000 7f7ff3400640 1 --2- 172.21.15.145:0/476225011 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7fd8004d40 0x7f7ff413fb50 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f7fe400e820 tx=0x7f7fe400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:52.122 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.119+0000 7f7feb400640 1 -- 172.21.15.145:0/476225011 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe400c8d0 con 0x7f7fd8004d40 2024-05-09T04:35:52.122 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.119+0000 7f7feb400640 1 -- 172.21.15.145:0/476225011 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fe4018070 con 0x7f7fd8004d40 2024-05-09T04:35:52.122 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.119+0000 7f7feb400640 1 -- 172.21.15.145:0/476225011 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe400ccb0 con 0x7f7fd8004d40 2024-05-09T04:35:52.122 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.119+0000 7f7ff8a00640 1 -- 172.21.15.145:0/476225011 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ff415a880 con 0x7f7fd8004d40 2024-05-09T04:35:52.122 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.119+0000 7f7ff8a00640 1 -- 172.21.15.145:0/476225011 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ff41404c0 con 0x7f7fd8004d40 2024-05-09T04:35:52.123 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.120+0000 7f7ff8a00640 1 -- 172.21.15.145:0/476225011 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7ff406a520 con 0x7f7fd8004d40 2024-05-09T04:35:52.124 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.121+0000 7f7feb400640 1 -- 172.21.15.145:0/476225011 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f7fe401d460 con 0x7f7fd8004d40 2024-05-09T04:35:52.124 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.122+0000 7f7feb400640 1 --2- 172.21.15.145:0/476225011 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f7fd40762d0 0x7f7fd4078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:52.124 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.122+0000 7f7feb400640 1 -- 172.21.15.145:0/476225011 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7f7fe4097290 con 0x7f7fd8004d40 2024-05-09T04:35:52.124 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.123+0000 7f7ff2a00640 1 --2- 172.21.15.145:0/476225011 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f7fd40762d0 0x7f7fd4078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:52.124 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.123+0000 7f7feb400640 1 --2- 172.21.15.145:0/476225011 >> [v2:172.21.15.145:6816/2168958993,v1:172.21.15.145:6817/2168958993] conn(0x7f7fd407c190 0x7f7fd407e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:52.124 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.123+0000 7f7feb400640 1 -- 172.21.15.145:0/476225011 --> [v2:172.21.15.145:6816/2168958993,v1:172.21.15.145:6817/2168958993] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7fe401be50 con 0x7f7fd407c190 2024-05-09T04:35:52.124 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.123+0000 7f7feb400640 1 -- 172.21.15.145:0/476225011 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f7fe409e050 con 0x7f7fd8004d40 2024-05-09T04:35:52.124 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.123+0000 7f7ff3e00640 1 --2- 172.21.15.145:0/476225011 >> [v2:172.21.15.145:6816/2168958993,v1:172.21.15.145:6817/2168958993] conn(0x7f7fd407c190 0x7f7fd407e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:52.124 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.123+0000 7f7ff2a00640 1 --2- 172.21.15.145:0/476225011 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f7fd40762d0 0x7f7fd4078790 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7ff413f0f0 tx=0x7f7fe0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:52.125 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.124+0000 7f7ff3e00640 1 --2- 172.21.15.145:0/476225011 >> [v2:172.21.15.145:6816/2168958993,v1:172.21.15.145:6817/2168958993] conn(0x7f7fd407c190 0x7f7fd407e5b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:52.125 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.124+0000 7f7feb400640 1 -- 172.21.15.145:0/476225011 <== osd.2 v2:172.21.15.145:6816/2168958993 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7fe401be50 con 0x7f7fd407c190 2024-05-09T04:35:52.139 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.139+0000 7f7ff8a00640 1 -- 172.21.15.145:0/476225011 --> [v2:172.21.15.145:6816/2168958993,v1:172.21.15.145:6817/2168958993] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f7ff4076930 con 0x7f7fd407c190 2024-05-09T04:35:52.139 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.139+0000 7f7feb400640 1 -- 172.21.15.145:0/476225011 <== osd.2 v2:172.21.15.145:6816/2168958993 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f7ff4076930 con 0x7f7fd407c190 2024-05-09T04:35:52.140 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-05-09T04:35:52.140 INFO:teuthology.orchestra.run.smithi145.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-09T04:35:52.140 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-05-09T04:35:52.141 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.140+0000 7f7ff8a00640 1 -- 172.21.15.145:0/476225011 >> [v2:172.21.15.145:6816/2168958993,v1:172.21.15.145:6817/2168958993] conn(0x7f7fd407c190 msgr2=0x7f7fd407e5b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:52.141 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.140+0000 7f7ff8a00640 1 --2- 172.21.15.145:0/476225011 >> [v2:172.21.15.145:6816/2168958993,v1:172.21.15.145:6817/2168958993] conn(0x7f7fd407c190 0x7f7fd407e5b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:52.142 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.140+0000 7f7ff8a00640 1 -- 172.21.15.145:0/476225011 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f7fd40762d0 msgr2=0x7f7fd4078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:52.142 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.140+0000 7f7ff8a00640 1 --2- 172.21.15.145:0/476225011 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f7fd40762d0 0x7f7fd4078790 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7ff413f0f0 tx=0x7f7fe0008040 comp rx=0 tx=0).stop 2024-05-09T04:35:52.142 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.140+0000 7f7ff8a00640 1 -- 172.21.15.145:0/476225011 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7fd8004d40 msgr2=0x7f7ff413fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:52.142 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.140+0000 7f7ff8a00640 1 --2- 172.21.15.145:0/476225011 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f7fd8004d40 0x7f7ff413fb50 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f7fe400e820 tx=0x7f7fe400ece0 comp rx=0 tx=0).stop 2024-05-09T04:35:52.142 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.140+0000 7f7ff3e00640 1 -- 172.21.15.145:0/476225011 reap_dead start 2024-05-09T04:35:52.142 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.141+0000 7f7ff8a00640 1 -- 172.21.15.145:0/476225011 shutdown_connections 2024-05-09T04:35:52.142 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.141+0000 7f7ff8a00640 1 -- 172.21.15.145:0/476225011 >> 172.21.15.145:0/476225011 conn(0x7f7ff40b2850 msgr2=0x7f7ff415bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:52.142 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.141+0000 7f7ff8a00640 1 -- 172.21.15.145:0/476225011 shutdown_connections 2024-05-09T04:35:52.142 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.141+0000 7f7ff8a00640 1 -- 172.21.15.145:0/476225011 wait complete. 2024-05-09T04:35:52.159 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-05-09T04:35:52.361 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.361+0000 7f33e4000640 1 Processor -- start 2024-05-09T04:35:52.361 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.361+0000 7f33e4000640 1 -- start start 2024-05-09T04:35:52.362 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.363+0000 7f33e4000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f33dc165bc0 0x7f33dc167fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:52.363 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.363+0000 7f33e4000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33dc15d7f0 con 0x7f33dc06c480 2024-05-09T04:35:52.363 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.363+0000 7f33e4000640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33dc15dad0 con 0x7f33dc0b6f20 2024-05-09T04:35:52.363 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.363+0000 7f33e4000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f33dc15e5b0 con 0x7f33dc165bc0 2024-05-09T04:35:52.363 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.363+0000 7f33e1600640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f33dc06c480 0x7f33dc06c860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:60536/0 (socket says 172.21.15.145:60536) 2024-05-09T04:35:52.363 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.363+0000 7f33e2000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f33dc165bc0 0x7f33dc167fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:52.363 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.363+0000 7f33e1600640 1 -- 172.21.15.145:0/4242654987 learned_addr learned my addr 172.21.15.145:0/4242654987 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:35:52.364 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.364+0000 7f33e2000640 1 -- 172.21.15.145:0/4242654987 >> v1:172.21.15.192:6789/0 conn(0x7f33dc0b6f20 legacy=0x7f33dc0aedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:52.364 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.364+0000 7f33e2000640 1 -- 172.21.15.145:0/4242654987 >> v1:172.21.15.145:6789/0 conn(0x7f33dc06c480 legacy=0x7f33dc06c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:52.364 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.364+0000 7f33e2000640 1 -- 172.21.15.145:0/4242654987 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33dc15ddb0 con 0x7f33dc165bc0 2024-05-09T04:35:52.364 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.364+0000 7f33dbe00640 1 -- 172.21.15.145:0/4242654987 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3536413182 0 0) 0x7f33dc15d7f0 con 0x7f33dc06c480 2024-05-09T04:35:52.366 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.365+0000 7f33e2000640 1 --2- 172.21.15.145:0/4242654987 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f33dc165bc0 0x7f33dc167fb0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f33d4009d90 tx=0x7f33d402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=131d3956faca6d68 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:52.366 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.365+0000 7f33dbe00640 1 -- 172.21.15.145:0/4242654987 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d4007740 con 0x7f33dc165bc0 2024-05-09T04:35:52.366 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.365+0000 7f33dbe00640 1 -- 172.21.15.145:0/4242654987 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33d40078e0 con 0x7f33dc165bc0 2024-05-09T04:35:52.366 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.365+0000 7f33dbe00640 1 -- 172.21.15.145:0/4242654987 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d4007cb0 con 0x7f33dc165bc0 2024-05-09T04:35:52.366 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.366+0000 7f33e4000640 1 -- 172.21.15.145:0/4242654987 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f33dc165bc0 msgr2=0x7f33dc167fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:52.366 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.366+0000 7f33e4000640 1 --2- 172.21.15.145:0/4242654987 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f33dc165bc0 0x7f33dc167fb0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f33d4009d90 tx=0x7f33d402a660 comp rx=0 tx=0).stop 2024-05-09T04:35:52.366 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.366+0000 7f33e4000640 1 -- 172.21.15.145:0/4242654987 shutdown_connections 2024-05-09T04:35:52.366 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.366+0000 7f33e4000640 1 --2- 172.21.15.145:0/4242654987 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f33dc165bc0 0x7f33dc167fb0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:52.367 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.366+0000 7f33e4000640 1 -- 172.21.15.145:0/4242654987 >> 172.21.15.145:0/4242654987 conn(0x7f33dc0b3ae0 msgr2=0x7f33dc0b3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:52.367 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.366+0000 7f33e4000640 1 -- 172.21.15.145:0/4242654987 shutdown_connections 2024-05-09T04:35:52.367 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.366+0000 7f33e4000640 1 -- 172.21.15.145:0/4242654987 wait complete. 2024-05-09T04:35:52.367 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.368+0000 7f33e4000640 1 Processor -- start 2024-05-09T04:35:52.367 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.368+0000 7f33e4000640 1 -- start start 2024-05-09T04:35:52.369 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.369+0000 7f33e4000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f33dc06c480 0x7f33dc0cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:52.369 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.369+0000 7f33e4000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33dc0b6f20 0x7f33dc154d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:52.369 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.369+0000 7f33e4000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33dc165bc0 0x7f33dc147810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:52.370 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.369+0000 7f33e4000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f33dc0af5f0 con 0x7f33dc06c480 2024-05-09T04:35:52.370 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.369+0000 7f33e4000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f33dc15d5b0 con 0x7f33dc165bc0 2024-05-09T04:35:52.370 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.369+0000 7f33e4000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f33dc15ae20 con 0x7f33dc0b6f20 2024-05-09T04:35:52.370 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.369+0000 7f33e1600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f33dc06c480 0x7f33dc0cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:52.370 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.369+0000 7f33e1600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f33dc06c480 0x7f33dc0cfda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:53006/0 (socket says 172.21.15.145:53006) 2024-05-09T04:35:52.370 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.369+0000 7f33e1600640 1 -- 172.21.15.145:0/317147992 learned_addr learned my addr 172.21.15.145:0/317147992 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:52.370 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.370+0000 7f33e1600640 1 -- 172.21.15.145:0/317147992 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33dc0b6f20 msgr2=0x7f33dc154d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:52.370 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.370+0000 7f33e1600640 1 --2- 172.21.15.145:0/317147992 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33dc0b6f20 0x7f33dc154d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:52.371 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.370+0000 7f33e1600640 1 -- 172.21.15.145:0/317147992 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33dc165bc0 msgr2=0x7f33dc147810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:52.371 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.370+0000 7f33e2000640 1 --2- 172.21.15.145:0/317147992 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33dc165bc0 0x7f33dc147810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:52.371 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.370+0000 7f33e1600640 1 --2- 172.21.15.145:0/317147992 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33dc165bc0 0x7f33dc147810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:52.371 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.370+0000 7f33e1600640 1 -- 172.21.15.145:0/317147992 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33dc155290 con 0x7f33dc06c480 2024-05-09T04:35:52.371 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.370+0000 7f33e1600640 1 --2- 172.21.15.145:0/317147992 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f33dc06c480 0x7f33dc0cfda0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f33d000c430 tx=0x7f33d000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:52.372 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.370+0000 7f33d9600640 1 -- 172.21.15.145:0/317147992 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d0017070 con 0x7f33dc06c480 2024-05-09T04:35:52.372 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.370+0000 7f33d9600640 1 -- 172.21.15.145:0/317147992 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33d0005ce0 con 0x7f33dc06c480 2024-05-09T04:35:52.372 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.370+0000 7f33e4000640 1 -- 172.21.15.145:0/317147992 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33dc155540 con 0x7f33dc06c480 2024-05-09T04:35:52.372 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.370+0000 7f33d9600640 1 -- 172.21.15.145:0/317147992 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d00072d0 con 0x7f33dc06c480 2024-05-09T04:35:52.372 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.370+0000 7f33e4000640 1 -- 172.21.15.145:0/317147992 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33dc155b00 con 0x7f33dc06c480 2024-05-09T04:35:52.373 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.371+0000 7f33e4000640 1 -- 172.21.15.145:0/317147992 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33dc0694e0 con 0x7f33dc06c480 2024-05-09T04:35:52.373 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.373+0000 7f33d9600640 1 -- 172.21.15.145:0/317147992 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f33d001a070 con 0x7f33dc06c480 2024-05-09T04:35:52.373 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.374+0000 7f33d9600640 1 --2- 172.21.15.145:0/317147992 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f33bc0762d0 0x7f33bc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:52.373 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.374+0000 7f33d9600640 1 -- 172.21.15.145:0/317147992 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7f33d0096630 con 0x7f33dc06c480 2024-05-09T04:35:52.374 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.374+0000 7f33e0c00640 1 --2- 172.21.15.145:0/317147992 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f33bc0762d0 0x7f33bc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:52.374 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.375+0000 7f33e0c00640 1 --2- 172.21.15.145:0/317147992 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f33bc0762d0 0x7f33bc078790 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f33c8006fd0 tx=0x7f33c8006bc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:52.377 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.378+0000 7f33d9600640 1 -- 172.21.15.145:0/317147992 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f33d005fba0 con 0x7f33dc06c480 2024-05-09T04:35:52.588 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.588+0000 7f33e4000640 1 -- 172.21.15.145:0/317147992 --> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f33dc06c900 con 0x7f33bc0762d0 2024-05-09T04:35:52.589 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.589+0000 7f33d9600640 1 -- 172.21.15.145:0/317147992 <== mgr.4104 v2:172.21.15.145:6832/2725855396 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f33dc06c900 con 0x7f33bc0762d0 2024-05-09T04:35:52.589 INFO:teuthology.orchestra.run.smithi145.stderr:instructed osd(s) 2 to deep-scrub 2024-05-09T04:35:52.592 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.592+0000 7f33e4000640 1 -- 172.21.15.145:0/317147992 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f33bc0762d0 msgr2=0x7f33bc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:52.592 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.592+0000 7f33e4000640 1 --2- 172.21.15.145:0/317147992 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f33bc0762d0 0x7f33bc078790 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f33c8006fd0 tx=0x7f33c8006bc0 comp rx=0 tx=0).stop 2024-05-09T04:35:52.592 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.592+0000 7f33e4000640 1 -- 172.21.15.145:0/317147992 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f33dc06c480 msgr2=0x7f33dc0cfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:52.593 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.592+0000 7f33e4000640 1 --2- 172.21.15.145:0/317147992 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f33dc06c480 0x7f33dc0cfda0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f33d000c430 tx=0x7f33d000c8f0 comp rx=0 tx=0).stop 2024-05-09T04:35:52.593 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.592+0000 7f33e4000640 1 -- 172.21.15.145:0/317147992 shutdown_connections 2024-05-09T04:35:52.593 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.592+0000 7f33e4000640 1 --2- 172.21.15.145:0/317147992 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f33bc0762d0 0x7f33bc078790 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:52.593 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.592+0000 7f33e4000640 1 --2- 172.21.15.145:0/317147992 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33dc165bc0 0x7f33dc147810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:52.593 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.592+0000 7f33e4000640 1 --2- 172.21.15.145:0/317147992 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33dc0b6f20 0x7f33dc154d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:52.593 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.592+0000 7f33e4000640 1 --2- 172.21.15.145:0/317147992 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f33dc06c480 0x7f33dc0cfda0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:52.593 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.592+0000 7f33e4000640 1 -- 172.21.15.145:0/317147992 >> 172.21.15.145:0/317147992 conn(0x7f33dc0b3ae0 msgr2=0x7f33dc15b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:52.593 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.592+0000 7f33e4000640 1 -- 172.21.15.145:0/317147992 shutdown_connections 2024-05-09T04:35:52.593 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.592+0000 7f33e4000640 1 -- 172.21.15.145:0/317147992 wait complete. 2024-05-09T04:35:52.611 INFO:tasks.ceph:Scrubbing osd.3 2024-05-09T04:35:52.612 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-05-09T04:35:52.808 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.808+0000 7efc2f200640 1 Processor -- start 2024-05-09T04:35:52.808 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.808+0000 7efc2f200640 1 -- start start 2024-05-09T04:35:52.810 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.810+0000 7efc2f200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efc30146970 0x7efc300cea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:52.810 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.810+0000 7efc2f200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7efc3006e060 0x7efc3015d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:52.810 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.810+0000 7efc2f200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc3015dc10 con 0x7efc300cd550 2024-05-09T04:35:52.810 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.810+0000 7efc2f200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7efc30072ee0 con 0x7efc3006e060 2024-05-09T04:35:52.810 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.810+0000 7efc2f200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7efc3015def0 con 0x7efc30146970 2024-05-09T04:35:52.811 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.811+0000 7efc2d400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7efc3006e060 0x7efc3015d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:52.811 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.811+0000 7efc2de00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efc30146970 0x7efc300cea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:52.812 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.811+0000 7efc2d400640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7efc3006e060 0x7efc3015d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.145:33430/0 (socket says 172.21.15.145:33430) 2024-05-09T04:35:52.812 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.811+0000 7efc2d400640 1 -- 172.21.15.145:0/2154767425 learned_addr learned my addr 172.21.15.145:0/2154767425 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:52.812 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.811+0000 7efc2e800640 1 --1- 172.21.15.145:0/2154767425 >> v1:172.21.15.192:6789/0 conn(0x7efc300cd550 0x7efc30073370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.145:44598/0 (socket says 172.21.15.145:44598) 2024-05-09T04:35:52.812 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.812+0000 7efc2de00640 1 -- 172.21.15.145:0/2154767425 >> v2:172.21.15.192:3300/0 conn(0x7efc3006e060 msgr2=0x7efc3015d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:52.812 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.812+0000 7efc2de00640 1 --2- 172.21.15.145:0/2154767425 >> v2:172.21.15.192:3300/0 conn(0x7efc3006e060 0x7efc3015d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:52.812 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.812+0000 7efc2de00640 1 -- 172.21.15.145:0/2154767425 >> v1:172.21.15.192:6789/0 conn(0x7efc300cd550 legacy=0x7efc30073370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:52.812 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.812+0000 7efc2de00640 1 -- 172.21.15.145:0/2154767425 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc3015e1d0 con 0x7efc30146970 2024-05-09T04:35:52.812 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.812+0000 7efc2d400640 1 --2- 172.21.15.145:0/2154767425 >> v2:172.21.15.192:3300/0 conn(0x7efc3006e060 0x7efc3015d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:35:52.813 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.813+0000 7efc2de00640 1 --2- 172.21.15.145:0/2154767425 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efc30146970 0x7efc300cea40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7efc20009d90 tx=0x7efc2002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c57c8c83d1f1298 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:52.813 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.814+0000 7efc2ca00640 1 -- 172.21.15.145:0/2154767425 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc20007740 con 0x7efc30146970 2024-05-09T04:35:52.814 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.814+0000 7efc2ca00640 1 -- 172.21.15.145:0/2154767425 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc200078e0 con 0x7efc30146970 2024-05-09T04:35:52.814 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.814+0000 7efc2ca00640 1 -- 172.21.15.145:0/2154767425 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc20007cb0 con 0x7efc30146970 2024-05-09T04:35:52.815 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.814+0000 7efc2f200640 1 -- 172.21.15.145:0/2154767425 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efc30146970 msgr2=0x7efc300cea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:52.815 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.814+0000 7efc2f200640 1 --2- 172.21.15.145:0/2154767425 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efc30146970 0x7efc300cea40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7efc20009d90 tx=0x7efc2002a660 comp rx=0 tx=0).stop 2024-05-09T04:35:52.815 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.815+0000 7efc2f200640 1 -- 172.21.15.145:0/2154767425 shutdown_connections 2024-05-09T04:35:52.815 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.815+0000 7efc2f200640 1 --2- 172.21.15.145:0/2154767425 >> v2:172.21.15.192:3300/0 conn(0x7efc3006e060 0x7efc3015d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:52.815 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.815+0000 7efc2f200640 1 --2- 172.21.15.145:0/2154767425 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efc30146970 0x7efc300cea40 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:52.815 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.815+0000 7efc2f200640 1 -- 172.21.15.145:0/2154767425 >> 172.21.15.145:0/2154767425 conn(0x7efc300b2850 msgr2=0x7efc300b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:52.815 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.815+0000 7efc2f200640 1 -- 172.21.15.145:0/2154767425 shutdown_connections 2024-05-09T04:35:52.816 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.815+0000 7efc2f200640 1 -- 172.21.15.145:0/2154767425 wait complete. 2024-05-09T04:35:52.816 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.816+0000 7efc2f200640 1 Processor -- start 2024-05-09T04:35:52.816 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.817+0000 7efc2f200640 1 -- start start 2024-05-09T04:35:52.819 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.817+0000 7efc2f200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efc3006e060 0x7efc301efd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:52.819 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.817+0000 7efc2f200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efc300cd550 0x7efc301f3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:52.819 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.817+0000 7efc2f200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efc30146970 0x7efc3013d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:52.819 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.818+0000 7efc2f200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7efc300ce7c0 con 0x7efc300cd550 2024-05-09T04:35:52.819 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.818+0000 7efc2f200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7efc300ce640 con 0x7efc30146970 2024-05-09T04:35:52.819 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.818+0000 7efc2f200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7efc300cd9e0 con 0x7efc3006e060 2024-05-09T04:35:52.821 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.818+0000 7efc2d400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efc300cd550 0x7efc301f3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:52.821 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.818+0000 7efc2d400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efc300cd550 0x7efc301f3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:53012/0 (socket says 172.21.15.145:53012) 2024-05-09T04:35:52.821 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.818+0000 7efc2d400640 1 -- 172.21.15.145:0/334301207 learned_addr learned my addr 172.21.15.145:0/334301207 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:52.822 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.818+0000 7efc2de00640 1 --2- 172.21.15.145:0/334301207 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efc3006e060 0x7efc301efd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:52.822 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.818+0000 7efc2e800640 1 --2- 172.21.15.145:0/334301207 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efc30146970 0x7efc3013d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:52.822 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.818+0000 7efc2d400640 1 -- 172.21.15.145:0/334301207 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efc3006e060 msgr2=0x7efc301efd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:52.822 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.818+0000 7efc2d400640 1 --2- 172.21.15.145:0/334301207 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efc3006e060 0x7efc301efd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:52.822 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.818+0000 7efc2d400640 1 -- 172.21.15.145:0/334301207 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efc30146970 msgr2=0x7efc3013d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:52.822 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.818+0000 7efc2d400640 1 --2- 172.21.15.145:0/334301207 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efc30146970 0x7efc3013d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:52.822 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.818+0000 7efc2d400640 1 -- 172.21.15.145:0/334301207 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc30139480 con 0x7efc300cd550 2024-05-09T04:35:52.822 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.819+0000 7efc2d400640 1 --2- 172.21.15.145:0/334301207 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efc300cd550 0x7efc301f3b40 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7efc1c00e9d0 tx=0x7efc1c00ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:52.823 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.819+0000 7efc26000640 1 -- 172.21.15.145:0/334301207 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1c017070 con 0x7efc300cd550 2024-05-09T04:35:52.823 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.819+0000 7efc26000640 1 -- 172.21.15.145:0/334301207 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc1c00ca60 con 0x7efc300cd550 2024-05-09T04:35:52.823 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.819+0000 7efc26000640 1 -- 172.21.15.145:0/334301207 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1c00ce10 con 0x7efc300cd550 2024-05-09T04:35:52.823 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.819+0000 7efc2f200640 1 -- 172.21.15.145:0/334301207 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc30139760 con 0x7efc300cd550 2024-05-09T04:35:52.823 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.819+0000 7efc2f200640 1 -- 172.21.15.145:0/334301207 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc30139d20 con 0x7efc300cd550 2024-05-09T04:35:52.824 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.820+0000 7efc2f200640 1 -- 172.21.15.145:0/334301207 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efc3013a120 con 0x7efc300cd550 2024-05-09T04:35:52.824 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.821+0000 7efc26000640 1 -- 172.21.15.145:0/334301207 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7efc1c01a070 con 0x7efc300cd550 2024-05-09T04:35:52.825 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.822+0000 7efc26000640 1 --2- 172.21.15.145:0/334301207 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7efc100762d0 0x7efc10078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:52.825 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.822+0000 7efc26000640 1 -- 172.21.15.145:0/334301207 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7efc1c097620 con 0x7efc300cd550 2024-05-09T04:35:52.825 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.823+0000 7efc26000640 1 --2- 172.21.15.145:0/334301207 >> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] conn(0x7efc1007c190 0x7efc1007e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:52.825 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.823+0000 7efc2de00640 1 --2- 172.21.15.145:0/334301207 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7efc100762d0 0x7efc10078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:52.825 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.823+0000 7efc26000640 1 -- 172.21.15.145:0/334301207 --> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efc1007ec60 con 0x7efc1007c190 2024-05-09T04:35:52.825 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.823+0000 7efc26000640 1 -- 172.21.15.145:0/334301207 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7efc1c09e050 con 0x7efc300cd550 2024-05-09T04:35:52.825 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.823+0000 7efc2e800640 1 --2- 172.21.15.145:0/334301207 >> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] conn(0x7efc1007c190 0x7efc1007e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:52.825 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.823+0000 7efc2de00640 1 --2- 172.21.15.145:0/334301207 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7efc100762d0 0x7efc10078790 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7efc20034000 tx=0x7efc20033040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:52.826 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.823+0000 7efc2e800640 1 --2- 172.21.15.145:0/334301207 >> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] conn(0x7efc1007c190 0x7efc1007e5b0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:52.826 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.824+0000 7efc26000640 1 -- 172.21.15.145:0/334301207 <== osd.3 v2:172.21.15.145:6808/337009826 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efc1007ec60 con 0x7efc1007c190 2024-05-09T04:35:52.838 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.839+0000 7efc2f200640 1 -- 172.21.15.145:0/334301207 --> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7efc30076930 con 0x7efc1007c190 2024-05-09T04:35:52.839 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.839+0000 7efc26000640 1 -- 172.21.15.145:0/334301207 <== osd.3 v2:172.21.15.145:6808/337009826 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7efc30076930 con 0x7efc1007c190 2024-05-09T04:35:52.839 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-05-09T04:35:52.839 INFO:teuthology.orchestra.run.smithi145.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-09T04:35:52.839 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-05-09T04:35:52.840 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.840+0000 7efc2f200640 1 -- 172.21.15.145:0/334301207 >> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] conn(0x7efc1007c190 msgr2=0x7efc1007e5b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:52.841 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.840+0000 7efc2f200640 1 --2- 172.21.15.145:0/334301207 >> [v2:172.21.15.145:6808/337009826,v1:172.21.15.145:6809/337009826] conn(0x7efc1007c190 0x7efc1007e5b0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:52.841 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.840+0000 7efc2f200640 1 -- 172.21.15.145:0/334301207 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7efc100762d0 msgr2=0x7efc10078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:52.841 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.840+0000 7efc2f200640 1 --2- 172.21.15.145:0/334301207 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7efc100762d0 0x7efc10078790 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7efc20034000 tx=0x7efc20033040 comp rx=0 tx=0).stop 2024-05-09T04:35:52.841 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.840+0000 7efc2f200640 1 -- 172.21.15.145:0/334301207 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efc300cd550 msgr2=0x7efc301f3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:52.841 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.840+0000 7efc2f200640 1 --2- 172.21.15.145:0/334301207 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7efc300cd550 0x7efc301f3b40 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7efc1c00e9d0 tx=0x7efc1c00ee90 comp rx=0 tx=0).stop 2024-05-09T04:35:52.841 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.840+0000 7efc2e800640 1 -- 172.21.15.145:0/334301207 reap_dead start 2024-05-09T04:35:52.841 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.840+0000 7efc2f200640 1 -- 172.21.15.145:0/334301207 shutdown_connections 2024-05-09T04:35:52.841 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.840+0000 7efc2f200640 1 -- 172.21.15.145:0/334301207 >> 172.21.15.145:0/334301207 conn(0x7efc300b2850 msgr2=0x7efc3015f510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:52.841 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.840+0000 7efc2f200640 1 -- 172.21.15.145:0/334301207 shutdown_connections 2024-05-09T04:35:52.842 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:52.840+0000 7efc2f200640 1 -- 172.21.15.145:0/334301207 wait complete. 2024-05-09T04:35:52.858 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-05-09T04:35:53.058 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.058+0000 7f1054e00640 1 Processor -- start 2024-05-09T04:35:53.058 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.059+0000 7f1054e00640 1 -- start start 2024-05-09T04:35:53.060 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.060+0000 7f1054e00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f105015d190 0x7f105006c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:53.060 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.060+0000 7f1054e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f10500d06f0 0x7f105006cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:53.060 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.060+0000 7f1054e00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10500b4ef0 con 0x7f10500cf210 2024-05-09T04:35:53.060 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.060+0000 7f1054e00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f10500b51d0 con 0x7f10500d06f0 2024-05-09T04:35:53.060 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.060+0000 7f1054e00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f10500b54b0 con 0x7f105015d190 2024-05-09T04:35:53.061 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.060+0000 7f104ea00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f10500d06f0 0x7f105006cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:53.061 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.060+0000 7f104ea00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f10500d06f0 0x7f105006cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:53018/0 (socket says 172.21.15.145:53018) 2024-05-09T04:35:53.061 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.060+0000 7f104ea00640 1 -- 172.21.15.145:0/1241054616 learned_addr learned my addr 172.21.15.145:0/1241054616 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:53.061 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.061+0000 7f104f400640 1 --2- 172.21.15.145:0/1241054616 >> v2:172.21.15.192:3300/0 conn(0x7f105015d190 0x7f105006c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:53.063 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.061+0000 7f104ea00640 1 -- 172.21.15.145:0/1241054616 >> v2:172.21.15.192:3300/0 conn(0x7f105015d190 msgr2=0x7f105006c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:53.063 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.061+0000 7f104ea00640 1 --2- 172.21.15.145:0/1241054616 >> v2:172.21.15.192:3300/0 conn(0x7f105015d190 0x7f105006c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:53.063 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.061+0000 7f104ea00640 1 -- 172.21.15.145:0/1241054616 >> v1:172.21.15.145:6789/0 conn(0x7f10500cf210 legacy=0x7f1050072140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:53.063 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.061+0000 7f104ea00640 1 -- 172.21.15.145:0/1241054616 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10500b5790 con 0x7f10500d06f0 2024-05-09T04:35:53.063 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.061+0000 7f104e000640 1 -- 172.21.15.145:0/1241054616 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 722924655 0 0) 0x7f10500b4ef0 con 0x7f10500cf210 2024-05-09T04:35:53.063 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.062+0000 7f104ea00640 1 --2- 172.21.15.145:0/1241054616 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10500d06f0 0x7f105006cde0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f10500731c0 tx=0x7f103802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44311655549f401d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:53.063 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.062+0000 7f104e000640 1 -- 172.21.15.145:0/1241054616 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1038036070 con 0x7f10500d06f0 2024-05-09T04:35:53.064 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.062+0000 7f104e000640 1 -- 172.21.15.145:0/1241054616 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1038031600 con 0x7f10500d06f0 2024-05-09T04:35:53.064 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.062+0000 7f104e000640 1 -- 172.21.15.145:0/1241054616 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10380319d0 con 0x7f10500d06f0 2024-05-09T04:35:53.064 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.063+0000 7f1054e00640 1 -- 172.21.15.145:0/1241054616 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10500d06f0 msgr2=0x7f105006cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:53.064 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.063+0000 7f1054e00640 1 --2- 172.21.15.145:0/1241054616 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10500d06f0 0x7f105006cde0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f10500731c0 tx=0x7f103802a730 comp rx=0 tx=0).stop 2024-05-09T04:35:53.064 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.063+0000 7f1054e00640 1 -- 172.21.15.145:0/1241054616 shutdown_connections 2024-05-09T04:35:53.064 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.063+0000 7f1054e00640 1 --2- 172.21.15.145:0/1241054616 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10500d06f0 0x7f105006cde0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:53.064 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.063+0000 7f1054e00640 1 --2- 172.21.15.145:0/1241054616 >> v2:172.21.15.192:3300/0 conn(0x7f105015d190 0x7f105006c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:53.064 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.063+0000 7f1054e00640 1 -- 172.21.15.145:0/1241054616 >> 172.21.15.145:0/1241054616 conn(0x7f10500b2d80 msgr2=0x7f10500b3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:53.064 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.063+0000 7f1054e00640 1 -- 172.21.15.145:0/1241054616 shutdown_connections 2024-05-09T04:35:53.065 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.063+0000 7f1054e00640 1 -- 172.21.15.145:0/1241054616 wait complete. 2024-05-09T04:35:53.067 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.065+0000 7f1054e00640 1 Processor -- start 2024-05-09T04:35:53.067 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.065+0000 7f1054e00640 1 -- start start 2024-05-09T04:35:53.067 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.066+0000 7f1054e00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10500cf210 0x7f10501f4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:53.067 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.066+0000 7f1054e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10500d06f0 0x7f105014bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:53.067 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.066+0000 7f1054e00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f105015d190 0x7f105006f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:53.067 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.066+0000 7f1054e00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f10500aeed0 con 0x7f10500d06f0 2024-05-09T04:35:53.067 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.066+0000 7f1054e00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f10500aed50 con 0x7f10500cf210 2024-05-09T04:35:53.068 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.066+0000 7f1054e00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f1050072850 con 0x7f105015d190 2024-05-09T04:35:53.068 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.066+0000 7f104ea00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10500d06f0 0x7f105014bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:53.068 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.066+0000 7f104ea00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10500d06f0 0x7f105014bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:53024/0 (socket says 172.21.15.145:53024) 2024-05-09T04:35:53.068 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.066+0000 7f104ea00640 1 -- 172.21.15.145:0/2542331467 learned_addr learned my addr 172.21.15.145:0/2542331467 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:53.068 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.066+0000 7f104ea00640 1 -- 172.21.15.145:0/2542331467 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f105015d190 msgr2=0x7f105006f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:53.068 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.066+0000 7f104ea00640 1 --2- 172.21.15.145:0/2542331467 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f105015d190 0x7f105006f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:53.068 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.066+0000 7f104ea00640 1 -- 172.21.15.145:0/2542331467 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10500cf210 msgr2=0x7f10501f4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:53.068 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.066+0000 7f104ea00640 1 --2- 172.21.15.145:0/2542331467 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10500cf210 0x7f10501f4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:53.068 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.066+0000 7f104ea00640 1 -- 172.21.15.145:0/2542331467 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f105014cf60 con 0x7f10500d06f0 2024-05-09T04:35:53.069 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.067+0000 7f104ea00640 1 --2- 172.21.15.145:0/2542331467 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10500d06f0 0x7f105014bfb0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f1038000f80 tx=0x7f1038004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:53.069 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.067+0000 7f1047400640 1 -- 172.21.15.145:0/2542331467 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1038036050 con 0x7f10500d06f0 2024-05-09T04:35:53.069 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.067+0000 7f1047400640 1 -- 172.21.15.145:0/2542331467 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1038042070 con 0x7f10500d06f0 2024-05-09T04:35:53.069 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.067+0000 7f1047400640 1 -- 172.21.15.145:0/2542331467 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1038009510 con 0x7f10500d06f0 2024-05-09T04:35:53.069 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.067+0000 7f1054e00640 1 -- 172.21.15.145:0/2542331467 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f105006fed0 con 0x7f10500d06f0 2024-05-09T04:35:53.069 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.067+0000 7f1054e00640 1 -- 172.21.15.145:0/2542331467 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f105014c9a0 con 0x7f10500d06f0 2024-05-09T04:35:53.070 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.068+0000 7f1054e00640 1 -- 172.21.15.145:0/2542331467 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10500cf9e0 con 0x7f10500d06f0 2024-05-09T04:35:53.070 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.069+0000 7f1047400640 1 -- 172.21.15.145:0/2542331467 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f1038007c70 con 0x7f10500d06f0 2024-05-09T04:35:53.070 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.070+0000 7f1047400640 1 --2- 172.21.15.145:0/2542331467 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f102c0762d0 0x7f102c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:53.070 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.070+0000 7f1047400640 1 -- 172.21.15.145:0/2542331467 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7f10380b8140 con 0x7f10500d06f0 2024-05-09T04:35:53.071 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.071+0000 7f104f400640 1 --2- 172.21.15.145:0/2542331467 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f102c0762d0 0x7f102c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:53.071 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.071+0000 7f104f400640 1 --2- 172.21.15.145:0/2542331467 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f102c0762d0 0x7f102c078790 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1040006fd0 tx=0x7f1040008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:53.074 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.075+0000 7f1047400640 1 -- 172.21.15.145:0/2542331467 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f10500cf9e0 con 0x7f10500d06f0 2024-05-09T04:35:53.280 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.281+0000 7f1054e00640 1 -- 172.21.15.145:0/2542331467 --> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f1050076760 con 0x7f102c0762d0 2024-05-09T04:35:53.281 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.281+0000 7f1047400640 1 -- 172.21.15.145:0/2542331467 <== mgr.4104 v2:172.21.15.145:6832/2725855396 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f1050076760 con 0x7f102c0762d0 2024-05-09T04:35:53.281 INFO:teuthology.orchestra.run.smithi145.stderr:instructed osd(s) 3 to deep-scrub 2024-05-09T04:35:53.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.284+0000 7f1054e00640 1 -- 172.21.15.145:0/2542331467 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f102c0762d0 msgr2=0x7f102c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:53.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.284+0000 7f1054e00640 1 --2- 172.21.15.145:0/2542331467 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f102c0762d0 0x7f102c078790 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1040006fd0 tx=0x7f1040008040 comp rx=0 tx=0).stop 2024-05-09T04:35:53.284 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.284+0000 7f1054e00640 1 -- 172.21.15.145:0/2542331467 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10500d06f0 msgr2=0x7f105014bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:53.285 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.284+0000 7f1054e00640 1 --2- 172.21.15.145:0/2542331467 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10500d06f0 0x7f105014bfb0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f1038000f80 tx=0x7f1038004a40 comp rx=0 tx=0).stop 2024-05-09T04:35:53.285 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.284+0000 7f1054e00640 1 -- 172.21.15.145:0/2542331467 shutdown_connections 2024-05-09T04:35:53.285 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.284+0000 7f1054e00640 1 --2- 172.21.15.145:0/2542331467 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f102c0762d0 0x7f102c078790 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:53.285 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.284+0000 7f1054e00640 1 --2- 172.21.15.145:0/2542331467 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f105015d190 0x7f105006f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:53.285 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.284+0000 7f1054e00640 1 --2- 172.21.15.145:0/2542331467 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f10500d06f0 0x7f105014bfb0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:53.285 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.284+0000 7f1054e00640 1 --2- 172.21.15.145:0/2542331467 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10500cf210 0x7f10501f4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:53.285 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.284+0000 7f1054e00640 1 -- 172.21.15.145:0/2542331467 >> 172.21.15.145:0/2542331467 conn(0x7f10500b2d80 msgr2=0x7f105009e240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:53.285 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.285+0000 7f1054e00640 1 -- 172.21.15.145:0/2542331467 shutdown_connections 2024-05-09T04:35:53.285 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.285+0000 7f1054e00640 1 -- 172.21.15.145:0/2542331467 wait complete. 2024-05-09T04:35:53.302 INFO:tasks.ceph:Scrubbing osd.4 2024-05-09T04:35:53.303 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-05-09T04:35:53.510 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.510+0000 7fb921600640 1 Processor -- start 2024-05-09T04:35:53.511 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.511+0000 7fb921600640 1 -- start start 2024-05-09T04:35:53.512 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.512+0000 7fb921600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb91c144c50 0x7fb91c0b3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:53.512 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.512+0000 7fb921600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb91c1455f0 con 0x7fb91c070230 2024-05-09T04:35:53.512 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.512+0000 7fb921600640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb91c06efd0 con 0x7fb91c06e510 2024-05-09T04:35:53.512 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.512+0000 7fb921600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fb91c06f2b0 con 0x7fb91c144c50 2024-05-09T04:35:53.512 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.513+0000 7fb91b400640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7fb91c070230 0x7fb91c073cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:60556/0 (socket says 172.21.15.145:60556) 2024-05-09T04:35:53.513 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.513+0000 7fb91b400640 1 -- 172.21.15.145:0/2582317084 learned_addr learned my addr 172.21.15.145:0/2582317084 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:35:53.515 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.513+0000 7fb91aa00640 1 --2- 172.21.15.145:0/2582317084 >> v2:172.21.15.192:3300/0 conn(0x7fb91c144c50 0x7fb91c0b3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:53.515 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.513+0000 7fb91a000640 1 -- 172.21.15.145:0/2582317084 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1061718160 0 0) 0x7fb91c1455f0 con 0x7fb91c070230 2024-05-09T04:35:53.515 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.513+0000 7fb91a000640 1 -- 172.21.15.145:0/2582317084 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb900005730 con 0x7fb91c070230 2024-05-09T04:35:53.515 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.513+0000 7fb91a000640 1 -- 172.21.15.145:0/2582317084 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2705883469 0 0) 0x7fb900005730 con 0x7fb91c070230 2024-05-09T04:35:53.515 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.513+0000 7fb91a000640 1 -- 172.21.15.145:0/2582317084 >> v2:172.21.15.192:3300/0 conn(0x7fb91c144c50 msgr2=0x7fb91c0b3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:53.515 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.513+0000 7fb91a000640 1 --2- 172.21.15.145:0/2582317084 >> v2:172.21.15.192:3300/0 conn(0x7fb91c144c50 0x7fb91c0b3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:53.516 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.513+0000 7fb91a000640 1 -- 172.21.15.145:0/2582317084 >> v1:172.21.15.192:6789/0 conn(0x7fb91c06e510 legacy=0x7fb91c06fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:53.516 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.514+0000 7fb91a000640 1 -- 172.21.15.145:0/2582317084 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb91c06f590 con 0x7fb91c070230 2024-05-09T04:35:53.516 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.514+0000 7fb91a000640 1 -- 172.21.15.145:0/2582317084 <== mon.0 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3474413261 0 0) 0x7fb914001a90 con 0x7fb91c070230 2024-05-09T04:35:53.516 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.514+0000 7fb91a000640 1 -- 172.21.15.145:0/2582317084 >> v1:172.21.15.145:6789/0 conn(0x7fb91c070230 legacy=0x7fb91c073cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:53.516 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.514+0000 7fb91a000640 1 --2- 172.21.15.145:0/2582317084 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb900005c60 0x7fb900006080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:53.516 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.514+0000 7fb91a000640 1 --2- 172.21.15.145:0/2582317084 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb900006a10 0x7fb900006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:53.516 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.514+0000 7fb91a000640 1 --2- 172.21.15.145:0/2582317084 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb9000073d0 0x7fb90000dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:53.516 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.514+0000 7fb91a000640 1 -- 172.21.15.145:0/2582317084 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb91c06f2b0 con 0x7fb900006a10 2024-05-09T04:35:53.516 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.514+0000 7fb91a000640 1 -- 172.21.15.145:0/2582317084 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb90000e370 con 0x7fb900005c60 2024-05-09T04:35:53.517 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.514+0000 7fb91a000640 1 -- 172.21.15.145:0/2582317084 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb90000e570 con 0x7fb9000073d0 2024-05-09T04:35:53.517 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.514+0000 7fb912000640 1 --2- 172.21.15.145:0/2582317084 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb900006a10 0x7fb900006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:53.517 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.514+0000 7fb91aa00640 1 --2- 172.21.15.145:0/2582317084 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb900005c60 0x7fb900006080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:53.517 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.514+0000 7fb912000640 1 -- 172.21.15.145:0/2582317084 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb9000073d0 msgr2=0x7fb90000dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:53.517 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.514+0000 7fb912000640 1 --2- 172.21.15.145:0/2582317084 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb9000073d0 0x7fb90000dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:53.517 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.514+0000 7fb912000640 1 -- 172.21.15.145:0/2582317084 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb900005c60 msgr2=0x7fb900006080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:53.517 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.514+0000 7fb912000640 1 --2- 172.21.15.145:0/2582317084 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb900005c60 0x7fb900006080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:53.517 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.514+0000 7fb912000640 1 -- 172.21.15.145:0/2582317084 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb900005730 con 0x7fb900006a10 2024-05-09T04:35:53.517 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.514+0000 7fb91b400640 1 -- 172.21.15.145:0/2582317084 reap_dead start 2024-05-09T04:35:53.518 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.515+0000 7fb912000640 1 --2- 172.21.15.145:0/2582317084 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb900006a10 0x7fb900006e90 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fb904001790 tx=0x7fb900007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d85b8a4d62524d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:53.520 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.516+0000 7fb91a000640 1 -- 172.21.15.145:0/2582317084 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9040013b0 con 0x7fb900006a10 2024-05-09T04:35:53.520 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.516+0000 7fb91a000640 1 -- 172.21.15.145:0/2582317084 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9040052d0 con 0x7fb900006a10 2024-05-09T04:35:53.521 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.516+0000 7fb91a000640 1 -- 172.21.15.145:0/2582317084 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90402aac0 con 0x7fb900006a10 2024-05-09T04:35:53.521 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.516+0000 7fb921600640 1 -- 172.21.15.145:0/2582317084 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb900006a10 msgr2=0x7fb900006e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:53.521 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.516+0000 7fb921600640 1 --2- 172.21.15.145:0/2582317084 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb900006a10 0x7fb900006e90 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fb904001790 tx=0x7fb900007a40 comp rx=0 tx=0).stop 2024-05-09T04:35:53.521 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.516+0000 7fb921600640 1 -- 172.21.15.145:0/2582317084 shutdown_connections 2024-05-09T04:35:53.521 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.516+0000 7fb921600640 1 --2- 172.21.15.145:0/2582317084 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb900006a10 0x7fb900006e90 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:53.521 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.516+0000 7fb921600640 1 -- 172.21.15.145:0/2582317084 >> 172.21.15.145:0/2582317084 conn(0x7fb91c06c8d0 msgr2=0x7fb91c06cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:53.521 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.516+0000 7fb921600640 1 -- 172.21.15.145:0/2582317084 shutdown_connections 2024-05-09T04:35:53.521 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.517+0000 7fb921600640 1 -- 172.21.15.145:0/2582317084 wait complete. 2024-05-09T04:35:53.521 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.518+0000 7fb921600640 1 Processor -- start 2024-05-09T04:35:53.522 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.518+0000 7fb921600640 1 -- start start 2024-05-09T04:35:53.522 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.519+0000 7fb921600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb900006a10 0x7fb91c15eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:53.522 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.519+0000 7fb921600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb91c15b590 0x7fb91c15c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:53.522 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.519+0000 7fb921600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb91c0b4470 0x7fb91c0b4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:53.522 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.519+0000 7fb921600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fb91c145030 con 0x7fb91c0b4470 2024-05-09T04:35:53.522 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.519+0000 7fb921600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fb91c073460 con 0x7fb900006a10 2024-05-09T04:35:53.522 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.519+0000 7fb921600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fb91c140a00 con 0x7fb91c15b590 2024-05-09T04:35:53.522 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.520+0000 7fb91b400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb91c0b4470 0x7fb91c0b4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:53.523 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.520+0000 7fb91b400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb91c0b4470 0x7fb91c0b4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:53050/0 (socket says 172.21.15.145:53050) 2024-05-09T04:35:53.523 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.520+0000 7fb91b400640 1 -- 172.21.15.145:0/3671135036 learned_addr learned my addr 172.21.15.145:0/3671135036 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:53.523 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.520+0000 7fb91aa00640 1 --2- 172.21.15.145:0/3671135036 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb900006a10 0x7fb91c15eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:53.523 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.520+0000 7fb912000640 1 --2- 172.21.15.145:0/3671135036 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb91c15b590 0x7fb91c15c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:53.523 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.520+0000 7fb91b400640 1 -- 172.21.15.145:0/3671135036 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb91c15b590 msgr2=0x7fb91c15c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:53.523 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.520+0000 7fb91b400640 1 --2- 172.21.15.145:0/3671135036 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb91c15b590 0x7fb91c15c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:53.523 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.520+0000 7fb91b400640 1 -- 172.21.15.145:0/3671135036 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb900006a10 msgr2=0x7fb91c15eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:53.523 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.520+0000 7fb91b400640 1 --2- 172.21.15.145:0/3671135036 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb900006a10 0x7fb91c15eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:53.523 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.520+0000 7fb91b400640 1 -- 172.21.15.145:0/3671135036 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb91c15cc30 con 0x7fb91c0b4470 2024-05-09T04:35:53.524 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.520+0000 7fb912000640 1 --2- 172.21.15.145:0/3671135036 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb91c15b590 0x7fb91c15c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:35:53.524 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.520+0000 7fb91b400640 1 --2- 172.21.15.145:0/3671135036 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb91c0b4470 0x7fb91c0b4920 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fb914001370 tx=0x7fb91400ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:53.524 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.521+0000 7fb913400640 1 -- 172.21.15.145:0/3671135036 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91400a610 con 0x7fb91c0b4470 2024-05-09T04:35:53.524 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.521+0000 7fb913400640 1 -- 172.21.15.145:0/3671135036 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb91400a7b0 con 0x7fb91c0b4470 2024-05-09T04:35:53.524 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.521+0000 7fb913400640 1 -- 172.21.15.145:0/3671135036 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91400ab60 con 0x7fb91c0b4470 2024-05-09T04:35:53.524 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.521+0000 7fb921600640 1 -- 172.21.15.145:0/3671135036 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb91c15ba10 con 0x7fb91c0b4470 2024-05-09T04:35:53.524 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.521+0000 7fb921600640 1 -- 172.21.15.145:0/3671135036 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb91c13c4d0 con 0x7fb91c0b4470 2024-05-09T04:35:53.526 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.522+0000 7fb921600640 1 -- 172.21.15.145:0/3671135036 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb91c06a310 con 0x7fb91c0b4470 2024-05-09T04:35:53.526 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.524+0000 7fb913400640 1 -- 172.21.15.145:0/3671135036 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fb914011070 con 0x7fb91c0b4470 2024-05-09T04:35:53.526 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.525+0000 7fb913400640 1 --2- 172.21.15.145:0/3671135036 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fb8f8076300 0x7fb8f80787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:53.526 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.525+0000 7fb913400640 1 -- 172.21.15.145:0/3671135036 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7fb914096880 con 0x7fb91c0b4470 2024-05-09T04:35:53.526 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.525+0000 7fb91aa00640 1 --2- 172.21.15.145:0/3671135036 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fb8f8076300 0x7fb8f80787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:53.526 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.525+0000 7fb913400640 1 --2- 172.21.15.145:0/3671135036 >> [v2:172.21.15.192:6816/3117700995,v1:172.21.15.192:6818/3117700995] conn(0x7fb8f807c1c0 0x7fb8f807e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:53.526 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.525+0000 7fb913400640 1 -- 172.21.15.145:0/3671135036 --> [v2:172.21.15.192:6816/3117700995,v1:172.21.15.192:6818/3117700995] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb914016e50 con 0x7fb8f807c1c0 2024-05-09T04:35:53.527 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.525+0000 7fb913400640 1 -- 172.21.15.145:0/3671135036 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fb914096d10 con 0x7fb91c0b4470 2024-05-09T04:35:53.527 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.526+0000 7fb91aa00640 1 --2- 172.21.15.145:0/3671135036 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fb8f8076300 0x7fb8f80787c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb91c0521a0 tx=0x7fb90c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:53.527 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.526+0000 7fb912000640 1 --2- 172.21.15.145:0/3671135036 >> [v2:172.21.15.192:6816/3117700995,v1:172.21.15.192:6818/3117700995] conn(0x7fb8f807c1c0 0x7fb8f807e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:53.527 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.527+0000 7fb912000640 1 --2- 172.21.15.145:0/3671135036 >> [v2:172.21.15.192:6816/3117700995,v1:172.21.15.192:6818/3117700995] conn(0x7fb8f807c1c0 0x7fb8f807e5e0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:53.528 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.529+0000 7fb913400640 1 -- 172.21.15.145:0/3671135036 <== osd.4 v2:172.21.15.192:6816/3117700995 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb914016e50 con 0x7fb8f807c1c0 2024-05-09T04:35:53.544 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.545+0000 7fb921600640 1 -- 172.21.15.145:0/3671135036 --> [v2:172.21.15.192:6816/3117700995,v1:172.21.15.192:6818/3117700995] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fb91c0b1560 con 0x7fb8f807c1c0 2024-05-09T04:35:53.545 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.545+0000 7fb913400640 1 -- 172.21.15.145:0/3671135036 <== osd.4 v2:172.21.15.192:6816/3117700995 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fb91c0b1560 con 0x7fb8f807c1c0 2024-05-09T04:35:53.546 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-05-09T04:35:53.546 INFO:teuthology.orchestra.run.smithi145.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-09T04:35:53.546 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-05-09T04:35:53.547 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.546+0000 7fb921600640 1 -- 172.21.15.145:0/3671135036 >> [v2:172.21.15.192:6816/3117700995,v1:172.21.15.192:6818/3117700995] conn(0x7fb8f807c1c0 msgr2=0x7fb8f807e5e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:53.547 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.546+0000 7fb921600640 1 --2- 172.21.15.145:0/3671135036 >> [v2:172.21.15.192:6816/3117700995,v1:172.21.15.192:6818/3117700995] conn(0x7fb8f807c1c0 0x7fb8f807e5e0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:53.548 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.546+0000 7fb921600640 1 -- 172.21.15.145:0/3671135036 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fb8f8076300 msgr2=0x7fb8f80787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:53.548 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.546+0000 7fb921600640 1 --2- 172.21.15.145:0/3671135036 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fb8f8076300 0x7fb8f80787c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb91c0521a0 tx=0x7fb90c008040 comp rx=0 tx=0).stop 2024-05-09T04:35:53.548 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.546+0000 7fb921600640 1 -- 172.21.15.145:0/3671135036 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb91c0b4470 msgr2=0x7fb91c0b4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:53.548 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.546+0000 7fb921600640 1 --2- 172.21.15.145:0/3671135036 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fb91c0b4470 0x7fb91c0b4920 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fb914001370 tx=0x7fb91400ccd0 comp rx=0 tx=0).stop 2024-05-09T04:35:53.548 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.546+0000 7fb91b400640 1 -- 172.21.15.145:0/3671135036 reap_dead start 2024-05-09T04:35:53.548 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.546+0000 7fb921600640 1 -- 172.21.15.145:0/3671135036 shutdown_connections 2024-05-09T04:35:53.548 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.546+0000 7fb921600640 1 -- 172.21.15.145:0/3671135036 >> 172.21.15.145:0/3671135036 conn(0x7fb91c06c8d0 msgr2=0x7fb91c0cdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:53.549 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.546+0000 7fb921600640 1 -- 172.21.15.145:0/3671135036 shutdown_connections 2024-05-09T04:35:53.549 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.546+0000 7fb921600640 1 -- 172.21.15.145:0/3671135036 wait complete. 2024-05-09T04:35:53.567 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-05-09T04:35:53.768 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.768+0000 7f66c1200640 1 Processor -- start 2024-05-09T04:35:53.769 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.768+0000 7f66c1200640 1 -- start start 2024-05-09T04:35:53.773 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.770+0000 7f66c1200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f66bc0ce060 0x7f66bc06e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:53.773 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.770+0000 7f66c1200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66bc06ebd0 con 0x7f66bc145ad0 2024-05-09T04:35:53.773 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.770+0000 7f66c1200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66bc06eeb0 con 0x7f66bc0ce440 2024-05-09T04:35:53.773 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.770+0000 7f66c1200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f66bc073460 con 0x7f66bc0ce060 2024-05-09T04:35:53.773 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.771+0000 7f66ba800640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f66bc145ad0 0x7f66bc0738d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:60570/0 (socket says 172.21.15.145:60570) 2024-05-09T04:35:53.774 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.771+0000 7f66ba800640 1 -- 172.21.15.145:0/590727671 learned_addr learned my addr 172.21.15.145:0/590727671 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:35:53.774 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.771+0000 7f66b9e00640 1 --2- 172.21.15.145:0/590727671 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f66bc0ce060 0x7f66bc06e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:53.774 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.771+0000 7f66b9400640 1 -- 172.21.15.145:0/590727671 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1003880075 0 0) 0x7f66bc06ebd0 con 0x7f66bc145ad0 2024-05-09T04:35:53.774 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.771+0000 7f66b9400640 1 -- 172.21.15.145:0/590727671 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f66a0003520 con 0x7f66bc145ad0 2024-05-09T04:35:53.774 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.771+0000 7f66b9400640 1 -- 172.21.15.145:0/590727671 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4286794627 0 0) 0x7f66a0003520 con 0x7f66bc145ad0 2024-05-09T04:35:53.775 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.771+0000 7f66b9400640 1 -- 172.21.15.145:0/590727671 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f66bc0ce060 msgr2=0x7f66bc06e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:53.775 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.771+0000 7f66b9400640 1 --2- 172.21.15.145:0/590727671 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f66bc0ce060 0x7f66bc06e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:53.775 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.771+0000 7f66b9400640 1 -- 172.21.15.145:0/590727671 >> v1:172.21.15.192:6789/0 conn(0x7f66bc0ce440 legacy=0x7f66bc1453c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:53.775 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.772+0000 7f66b9400640 1 -- 172.21.15.145:0/590727671 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66bc06f190 con 0x7f66bc145ad0 2024-05-09T04:35:53.775 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.772+0000 7f66b9400640 1 -- 172.21.15.145:0/590727671 <== mon.0 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3474413261 0 0) 0x7f66b4001a70 con 0x7f66bc145ad0 2024-05-09T04:35:53.775 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.772+0000 7f66b9400640 1 -- 172.21.15.145:0/590727671 >> v1:172.21.15.145:6789/0 conn(0x7f66bc145ad0 legacy=0x7f66bc0738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:53.775 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.772+0000 7f66b9400640 1 --2- 172.21.15.145:0/590727671 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66a0003a30 0x7f66a0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:53.776 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.772+0000 7f66b9400640 1 --2- 172.21.15.145:0/590727671 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f66bc0ce060 0x7f66a0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:53.776 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.772+0000 7f66b9400640 1 --2- 172.21.15.145:0/590727671 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66a0004d40 0x7f66a00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:53.776 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.772+0000 7f66b9400640 1 -- 172.21.15.145:0/590727671 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f66a000dcd0 con 0x7f66a0003a30 2024-05-09T04:35:53.776 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.772+0000 7f66b9400640 1 -- 172.21.15.145:0/590727671 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f66a000de80 con 0x7f66a0004d40 2024-05-09T04:35:53.776 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.772+0000 7f66b9400640 1 -- 172.21.15.145:0/590727671 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f66a000e080 con 0x7f66bc0ce060 2024-05-09T04:35:53.777 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.772+0000 7f66b9e00640 1 --2- 172.21.15.145:0/590727671 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66a0003a30 0x7f66a0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:53.777 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.772+0000 7f66b9e00640 1 -- 172.21.15.145:0/590727671 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f66bc0ce060 msgr2=0x7f66a0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:53.777 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.772+0000 7f66b9e00640 1 --2- 172.21.15.145:0/590727671 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f66bc0ce060 0x7f66a0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:53.777 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.772+0000 7f66ba800640 1 --2- 172.21.15.145:0/590727671 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66a0004d40 0x7f66a00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:53.777 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.772+0000 7f66b9e00640 1 -- 172.21.15.145:0/590727671 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66a0004d40 msgr2=0x7f66a00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:53.777 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.772+0000 7f66b9e00640 1 --2- 172.21.15.145:0/590727671 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66a0004d40 0x7f66a00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:53.778 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.772+0000 7f66b9e00640 1 -- 172.21.15.145:0/590727671 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a0003520 con 0x7f66a0003a30 2024-05-09T04:35:53.778 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.772+0000 7f66ba800640 1 --2- 172.21.15.145:0/590727671 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66a0004d40 0x7f66a00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:35:53.781 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.774+0000 7f66b9e00640 1 --2- 172.21.15.145:0/590727671 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66a0003a30 0x7f66a0003e50 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f66ac009ac0 tx=0x7f66a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52c3401e56417fcf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:53.781 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.774+0000 7f66b9400640 1 -- 172.21.15.145:0/590727671 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ac036070 con 0x7f66a0003a30 2024-05-09T04:35:53.781 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.774+0000 7f66b9400640 1 -- 172.21.15.145:0/590727671 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66ac031400 con 0x7f66a0003a30 2024-05-09T04:35:53.781 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.774+0000 7f66b9400640 1 -- 172.21.15.145:0/590727671 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ac009690 con 0x7f66a0003a30 2024-05-09T04:35:53.781 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.774+0000 7f66c1200640 1 -- 172.21.15.145:0/590727671 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66a0003a30 msgr2=0x7f66a0003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:53.782 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.774+0000 7f66c1200640 1 --2- 172.21.15.145:0/590727671 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66a0003a30 0x7f66a0003e50 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f66ac009ac0 tx=0x7f66a0005830 comp rx=0 tx=0).stop 2024-05-09T04:35:53.782 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.774+0000 7f66ba800640 1 -- 172.21.15.145:0/590727671 reap_dead start 2024-05-09T04:35:53.782 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.774+0000 7f66c1200640 1 -- 172.21.15.145:0/590727671 shutdown_connections 2024-05-09T04:35:53.782 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.775+0000 7f66c1200640 1 -- 172.21.15.145:0/590727671 >> 172.21.15.145:0/590727671 conn(0x7f66bc06c8d0 msgr2=0x7f66bc06cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:53.782 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.775+0000 7f66c1200640 1 -- 172.21.15.145:0/590727671 shutdown_connections 2024-05-09T04:35:53.782 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.775+0000 7f66c1200640 1 -- 172.21.15.145:0/590727671 wait complete. 2024-05-09T04:35:53.782 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.776+0000 7f66c1200640 1 Processor -- start 2024-05-09T04:35:53.783 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.776+0000 7f66c1200640 1 -- start start 2024-05-09T04:35:53.783 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.777+0000 7f66c1200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66a0003a30 0x7f66bc1417b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:53.783 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.777+0000 7f66c1200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66bc157c20 0x7f66bc0af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:53.783 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.777+0000 7f66c1200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f66bc0afe10 0x7f66bc0b3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:53.783 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.777+0000 7f66c1200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f66bc15da40 con 0x7f66a0003a30 2024-05-09T04:35:53.783 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.777+0000 7f66c1200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f66bc15d8c0 con 0x7f66bc157c20 2024-05-09T04:35:53.783 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.777+0000 7f66c1200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f66bc06e510 con 0x7f66bc0afe10 2024-05-09T04:35:53.783 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.778+0000 7f66b9e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66a0003a30 0x7f66bc1417b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:53.784 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.778+0000 7f66b9e00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66a0003a30 0x7f66bc1417b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:53076/0 (socket says 172.21.15.145:53076) 2024-05-09T04:35:53.784 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.778+0000 7f66b9e00640 1 -- 172.21.15.145:0/3429535852 learned_addr learned my addr 172.21.15.145:0/3429535852 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:53.784 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.778+0000 7f66b3e00640 1 --2- 172.21.15.145:0/3429535852 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66bc157c20 0x7f66bc0af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:53.784 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.778+0000 7f66b9e00640 1 -- 172.21.15.145:0/3429535852 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f66bc0afe10 msgr2=0x7f66bc0b3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:53.784 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.778+0000 7f66b9e00640 1 --2- 172.21.15.145:0/3429535852 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f66bc0afe10 0x7f66bc0b3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:53.784 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.778+0000 7f66b9e00640 1 -- 172.21.15.145:0/3429535852 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66bc157c20 msgr2=0x7f66bc0af8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:53.784 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.778+0000 7f66b9e00640 1 --2- 172.21.15.145:0/3429535852 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66bc157c20 0x7f66bc0af8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:53.784 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.778+0000 7f66b9e00640 1 -- 172.21.15.145:0/3429535852 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66bc141cf0 con 0x7f66a0003a30 2024-05-09T04:35:53.784 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.778+0000 7f66b9e00640 1 --2- 172.21.15.145:0/3429535852 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66a0003a30 0x7f66bc1417b0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f66ac034040 tx=0x7f66ac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:53.785 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.779+0000 7f66b2000640 1 -- 172.21.15.145:0/3429535852 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ac040020 con 0x7f66a0003a30 2024-05-09T04:35:53.785 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.779+0000 7f66b2000640 1 -- 172.21.15.145:0/3429535852 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66ac003940 con 0x7f66a0003a30 2024-05-09T04:35:53.785 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.779+0000 7f66c1200640 1 -- 172.21.15.145:0/3429535852 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66bc156e60 con 0x7f66a0003a30 2024-05-09T04:35:53.785 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.779+0000 7f66b2000640 1 -- 172.21.15.145:0/3429535852 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66ac036040 con 0x7f66a0003a30 2024-05-09T04:35:53.785 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.779+0000 7f66c1200640 1 -- 172.21.15.145:0/3429535852 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66bc157420 con 0x7f66a0003a30 2024-05-09T04:35:53.785 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.780+0000 7f66c1200640 1 -- 172.21.15.145:0/3429535852 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66bc0b0230 con 0x7f66a0003a30 2024-05-09T04:35:53.785 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.781+0000 7f66b2000640 1 -- 172.21.15.145:0/3429535852 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f66ac043020 con 0x7f66a0003a30 2024-05-09T04:35:53.786 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.783+0000 7f66b2000640 1 --2- 172.21.15.145:0/3429535852 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f6698076300 0x7f66980787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:53.786 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.783+0000 7f66b2000640 1 -- 172.21.15.145:0/3429535852 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7f66ac0b7480 con 0x7f66a0003a30 2024-05-09T04:35:53.786 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.783+0000 7f66b3e00640 1 --2- 172.21.15.145:0/3429535852 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f6698076300 0x7f66980787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:53.787 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.784+0000 7f66b3e00640 1 --2- 172.21.15.145:0/3429535852 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f6698076300 0x7f66980787c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f66bc158340 tx=0x7f66a4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:53.787 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.787+0000 7f66b2000640 1 -- 172.21.15.145:0/3429535852 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f66ac04a070 con 0x7f66a0003a30 2024-05-09T04:35:53.994 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.994+0000 7f66c1200640 1 -- 172.21.15.145:0/3429535852 --> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f66bc06a310 con 0x7f6698076300 2024-05-09T04:35:53.994 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.995+0000 7f66b2000640 1 -- 172.21.15.145:0/3429535852 <== mgr.4104 v2:172.21.15.145:6832/2725855396 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f66bc06a310 con 0x7f6698076300 2024-05-09T04:35:53.994 INFO:teuthology.orchestra.run.smithi145.stderr:instructed osd(s) 4 to deep-scrub 2024-05-09T04:35:53.998 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.997+0000 7f66c1200640 1 -- 172.21.15.145:0/3429535852 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f6698076300 msgr2=0x7f66980787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:53.998 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.997+0000 7f66c1200640 1 --2- 172.21.15.145:0/3429535852 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f6698076300 0x7f66980787c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f66bc158340 tx=0x7f66a4009040 comp rx=0 tx=0).stop 2024-05-09T04:35:53.998 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.997+0000 7f66c1200640 1 -- 172.21.15.145:0/3429535852 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66a0003a30 msgr2=0x7f66bc1417b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:53.998 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.997+0000 7f66c1200640 1 --2- 172.21.15.145:0/3429535852 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66a0003a30 0x7f66bc1417b0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f66ac034040 tx=0x7f66ac004730 comp rx=0 tx=0).stop 2024-05-09T04:35:53.998 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.998+0000 7f66c1200640 1 -- 172.21.15.145:0/3429535852 shutdown_connections 2024-05-09T04:35:53.998 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.998+0000 7f66c1200640 1 --2- 172.21.15.145:0/3429535852 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f6698076300 0x7f66980787c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:53.999 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.998+0000 7f66c1200640 1 --2- 172.21.15.145:0/3429535852 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f66bc0afe10 0x7f66bc0b3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:53.999 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.998+0000 7f66c1200640 1 --2- 172.21.15.145:0/3429535852 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66bc157c20 0x7f66bc0af8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:53.999 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.998+0000 7f66c1200640 1 --2- 172.21.15.145:0/3429535852 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f66a0003a30 0x7f66bc1417b0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:53.999 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.998+0000 7f66c1200640 1 -- 172.21.15.145:0/3429535852 >> 172.21.15.145:0/3429535852 conn(0x7f66bc06c8d0 msgr2=0x7f66bc15c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:53.999 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.998+0000 7f66c1200640 1 -- 172.21.15.145:0/3429535852 shutdown_connections 2024-05-09T04:35:53.999 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:53.998+0000 7f66c1200640 1 -- 172.21.15.145:0/3429535852 wait complete. 2024-05-09T04:35:54.017 INFO:tasks.ceph:Scrubbing osd.5 2024-05-09T04:35:54.017 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-05-09T04:35:54.227 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.227+0000 7f033f200640 1 Processor -- start 2024-05-09T04:35:54.227 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.227+0000 7f033f200640 1 -- start start 2024-05-09T04:35:54.228 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.229+0000 7f033f200640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f0340145860 0x7f034015c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:54.229 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.229+0000 7f033f200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f03400b3420 0x7f0340072060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:54.229 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.229+0000 7f033f200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f034015ce50 con 0x7f03400725a0 2024-05-09T04:35:54.229 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.229+0000 7f033f200640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f034015d410 con 0x7f0340145860 2024-05-09T04:35:54.229 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.229+0000 7f033f200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f034006e9d0 con 0x7f03400b3420 2024-05-09T04:35:54.229 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.229+0000 7f033e800640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f03400725a0 0x7f034006ee60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:60572/0 (socket says 172.21.15.145:60572) 2024-05-09T04:35:54.229 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.229+0000 7f033e800640 1 -- 172.21.15.145:0/365264550 learned_addr learned my addr 172.21.15.145:0/365264550 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:35:54.231 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.229+0000 7f033de00640 1 --2- 172.21.15.145:0/365264550 >> v2:172.21.15.192:3300/0 conn(0x7f0340145860 0x7f034015c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:54.232 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.229+0000 7f0335400640 1 --2- 172.21.15.145:0/365264550 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f03400b3420 0x7f0340072060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:54.232 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.230+0000 7f033d400640 1 -- 172.21.15.145:0/365264550 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 400283542 0 0) 0x7f034015ce50 con 0x7f03400725a0 2024-05-09T04:35:54.232 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.230+0000 7f033d400640 1 -- 172.21.15.145:0/365264550 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0324005730 con 0x7f03400725a0 2024-05-09T04:35:54.232 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.230+0000 7f033d400640 1 -- 172.21.15.145:0/365264550 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3978451948 0 0) 0x7f0324005730 con 0x7f03400725a0 2024-05-09T04:35:54.232 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.230+0000 7f033d400640 1 -- 172.21.15.145:0/365264550 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f03400b3420 msgr2=0x7f0340072060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:54.232 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.230+0000 7f033d400640 1 --2- 172.21.15.145:0/365264550 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f03400b3420 0x7f0340072060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:54.232 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.230+0000 7f033d400640 1 -- 172.21.15.145:0/365264550 >> v2:172.21.15.192:3300/0 conn(0x7f0340145860 msgr2=0x7f034015c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:54.232 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.230+0000 7f033d400640 1 --2- 172.21.15.145:0/365264550 >> v2:172.21.15.192:3300/0 conn(0x7f0340145860 0x7f034015c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:54.232 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.230+0000 7f033d400640 1 -- 172.21.15.145:0/365264550 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f034015d130 con 0x7f03400725a0 2024-05-09T04:35:54.233 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.230+0000 7f033d400640 1 -- 172.21.15.145:0/365264550 <== mon.0 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3474413261 0 0) 0x7f0338001710 con 0x7f03400725a0 2024-05-09T04:35:54.233 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.230+0000 7f033d400640 1 -- 172.21.15.145:0/365264550 >> v1:172.21.15.145:6789/0 conn(0x7f03400725a0 legacy=0x7f034006ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:54.233 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.230+0000 7f033d400640 1 --2- 172.21.15.145:0/365264550 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0324005c10 0x7f0324006030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:54.233 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.230+0000 7f033d400640 1 --2- 172.21.15.145:0/365264550 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0324006a10 0x7f0324006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:54.233 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.230+0000 7f033d400640 1 --2- 172.21.15.145:0/365264550 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f03400b3420 0x7f03240073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:54.233 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.230+0000 7f033d400640 1 -- 172.21.15.145:0/365264550 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f034015d410 con 0x7f0324006a10 2024-05-09T04:35:54.233 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.230+0000 7f033d400640 1 -- 172.21.15.145:0/365264550 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f034006e9d0 con 0x7f0324005c10 2024-05-09T04:35:54.234 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.230+0000 7f033d400640 1 -- 172.21.15.145:0/365264550 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f032400dc50 con 0x7f03400b3420 2024-05-09T04:35:54.234 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.231+0000 7f0335400640 1 --2- 172.21.15.145:0/365264550 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0324006a10 0x7f0324006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:54.234 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.231+0000 7f033de00640 1 --2- 172.21.15.145:0/365264550 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0324005c10 0x7f0324006030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:54.234 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.231+0000 7f033e800640 1 --2- 172.21.15.145:0/365264550 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f03400b3420 0x7f03240073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:54.234 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.231+0000 7f0335400640 1 -- 172.21.15.145:0/365264550 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f03400b3420 msgr2=0x7f03240073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:54.234 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.231+0000 7f0335400640 1 --2- 172.21.15.145:0/365264550 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f03400b3420 0x7f03240073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:54.234 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.231+0000 7f0335400640 1 -- 172.21.15.145:0/365264550 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0324005c10 msgr2=0x7f0324006030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:54.235 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.231+0000 7f0335400640 1 --2- 172.21.15.145:0/365264550 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0324005c10 0x7f0324006030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:54.235 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.231+0000 7f0335400640 1 -- 172.21.15.145:0/365264550 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0324005730 con 0x7f0324006a10 2024-05-09T04:35:54.235 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.231+0000 7f033e800640 1 --2- 172.21.15.145:0/365264550 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f03400b3420 0x7f03240073d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:35:54.235 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.232+0000 7f0335400640 1 --2- 172.21.15.145:0/365264550 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0324006a10 0x7f0324006e90 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f0328009ac0 tx=0x7f0324007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7759ea8303009589 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:54.239 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.232+0000 7f033d400640 1 -- 172.21.15.145:0/365264550 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0328036070 con 0x7f0324006a10 2024-05-09T04:35:54.239 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.232+0000 7f033d400640 1 -- 172.21.15.145:0/365264550 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0328031400 con 0x7f0324006a10 2024-05-09T04:35:54.239 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.232+0000 7f033d400640 1 -- 172.21.15.145:0/365264550 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0328009690 con 0x7f0324006a10 2024-05-09T04:35:54.239 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.233+0000 7f033f200640 1 -- 172.21.15.145:0/365264550 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0324006a10 msgr2=0x7f0324006e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:54.239 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.233+0000 7f033f200640 1 --2- 172.21.15.145:0/365264550 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0324006a10 0x7f0324006e90 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f0328009ac0 tx=0x7f0324007a40 comp rx=0 tx=0).stop 2024-05-09T04:35:54.240 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.233+0000 7f033e800640 1 -- 172.21.15.145:0/365264550 reap_dead start 2024-05-09T04:35:54.240 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.233+0000 7f033f200640 1 -- 172.21.15.145:0/365264550 shutdown_connections 2024-05-09T04:35:54.240 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.233+0000 7f033f200640 1 -- 172.21.15.145:0/365264550 >> 172.21.15.145:0/365264550 conn(0x7f03400b2850 msgr2=0x7f03400b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:54.240 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.233+0000 7f033f200640 1 -- 172.21.15.145:0/365264550 shutdown_connections 2024-05-09T04:35:54.240 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.233+0000 7f033f200640 1 -- 172.21.15.145:0/365264550 wait complete. 2024-05-09T04:35:54.240 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.235+0000 7f033f200640 1 Processor -- start 2024-05-09T04:35:54.240 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.235+0000 7f033f200640 1 -- start start 2024-05-09T04:35:54.241 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.236+0000 7f033f200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0324006a10 0x7f0340152470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:54.241 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.236+0000 7f033f200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f034013fdb0 0x7f0340140210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:54.241 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.236+0000 7f033f200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0340141dc0 0x7f0340142240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:54.241 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.236+0000 7f033f200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f034015cb70 con 0x7f0324006a10 2024-05-09T04:35:54.241 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.236+0000 7f033f200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f034015c9f0 con 0x7f034013fdb0 2024-05-09T04:35:54.241 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.236+0000 7f033f200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f034006f570 con 0x7f0340141dc0 2024-05-09T04:35:54.241 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.236+0000 7f033de00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0324006a10 0x7f0340152470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:54.242 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.236+0000 7f033de00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0324006a10 0x7f0340152470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:53098/0 (socket says 172.21.15.145:53098) 2024-05-09T04:35:54.242 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.237+0000 7f033de00640 1 -- 172.21.15.145:0/357136491 learned_addr learned my addr 172.21.15.145:0/357136491 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:54.242 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.237+0000 7f0335400640 1 --2- 172.21.15.145:0/357136491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f034013fdb0 0x7f0340140210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:54.242 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.237+0000 7f033de00640 1 -- 172.21.15.145:0/357136491 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0340141dc0 msgr2=0x7f0340142240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:54.242 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.237+0000 7f033e800640 1 --2- 172.21.15.145:0/357136491 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0340141dc0 0x7f0340142240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:54.242 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.237+0000 7f033de00640 1 --2- 172.21.15.145:0/357136491 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0340141dc0 0x7f0340142240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:54.242 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.237+0000 7f033de00640 1 -- 172.21.15.145:0/357136491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f034013fdb0 msgr2=0x7f0340140210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:54.242 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.237+0000 7f033de00640 1 --2- 172.21.15.145:0/357136491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f034013fdb0 0x7f0340140210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:54.243 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.237+0000 7f033de00640 1 -- 172.21.15.145:0/357136491 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0340144fe0 con 0x7f0324006a10 2024-05-09T04:35:54.243 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.237+0000 7f0335400640 1 --2- 172.21.15.145:0/357136491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f034013fdb0 0x7f0340140210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:35:54.243 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.237+0000 7f033de00640 1 --2- 172.21.15.145:0/357136491 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0324006a10 0x7f0340152470 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f033000f3d0 tx=0x7f033000f890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:54.243 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.238+0000 7f0336a00640 1 -- 172.21.15.145:0/357136491 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0330016620 con 0x7f0324006a10 2024-05-09T04:35:54.243 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.238+0000 7f0336a00640 1 -- 172.21.15.145:0/357136491 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f033001b070 con 0x7f0324006a10 2024-05-09T04:35:54.243 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.238+0000 7f033f200640 1 -- 172.21.15.145:0/357136491 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0340143ca0 con 0x7f0324006a10 2024-05-09T04:35:54.243 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.238+0000 7f0336a00640 1 -- 172.21.15.145:0/357136491 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0330016a20 con 0x7f0324006a10 2024-05-09T04:35:54.244 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.238+0000 7f033f200640 1 -- 172.21.15.145:0/357136491 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0340144220 con 0x7f0324006a10 2024-05-09T04:35:54.244 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.239+0000 7f033f200640 1 -- 172.21.15.145:0/357136491 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0340070910 con 0x7f0324006a10 2024-05-09T04:35:54.245 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.240+0000 7f0336a00640 1 -- 172.21.15.145:0/357136491 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f0330005d10 con 0x7f0324006a10 2024-05-09T04:35:54.245 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.241+0000 7f0336a00640 1 --2- 172.21.15.145:0/357136491 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f031c0762d0 0x7f031c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:54.245 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.241+0000 7f0336a00640 1 -- 172.21.15.145:0/357136491 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7f0330098ea0 con 0x7f0324006a10 2024-05-09T04:35:54.245 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.241+0000 7f0336a00640 1 --2- 172.21.15.145:0/357136491 >> [v2:172.21.15.192:6800/1328373268,v1:172.21.15.192:6801/1328373268] conn(0x7f031c07c190 0x7f031c07e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:54.246 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.241+0000 7f0335400640 1 --2- 172.21.15.145:0/357136491 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f031c0762d0 0x7f031c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:54.246 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.241+0000 7f0336a00640 1 -- 172.21.15.145:0/357136491 --> [v2:172.21.15.192:6800/1328373268,v1:172.21.15.192:6801/1328373268] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f031c07ec60 con 0x7f031c07c190 2024-05-09T04:35:54.246 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.241+0000 7f0336a00640 1 -- 172.21.15.145:0/357136491 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f0330099250 con 0x7f0324006a10 2024-05-09T04:35:54.246 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.242+0000 7f0335400640 1 --2- 172.21.15.145:0/357136491 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f031c0762d0 0x7f031c078790 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0328033000 tx=0x7f0328008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:54.250 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.242+0000 7f033e800640 1 --2- 172.21.15.145:0/357136491 >> [v2:172.21.15.192:6800/1328373268,v1:172.21.15.192:6801/1328373268] conn(0x7f031c07c190 0x7f031c07e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:54.250 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.243+0000 7f033e800640 1 --2- 172.21.15.145:0/357136491 >> [v2:172.21.15.192:6800/1328373268,v1:172.21.15.192:6801/1328373268] conn(0x7f031c07c190 0x7f031c07e5b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:54.251 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.245+0000 7f0336a00640 1 -- 172.21.15.145:0/357136491 <== osd.5 v2:172.21.15.192:6800/1328373268 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f031c07ec60 con 0x7f031c07c190 2024-05-09T04:35:54.259 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.259+0000 7f033f200640 1 -- 172.21.15.145:0/357136491 --> [v2:172.21.15.192:6800/1328373268,v1:172.21.15.192:6801/1328373268] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f0340076930 con 0x7f031c07c190 2024-05-09T04:35:54.260 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.260+0000 7f0336a00640 1 -- 172.21.15.145:0/357136491 <== osd.5 v2:172.21.15.192:6800/1328373268 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f0340076930 con 0x7f031c07c190 2024-05-09T04:35:54.260 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-05-09T04:35:54.260 INFO:teuthology.orchestra.run.smithi145.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-09T04:35:54.261 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-05-09T04:35:54.262 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.261+0000 7f033f200640 1 -- 172.21.15.145:0/357136491 >> [v2:172.21.15.192:6800/1328373268,v1:172.21.15.192:6801/1328373268] conn(0x7f031c07c190 msgr2=0x7f031c07e5b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:54.262 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.261+0000 7f033f200640 1 --2- 172.21.15.145:0/357136491 >> [v2:172.21.15.192:6800/1328373268,v1:172.21.15.192:6801/1328373268] conn(0x7f031c07c190 0x7f031c07e5b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:54.262 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.261+0000 7f033f200640 1 -- 172.21.15.145:0/357136491 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f031c0762d0 msgr2=0x7f031c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:54.262 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.261+0000 7f033f200640 1 --2- 172.21.15.145:0/357136491 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f031c0762d0 0x7f031c078790 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0328033000 tx=0x7f0328008030 comp rx=0 tx=0).stop 2024-05-09T04:35:54.262 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.261+0000 7f033f200640 1 -- 172.21.15.145:0/357136491 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0324006a10 msgr2=0x7f0340152470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:54.262 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.261+0000 7f033f200640 1 --2- 172.21.15.145:0/357136491 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0324006a10 0x7f0340152470 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f033000f3d0 tx=0x7f033000f890 comp rx=0 tx=0).stop 2024-05-09T04:35:54.263 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.261+0000 7f033e800640 1 -- 172.21.15.145:0/357136491 reap_dead start 2024-05-09T04:35:54.263 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.261+0000 7f033f200640 1 -- 172.21.15.145:0/357136491 shutdown_connections 2024-05-09T04:35:54.263 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.261+0000 7f033f200640 1 -- 172.21.15.145:0/357136491 >> 172.21.15.145:0/357136491 conn(0x7f03400b2850 msgr2=0x7f03400d0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:54.263 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.261+0000 7f033f200640 1 -- 172.21.15.145:0/357136491 shutdown_connections 2024-05-09T04:35:54.263 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.261+0000 7f033f200640 1 -- 172.21.15.145:0/357136491 wait complete. 2024-05-09T04:35:54.279 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-05-09T04:35:54.478 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.478+0000 7f46a5600640 1 Processor -- start 2024-05-09T04:35:54.478 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.478+0000 7f46a5600640 1 -- start start 2024-05-09T04:35:54.480 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.480+0000 7f46a5600640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f46a0140070 0x7f46a00cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:54.480 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.480+0000 7f46a5600640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f46a00ce330 0x7f46a0142ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:54.481 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.480+0000 7f46a5600640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46a015d780 con 0x7f46a01430e0 2024-05-09T04:35:54.481 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.480+0000 7f46a5600640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f46a00ce790 con 0x7f46a00ce330 2024-05-09T04:35:54.481 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.480+0000 7f46a5600640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f46a015e330 con 0x7f46a0140070 2024-05-09T04:35:54.481 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.481+0000 7f4697e00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f46a00ce330 0x7f46a0142ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:54.481 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.481+0000 7f469f400640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f46a01430e0 0x7f46a0072270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:60588/0 (socket says 172.21.15.145:60588) 2024-05-09T04:35:54.481 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.481+0000 7f469f400640 1 -- 172.21.15.145:0/353400418 learned_addr learned my addr 172.21.15.145:0/353400418 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:35:54.481 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.481+0000 7f469ea00640 1 --2- 172.21.15.145:0/353400418 >> v2:172.21.15.192:3300/0 conn(0x7f46a0140070 0x7f46a00cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:54.482 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.481+0000 7f4697e00640 1 -- 172.21.15.145:0/353400418 >> v2:172.21.15.192:3300/0 conn(0x7f46a0140070 msgr2=0x7f46a00cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:54.482 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.481+0000 7f4697e00640 1 --2- 172.21.15.145:0/353400418 >> v2:172.21.15.192:3300/0 conn(0x7f46a0140070 0x7f46a00cddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:54.482 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.481+0000 7f4697e00640 1 -- 172.21.15.145:0/353400418 >> v1:172.21.15.145:6789/0 conn(0x7f46a01430e0 legacy=0x7f46a0072270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:54.482 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.481+0000 7f4697e00640 1 -- 172.21.15.145:0/353400418 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46a015da00 con 0x7f46a00ce330 2024-05-09T04:35:54.483 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.483+0000 7f4697e00640 1 --2- 172.21.15.145:0/353400418 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46a00ce330 0x7f46a0142ba0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f46a00cea80 tx=0x7f468802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dd8f008238c296a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:54.483 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.483+0000 7f469e000640 1 -- 172.21.15.145:0/353400418 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4688037070 con 0x7f46a00ce330 2024-05-09T04:35:54.483 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.484+0000 7f469e000640 1 -- 172.21.15.145:0/353400418 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4688032440 con 0x7f46a00ce330 2024-05-09T04:35:54.483 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.484+0000 7f469e000640 1 -- 172.21.15.145:0/353400418 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4688032810 con 0x7f46a00ce330 2024-05-09T04:35:54.484 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.484+0000 7f46a5600640 1 -- 172.21.15.145:0/353400418 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46a00ce330 msgr2=0x7f46a0142ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:54.484 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.484+0000 7f46a5600640 1 --2- 172.21.15.145:0/353400418 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46a00ce330 0x7f46a0142ba0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f46a00cea80 tx=0x7f468802a1a0 comp rx=0 tx=0).stop 2024-05-09T04:35:54.484 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.484+0000 7f46a5600640 1 -- 172.21.15.145:0/353400418 shutdown_connections 2024-05-09T04:35:54.484 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.484+0000 7f46a5600640 1 --2- 172.21.15.145:0/353400418 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46a00ce330 0x7f46a0142ba0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:54.484 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.484+0000 7f46a5600640 1 --2- 172.21.15.145:0/353400418 >> v2:172.21.15.192:3300/0 conn(0x7f46a0140070 0x7f46a00cddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:54.485 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.484+0000 7f46a5600640 1 -- 172.21.15.145:0/353400418 >> 172.21.15.145:0/353400418 conn(0x7f46a006c8d0 msgr2=0x7f46a006cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:54.485 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.484+0000 7f46a5600640 1 -- 172.21.15.145:0/353400418 shutdown_connections 2024-05-09T04:35:54.485 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.485+0000 7f46a5600640 1 -- 172.21.15.145:0/353400418 wait complete. 2024-05-09T04:35:54.486 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.486+0000 7f46a5600640 1 Processor -- start 2024-05-09T04:35:54.486 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.487+0000 7f46a5600640 1 -- start start 2024-05-09T04:35:54.488 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.487+0000 7f46a5600640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f46a00ce330 0x7f46a013e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:54.488 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.487+0000 7f46a5600640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46a0140070 0x7f46a01565e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:54.488 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.487+0000 7f46a5600640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46a01430e0 0x7f46a0156b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:54.488 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.488+0000 7f46a5600640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f46a0140850 con 0x7f46a01430e0 2024-05-09T04:35:54.488 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.488+0000 7f46a5600640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f46a015dfc0 con 0x7f46a0140070 2024-05-09T04:35:54.488 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.488+0000 7f46a5600640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f46a0140450 con 0x7f46a00ce330 2024-05-09T04:35:54.488 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.488+0000 7f469f400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46a01430e0 0x7f46a0156b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:54.489 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.488+0000 7f469f400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46a01430e0 0x7f46a0156b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:53112/0 (socket says 172.21.15.145:53112) 2024-05-09T04:35:54.489 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.488+0000 7f469f400640 1 -- 172.21.15.145:0/2567399977 learned_addr learned my addr 172.21.15.145:0/2567399977 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:54.489 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.488+0000 7f4697e00640 1 --2- 172.21.15.145:0/2567399977 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46a0140070 0x7f46a01565e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:54.489 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.488+0000 7f469ea00640 1 --2- 172.21.15.145:0/2567399977 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f46a00ce330 0x7f46a013e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:54.489 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.488+0000 7f469f400640 1 -- 172.21.15.145:0/2567399977 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f46a00ce330 msgr2=0x7f46a013e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:54.489 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.488+0000 7f469f400640 1 --2- 172.21.15.145:0/2567399977 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f46a00ce330 0x7f46a013e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:54.489 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.488+0000 7f469f400640 1 -- 172.21.15.145:0/2567399977 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46a0140070 msgr2=0x7f46a01565e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:54.489 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.488+0000 7f469f400640 1 --2- 172.21.15.145:0/2567399977 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46a0140070 0x7f46a01565e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:54.489 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.488+0000 7f469f400640 1 -- 172.21.15.145:0/2567399977 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46a0155540 con 0x7f46a01430e0 2024-05-09T04:35:54.491 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.489+0000 7f469f400640 1 --2- 172.21.15.145:0/2567399977 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46a01430e0 0x7f46a0156b20 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f469800cb50 tx=0x7f469800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:54.491 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.489+0000 7f4696a00640 1 -- 172.21.15.145:0/2567399977 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4698017070 con 0x7f46a01430e0 2024-05-09T04:35:54.491 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.489+0000 7f4696a00640 1 -- 172.21.15.145:0/2567399977 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f469800ad40 con 0x7f46a01430e0 2024-05-09T04:35:54.491 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.489+0000 7f4696a00640 1 -- 172.21.15.145:0/2567399977 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4698012600 con 0x7f46a01430e0 2024-05-09T04:35:54.491 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.489+0000 7f46a5600640 1 -- 172.21.15.145:0/2567399977 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46a0155820 con 0x7f46a01430e0 2024-05-09T04:35:54.491 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.489+0000 7f46a5600640 1 -- 172.21.15.145:0/2567399977 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46a0155de0 con 0x7f46a01430e0 2024-05-09T04:35:54.491 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.490+0000 7f46a5600640 1 -- 172.21.15.145:0/2567399977 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46a006a310 con 0x7f46a01430e0 2024-05-09T04:35:54.493 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.494+0000 7f4696a00640 1 -- 172.21.15.145:0/2567399977 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f469801a070 con 0x7f46a01430e0 2024-05-09T04:35:54.495 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.495+0000 7f4696a00640 1 --2- 172.21.15.145:0/2567399977 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f467c0762d0 0x7f467c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:54.495 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.495+0000 7f4696a00640 1 -- 172.21.15.145:0/2567399977 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7f4698097880 con 0x7f46a01430e0 2024-05-09T04:35:54.495 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.495+0000 7f469ea00640 1 --2- 172.21.15.145:0/2567399977 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f467c0762d0 0x7f467c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:54.495 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.496+0000 7f469ea00640 1 --2- 172.21.15.145:0/2567399977 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f467c0762d0 0x7f467c078790 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4690006fd0 tx=0x7f4690008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:54.496 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.497+0000 7f4696a00640 1 -- 172.21.15.145:0/2567399977 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f46a015d780 con 0x7f46a01430e0 2024-05-09T04:35:54.705 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.705+0000 7f46a5600640 1 -- 172.21.15.145:0/2567399977 --> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f46a006cfc0 con 0x7f467c0762d0 2024-05-09T04:35:54.705 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.706+0000 7f4696a00640 1 -- 172.21.15.145:0/2567399977 <== mgr.4104 v2:172.21.15.145:6832/2725855396 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f46a006cfc0 con 0x7f467c0762d0 2024-05-09T04:35:54.706 INFO:teuthology.orchestra.run.smithi145.stderr:instructed osd(s) 5 to deep-scrub 2024-05-09T04:35:54.709 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.709+0000 7f46a5600640 1 -- 172.21.15.145:0/2567399977 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f467c0762d0 msgr2=0x7f467c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:54.709 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.709+0000 7f46a5600640 1 --2- 172.21.15.145:0/2567399977 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f467c0762d0 0x7f467c078790 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4690006fd0 tx=0x7f4690008040 comp rx=0 tx=0).stop 2024-05-09T04:35:54.709 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.709+0000 7f46a5600640 1 -- 172.21.15.145:0/2567399977 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46a01430e0 msgr2=0x7f46a0156b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:54.709 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.709+0000 7f46a5600640 1 --2- 172.21.15.145:0/2567399977 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46a01430e0 0x7f46a0156b20 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f469800cb50 tx=0x7f469800a430 comp rx=0 tx=0).stop 2024-05-09T04:35:54.709 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.709+0000 7f46a5600640 1 -- 172.21.15.145:0/2567399977 shutdown_connections 2024-05-09T04:35:54.709 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.709+0000 7f46a5600640 1 --2- 172.21.15.145:0/2567399977 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f467c0762d0 0x7f467c078790 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:54.709 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.709+0000 7f46a5600640 1 --2- 172.21.15.145:0/2567399977 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46a01430e0 0x7f46a0156b20 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:54.710 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.709+0000 7f46a5600640 1 --2- 172.21.15.145:0/2567399977 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f46a0140070 0x7f46a01565e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:54.710 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.709+0000 7f46a5600640 1 --2- 172.21.15.145:0/2567399977 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f46a00ce330 0x7f46a013e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:54.710 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.709+0000 7f46a5600640 1 -- 172.21.15.145:0/2567399977 >> 172.21.15.145:0/2567399977 conn(0x7f46a006c8d0 msgr2=0x7f46a00cd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:54.710 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.709+0000 7f46a5600640 1 -- 172.21.15.145:0/2567399977 shutdown_connections 2024-05-09T04:35:54.710 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.709+0000 7f46a5600640 1 -- 172.21.15.145:0/2567399977 wait complete. 2024-05-09T04:35:54.728 INFO:tasks.ceph:Scrubbing osd.6 2024-05-09T04:35:54.728 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-05-09T04:35:54.934 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.934+0000 7f8194a00640 1 Processor -- start 2024-05-09T04:35:54.935 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.934+0000 7f8194a00640 1 -- start start 2024-05-09T04:35:54.936 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.936+0000 7f8194a00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f8190145860 0x7f819015c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:54.936 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.936+0000 7f8194a00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f819015ce50 con 0x7f81900b3420 2024-05-09T04:35:54.936 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.936+0000 7f8194a00640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f819015d130 con 0x7f819006ee60 2024-05-09T04:35:54.936 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.936+0000 7f8194a00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f819015d410 con 0x7f8190145860 2024-05-09T04:35:54.937 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.936+0000 7f818e800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f8190145860 0x7f819015c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:54.937 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.936+0000 7f8185e00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f81900b3420 0x7f8190072060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:60602/0 (socket says 172.21.15.145:60602) 2024-05-09T04:35:54.937 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.936+0000 7f818e800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f8190145860 0x7f819015c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:53128/0 (socket says 172.21.15.145:53128) 2024-05-09T04:35:54.937 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.936+0000 7f8185e00640 1 -- 172.21.15.145:0/1751472091 learned_addr learned my addr 172.21.15.145:0/1751472091 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:35:54.938 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.937+0000 7f818e800640 1 -- 172.21.15.145:0/1751472091 >> v1:172.21.15.192:6789/0 conn(0x7f819006ee60 legacy=0x7f81900cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:54.938 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.937+0000 7f818e800640 1 -- 172.21.15.145:0/1751472091 >> v1:172.21.15.145:6789/0 conn(0x7f81900b3420 legacy=0x7f8190072060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:54.938 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.937+0000 7f818e800640 1 -- 172.21.15.145:0/1751472091 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f819006e660 con 0x7f8190145860 2024-05-09T04:35:54.938 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.937+0000 7f818de00640 1 -- 172.21.15.145:0/1751472091 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1442329893 0 0) 0x7f819015ce50 con 0x7f81900b3420 2024-05-09T04:35:54.939 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.938+0000 7f818e800640 1 --2- 172.21.15.145:0/1751472091 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8190145860 0x7f819015c4b0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f819015d590 tx=0x7f818002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3a4ef273410cb52 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:54.939 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.938+0000 7f818de00640 1 -- 172.21.15.145:0/1751472091 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8180037070 con 0x7f8190145860 2024-05-09T04:35:54.939 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.938+0000 7f818de00640 1 -- 172.21.15.145:0/1751472091 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f818002ae50 con 0x7f8190145860 2024-05-09T04:35:54.939 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.938+0000 7f818de00640 1 -- 172.21.15.145:0/1751472091 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8180032660 con 0x7f8190145860 2024-05-09T04:35:54.940 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.939+0000 7f8194a00640 1 -- 172.21.15.145:0/1751472091 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8190145860 msgr2=0x7f819015c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:54.940 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.939+0000 7f8194a00640 1 --2- 172.21.15.145:0/1751472091 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8190145860 0x7f819015c4b0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f819015d590 tx=0x7f818002a100 comp rx=0 tx=0).stop 2024-05-09T04:35:54.940 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.939+0000 7f8194a00640 1 -- 172.21.15.145:0/1751472091 shutdown_connections 2024-05-09T04:35:54.940 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.939+0000 7f8194a00640 1 --2- 172.21.15.145:0/1751472091 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8190145860 0x7f819015c4b0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:54.940 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.939+0000 7f8194a00640 1 -- 172.21.15.145:0/1751472091 >> 172.21.15.145:0/1751472091 conn(0x7f81900b2850 msgr2=0x7f81900b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:54.940 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.939+0000 7f8194a00640 1 -- 172.21.15.145:0/1751472091 shutdown_connections 2024-05-09T04:35:54.940 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.939+0000 7f8194a00640 1 -- 172.21.15.145:0/1751472091 wait complete. 2024-05-09T04:35:54.941 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.941+0000 7f8194a00640 1 Processor -- start 2024-05-09T04:35:54.941 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.941+0000 7f8194a00640 1 -- start start 2024-05-09T04:35:54.943 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.942+0000 7f8194a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f819006ee60 0x7f81900c6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:54.943 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.942+0000 7f8194a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81900b3420 0x7f819015e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:54.943 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.942+0000 7f8194a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8190145860 0x7f819015eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:54.944 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.942+0000 7f8194a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f819015c9f0 con 0x7f8190145860 2024-05-09T04:35:54.944 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.942+0000 7f8194a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f819006e9d0 con 0x7f81900b3420 2024-05-09T04:35:54.944 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.942+0000 7f8194a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f819015cb70 con 0x7f819006ee60 2024-05-09T04:35:54.944 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.942+0000 7f818f200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8190145860 0x7f819015eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:54.944 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.942+0000 7f818f200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8190145860 0x7f819015eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:53132/0 (socket says 172.21.15.145:53132) 2024-05-09T04:35:54.944 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.942+0000 7f818f200640 1 -- 172.21.15.145:0/695795475 learned_addr learned my addr 172.21.15.145:0/695795475 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:54.944 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.943+0000 7f8185e00640 1 --2- 172.21.15.145:0/695795475 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81900b3420 0x7f819015e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:54.944 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.943+0000 7f818e800640 1 --2- 172.21.15.145:0/695795475 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f819006ee60 0x7f81900c6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:54.944 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.943+0000 7f818f200640 1 -- 172.21.15.145:0/695795475 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f819006ee60 msgr2=0x7f81900c6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:54.945 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.943+0000 7f818f200640 1 --2- 172.21.15.145:0/695795475 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f819006ee60 0x7f81900c6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:54.945 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.943+0000 7f818f200640 1 -- 172.21.15.145:0/695795475 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81900b3420 msgr2=0x7f819015e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:54.945 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.943+0000 7f818f200640 1 --2- 172.21.15.145:0/695795475 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81900b3420 0x7f819015e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:54.945 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.943+0000 7f818f200640 1 -- 172.21.15.145:0/695795475 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f819014fb50 con 0x7f8190145860 2024-05-09T04:35:54.945 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.943+0000 7f818f200640 1 --2- 172.21.15.145:0/695795475 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8190145860 0x7f819015eb20 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f818800c7b0 tx=0x7f818800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:54.945 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.943+0000 7f8187400640 1 -- 172.21.15.145:0/695795475 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8188004b60 con 0x7f8190145860 2024-05-09T04:35:54.945 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.943+0000 7f8187400640 1 -- 172.21.15.145:0/695795475 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8188004d00 con 0x7f8190145860 2024-05-09T04:35:54.945 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.943+0000 7f8187400640 1 -- 172.21.15.145:0/695795475 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818800aa30 con 0x7f8190145860 2024-05-09T04:35:54.945 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.943+0000 7f8194a00640 1 -- 172.21.15.145:0/695795475 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81900af540 con 0x7f8190145860 2024-05-09T04:35:54.946 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.944+0000 7f8194a00640 1 -- 172.21.15.145:0/695795475 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f819014fdf0 con 0x7f8190145860 2024-05-09T04:35:54.946 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.944+0000 7f8194a00640 1 -- 172.21.15.145:0/695795475 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8190076930 con 0x7f8190145860 2024-05-09T04:35:54.946 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.946+0000 7f8187400640 1 -- 172.21.15.145:0/695795475 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f818800abd0 con 0x7f8190145860 2024-05-09T04:35:54.947 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.947+0000 7f8187400640 1 --2- 172.21.15.145:0/695795475 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f816c0762d0 0x7f816c078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:54.947 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.947+0000 7f8187400640 1 -- 172.21.15.145:0/695795475 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7f8188097cc0 con 0x7f8190145860 2024-05-09T04:35:54.947 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.947+0000 7f818e800640 1 --2- 172.21.15.145:0/695795475 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f816c0762d0 0x7f816c078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:54.947 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.948+0000 7f8187400640 1 --2- 172.21.15.145:0/695795475 >> [v2:172.21.15.192:6817/1868011057,v1:172.21.15.192:6820/1868011057] conn(0x7f816c07c190 0x7f816c07e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:54.948 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.948+0000 7f8187400640 1 -- 172.21.15.145:0/695795475 --> [v2:172.21.15.192:6817/1868011057,v1:172.21.15.192:6820/1868011057] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f816c07ec60 con 0x7f816c07c190 2024-05-09T04:35:54.948 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.948+0000 7f8187400640 1 -- 172.21.15.145:0/695795475 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f8188098070 con 0x7f8190145860 2024-05-09T04:35:54.948 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.948+0000 7f818e800640 1 --2- 172.21.15.145:0/695795475 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f816c0762d0 0x7f816c078790 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8180034000 tx=0x7f818002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:54.948 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.948+0000 7f8185e00640 1 --2- 172.21.15.145:0/695795475 >> [v2:172.21.15.192:6817/1868011057,v1:172.21.15.192:6820/1868011057] conn(0x7f816c07c190 0x7f816c07e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:54.949 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.949+0000 7f8185e00640 1 --2- 172.21.15.145:0/695795475 >> [v2:172.21.15.192:6817/1868011057,v1:172.21.15.192:6820/1868011057] conn(0x7f816c07c190 0x7f816c07e5b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:54.951 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.951+0000 7f8187400640 1 -- 172.21.15.145:0/695795475 <== osd.6 v2:172.21.15.192:6817/1868011057 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f816c07ec60 con 0x7f816c07c190 2024-05-09T04:35:54.965 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.966+0000 7f8194a00640 1 -- 172.21.15.145:0/695795475 --> [v2:172.21.15.192:6817/1868011057,v1:172.21.15.192:6820/1868011057] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f8190072400 con 0x7f816c07c190 2024-05-09T04:35:54.966 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.967+0000 7f8187400640 1 -- 172.21.15.145:0/695795475 <== osd.6 v2:172.21.15.192:6817/1868011057 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f8190072400 con 0x7f816c07c190 2024-05-09T04:35:54.967 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-05-09T04:35:54.967 INFO:teuthology.orchestra.run.smithi145.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-09T04:35:54.967 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-05-09T04:35:54.968 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.967+0000 7f8194a00640 1 -- 172.21.15.145:0/695795475 >> [v2:172.21.15.192:6817/1868011057,v1:172.21.15.192:6820/1868011057] conn(0x7f816c07c190 msgr2=0x7f816c07e5b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:54.968 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.967+0000 7f8194a00640 1 --2- 172.21.15.145:0/695795475 >> [v2:172.21.15.192:6817/1868011057,v1:172.21.15.192:6820/1868011057] conn(0x7f816c07c190 0x7f816c07e5b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:54.968 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.968+0000 7f8194a00640 1 -- 172.21.15.145:0/695795475 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f816c0762d0 msgr2=0x7f816c078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:54.968 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.968+0000 7f8194a00640 1 --2- 172.21.15.145:0/695795475 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f816c0762d0 0x7f816c078790 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8180034000 tx=0x7f818002e000 comp rx=0 tx=0).stop 2024-05-09T04:35:54.968 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.968+0000 7f8194a00640 1 -- 172.21.15.145:0/695795475 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8190145860 msgr2=0x7f819015eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:54.969 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.968+0000 7f8194a00640 1 --2- 172.21.15.145:0/695795475 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8190145860 0x7f819015eb20 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f818800c7b0 tx=0x7f818800cc70 comp rx=0 tx=0).stop 2024-05-09T04:35:54.969 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.968+0000 7f818f200640 1 -- 172.21.15.145:0/695795475 reap_dead start 2024-05-09T04:35:54.969 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.968+0000 7f8194a00640 1 -- 172.21.15.145:0/695795475 shutdown_connections 2024-05-09T04:35:54.969 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.968+0000 7f8194a00640 1 -- 172.21.15.145:0/695795475 >> 172.21.15.145:0/695795475 conn(0x7f81900b2850 msgr2=0x7f819013fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:54.969 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.968+0000 7f8194a00640 1 -- 172.21.15.145:0/695795475 shutdown_connections 2024-05-09T04:35:54.969 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:54.968+0000 7f8194a00640 1 -- 172.21.15.145:0/695795475 wait complete. 2024-05-09T04:35:54.985 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-05-09T04:35:55.178 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.178+0000 7fe853200640 1 Processor -- start 2024-05-09T04:35:55.178 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.179+0000 7fe853200640 1 -- start start 2024-05-09T04:35:55.180 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.180+0000 7fe853200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fe85415f4e0 0x7fe85416a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:55.180 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.180+0000 7fe853200640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8540cfda0 con 0x7fe85415f100 2024-05-09T04:35:55.180 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.180+0000 7fe853200640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8540d0080 con 0x7fe8540d3cd0 2024-05-09T04:35:55.180 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.180+0000 7fe853200640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fe8540b8e70 con 0x7fe85415f4e0 2024-05-09T04:35:55.181 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.180+0000 7fe852800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fe85415f4e0 0x7fe85416a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:55.181 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.180+0000 7fe851e00640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7fe85415f100 0x7fe8540d35c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:60610/0 (socket says 172.21.15.145:60610) 2024-05-09T04:35:55.181 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.180+0000 7fe851e00640 1 -- 172.21.15.145:0/536171980 learned_addr learned my addr 172.21.15.145:0/536171980 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:35:55.181 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.181+0000 7fe852800640 1 -- 172.21.15.145:0/536171980 >> v1:172.21.15.192:6789/0 conn(0x7fe8540d3cd0 legacy=0x7fe8540d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:55.181 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.181+0000 7fe852800640 1 -- 172.21.15.145:0/536171980 >> v1:172.21.15.145:6789/0 conn(0x7fe85415f100 legacy=0x7fe8540d35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:55.181 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.181+0000 7fe852800640 1 -- 172.21.15.145:0/536171980 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8540d0360 con 0x7fe85415f4e0 2024-05-09T04:35:55.182 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.181+0000 7fe850a00640 1 -- 172.21.15.145:0/536171980 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1511683755 0 0) 0x7fe8540cfda0 con 0x7fe85415f100 2024-05-09T04:35:55.183 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.182+0000 7fe852800640 1 --2- 172.21.15.145:0/536171980 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe85415f4e0 0x7fe85416a120 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fe84c0097d0 tx=0x7fe84c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=984328df615696d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:55.183 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.182+0000 7fe850a00640 1 -- 172.21.15.145:0/536171980 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe84c037070 con 0x7fe85415f4e0 2024-05-09T04:35:55.183 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.182+0000 7fe850a00640 1 -- 172.21.15.145:0/536171980 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe84c032440 con 0x7fe85415f4e0 2024-05-09T04:35:55.183 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.182+0000 7fe850a00640 1 -- 172.21.15.145:0/536171980 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe84c032810 con 0x7fe85415f4e0 2024-05-09T04:35:55.183 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.183+0000 7fe853200640 1 -- 172.21.15.145:0/536171980 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe85415f4e0 msgr2=0x7fe85416a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:55.183 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.183+0000 7fe853200640 1 --2- 172.21.15.145:0/536171980 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe85415f4e0 0x7fe85416a120 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fe84c0097d0 tx=0x7fe84c02a1c0 comp rx=0 tx=0).stop 2024-05-09T04:35:55.184 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.183+0000 7fe853200640 1 -- 172.21.15.145:0/536171980 shutdown_connections 2024-05-09T04:35:55.184 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.183+0000 7fe853200640 1 --2- 172.21.15.145:0/536171980 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe85415f4e0 0x7fe85416a120 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:55.184 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.183+0000 7fe853200640 1 -- 172.21.15.145:0/536171980 >> 172.21.15.145:0/536171980 conn(0x7fe85409e2e0 msgr2=0x7fe8540d5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:55.184 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.183+0000 7fe853200640 1 -- 172.21.15.145:0/536171980 shutdown_connections 2024-05-09T04:35:55.184 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.183+0000 7fe853200640 1 -- 172.21.15.145:0/536171980 wait complete. 2024-05-09T04:35:55.184 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.185+0000 7fe853200640 1 Processor -- start 2024-05-09T04:35:55.185 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.185+0000 7fe853200640 1 -- start start 2024-05-09T04:35:55.186 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.186+0000 7fe853200640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8540d3cd0 0x7fe85415be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:55.186 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.186+0000 7fe853200640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe85415f100 0x7fe85409ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:55.186 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.186+0000 7fe853200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe85415f4e0 0x7fe85409f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:55.186 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.186+0000 7fe853200640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fe8540d0e40 con 0x7fe85415f4e0 2024-05-09T04:35:55.186 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.186+0000 7fe853200640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fe8540d0920 con 0x7fe8540d3cd0 2024-05-09T04:35:55.186 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.186+0000 7fe853200640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fe85415d040 con 0x7fe85415f100 2024-05-09T04:35:55.186 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.186+0000 7fe852800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe85415f4e0 0x7fe85409f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:55.187 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.186+0000 7fe852800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe85415f4e0 0x7fe85409f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:53158/0 (socket says 172.21.15.145:53158) 2024-05-09T04:35:55.187 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.186+0000 7fe852800640 1 -- 172.21.15.145:0/2552581243 learned_addr learned my addr 172.21.15.145:0/2552581243 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:55.188 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.186+0000 7fe851400640 1 --2- 172.21.15.145:0/2552581243 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe85415f100 0x7fe85409ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:55.188 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.187+0000 7fe852800640 1 -- 172.21.15.145:0/2552581243 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe85415f100 msgr2=0x7fe85409ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:55.189 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.187+0000 7fe851e00640 1 --2- 172.21.15.145:0/2552581243 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8540d3cd0 0x7fe85415be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:55.189 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.187+0000 7fe852800640 1 --2- 172.21.15.145:0/2552581243 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe85415f100 0x7fe85409ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:55.189 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.187+0000 7fe852800640 1 -- 172.21.15.145:0/2552581243 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8540d3cd0 msgr2=0x7fe85415be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:55.189 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.187+0000 7fe852800640 1 --2- 172.21.15.145:0/2552581243 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8540d3cd0 0x7fe85415be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:55.189 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.187+0000 7fe852800640 1 -- 172.21.15.145:0/2552581243 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8540cd410 con 0x7fe85415f4e0 2024-05-09T04:35:55.189 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.187+0000 7fe851400640 1 --2- 172.21.15.145:0/2552581243 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe85415f100 0x7fe85409ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:35:55.189 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.187+0000 7fe851e00640 1 --2- 172.21.15.145:0/2552581243 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8540d3cd0 0x7fe85415be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:35:55.189 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.187+0000 7fe852800640 1 --2- 172.21.15.145:0/2552581243 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe85415f4e0 0x7fe85409f290 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fe84c009720 tx=0x7fe84c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:55.190 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.187+0000 7fe84a000640 1 -- 172.21.15.145:0/2552581243 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe84c037040 con 0x7fe85415f4e0 2024-05-09T04:35:55.190 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.187+0000 7fe84a000640 1 -- 172.21.15.145:0/2552581243 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe84c004230 con 0x7fe85415f4e0 2024-05-09T04:35:55.190 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.187+0000 7fe853200640 1 -- 172.21.15.145:0/2552581243 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe85415d290 con 0x7fe85415f4e0 2024-05-09T04:35:55.190 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.187+0000 7fe84a000640 1 -- 172.21.15.145:0/2552581243 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe84c032e20 con 0x7fe85415f4e0 2024-05-09T04:35:55.190 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.188+0000 7fe853200640 1 -- 172.21.15.145:0/2552581243 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe85415d850 con 0x7fe85415f4e0 2024-05-09T04:35:55.190 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.189+0000 7fe853200640 1 -- 172.21.15.145:0/2552581243 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe854076780 con 0x7fe85415f4e0 2024-05-09T04:35:55.191 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.190+0000 7fe84a000640 1 -- 172.21.15.145:0/2552581243 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fe84c030770 con 0x7fe85415f4e0 2024-05-09T04:35:55.191 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.191+0000 7fe84a000640 1 --2- 172.21.15.145:0/2552581243 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fe830076300 0x7fe8300787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:55.191 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.191+0000 7fe84a000640 1 -- 172.21.15.145:0/2552581243 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7fe84c0b85e0 con 0x7fe85415f4e0 2024-05-09T04:35:55.191 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.191+0000 7fe851e00640 1 --2- 172.21.15.145:0/2552581243 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fe830076300 0x7fe8300787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:55.192 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.192+0000 7fe851e00640 1 --2- 172.21.15.145:0/2552581243 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fe830076300 0x7fe8300787c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe844000f60 tx=0x7fe844009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:55.194 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.195+0000 7fe84a000640 1 -- 172.21.15.145:0/2552581243 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe84c0821a0 con 0x7fe85415f4e0 2024-05-09T04:35:55.400 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.400+0000 7fe853200640 1 -- 172.21.15.145:0/2552581243 --> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7fe85406a520 con 0x7fe830076300 2024-05-09T04:35:55.401 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.401+0000 7fe84a000640 1 -- 172.21.15.145:0/2552581243 <== mgr.4104 v2:172.21.15.145:6832/2725855396 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fe85406a520 con 0x7fe830076300 2024-05-09T04:35:55.401 INFO:teuthology.orchestra.run.smithi145.stderr:instructed osd(s) 6 to deep-scrub 2024-05-09T04:35:55.404 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.404+0000 7fe853200640 1 -- 172.21.15.145:0/2552581243 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fe830076300 msgr2=0x7fe8300787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:55.404 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.404+0000 7fe853200640 1 --2- 172.21.15.145:0/2552581243 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fe830076300 0x7fe8300787c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe844000f60 tx=0x7fe844009040 comp rx=0 tx=0).stop 2024-05-09T04:35:55.404 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.404+0000 7fe853200640 1 -- 172.21.15.145:0/2552581243 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe85415f4e0 msgr2=0x7fe85409f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:55.404 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.404+0000 7fe853200640 1 --2- 172.21.15.145:0/2552581243 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe85415f4e0 0x7fe85409f290 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fe84c009720 tx=0x7fe84c0046c0 comp rx=0 tx=0).stop 2024-05-09T04:35:55.404 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.404+0000 7fe853200640 1 -- 172.21.15.145:0/2552581243 shutdown_connections 2024-05-09T04:35:55.404 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.404+0000 7fe853200640 1 --2- 172.21.15.145:0/2552581243 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fe830076300 0x7fe8300787c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:55.405 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.404+0000 7fe853200640 1 --2- 172.21.15.145:0/2552581243 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fe85415f4e0 0x7fe85409f290 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:55.405 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.404+0000 7fe853200640 1 --2- 172.21.15.145:0/2552581243 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe85415f100 0x7fe85409ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:55.405 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.404+0000 7fe853200640 1 --2- 172.21.15.145:0/2552581243 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8540d3cd0 0x7fe85415be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:55.405 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.404+0000 7fe853200640 1 -- 172.21.15.145:0/2552581243 >> 172.21.15.145:0/2552581243 conn(0x7fe85409e2e0 msgr2=0x7fe8540ba970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:55.405 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.404+0000 7fe853200640 1 -- 172.21.15.145:0/2552581243 shutdown_connections 2024-05-09T04:35:55.405 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.404+0000 7fe853200640 1 -- 172.21.15.145:0/2552581243 wait complete. 2024-05-09T04:35:55.422 INFO:tasks.ceph:Scrubbing osd.7 2024-05-09T04:35:55.423 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-05-09T04:35:55.629 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.629+0000 7f38f0800640 1 Processor -- start 2024-05-09T04:35:55.629 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.629+0000 7f38f0800640 1 -- start start 2024-05-09T04:35:55.631 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.631+0000 7f38f0800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38e815f210 0x7f38e80b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:55.631 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.631+0000 7f38f0800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f38e80b61e0 0x7f38e80cdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:55.632 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.631+0000 7f38f0800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f38e80ce2b0 0x7f38e80ced50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:55.632 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.631+0000 7f38f0800640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f38e80aeeb0 con 0x7f38e80ce2b0 2024-05-09T04:35:55.632 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.631+0000 7f38f0800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f38e80af190 con 0x7f38e80b61e0 2024-05-09T04:35:55.632 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.631+0000 7f38f0800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f38e80af470 con 0x7f38e815f210 2024-05-09T04:35:55.632 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.632+0000 7f38ee800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f38e80ce2b0 0x7f38e80ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:55.632 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.632+0000 7f38ee800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f38e80ce2b0 0x7f38e80ced50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:53168/0 (socket says 172.21.15.145:53168) 2024-05-09T04:35:55.632 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.632+0000 7f38ee800640 1 -- 172.21.15.145:0/1319407629 learned_addr learned my addr 172.21.15.145:0/1319407629 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:55.632 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.632+0000 7f38ed400640 1 --2- 172.21.15.145:0/1319407629 >> v2:172.21.15.192:3300/0 conn(0x7f38e80b61e0 0x7f38e80cdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:55.632 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.632+0000 7f38ede00640 1 --2- 172.21.15.145:0/1319407629 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38e815f210 0x7f38e80b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:55.633 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.632+0000 7f38ee800640 1 -- 172.21.15.145:0/1319407629 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38e815f210 msgr2=0x7f38e80b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:55.633 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.632+0000 7f38ee800640 1 --2- 172.21.15.145:0/1319407629 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38e815f210 0x7f38e80b5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:55.634 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.632+0000 7f38ee800640 1 -- 172.21.15.145:0/1319407629 >> v2:172.21.15.192:3300/0 conn(0x7f38e80b61e0 msgr2=0x7f38e80cdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:55.634 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.632+0000 7f38ee800640 1 --2- 172.21.15.145:0/1319407629 >> v2:172.21.15.192:3300/0 conn(0x7f38e80b61e0 0x7f38e80cdd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:55.634 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.632+0000 7f38ee800640 1 -- 172.21.15.145:0/1319407629 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38e80af750 con 0x7f38e80ce2b0 2024-05-09T04:35:55.634 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.633+0000 7f38ee800640 1 --2- 172.21.15.145:0/1319407629 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38e80ce2b0 0x7f38e80ced50 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f38e80ceaa0 tx=0x7f38e402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9b4b8e988e0b1db server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:55.634 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.634+0000 7f38eca00640 1 -- 172.21.15.145:0/1319407629 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e4036070 con 0x7f38e80ce2b0 2024-05-09T04:35:55.634 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.634+0000 7f38eca00640 1 -- 172.21.15.145:0/1319407629 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38e4031600 con 0x7f38e80ce2b0 2024-05-09T04:35:55.634 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.634+0000 7f38eca00640 1 -- 172.21.15.145:0/1319407629 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e40319d0 con 0x7f38e80ce2b0 2024-05-09T04:35:55.635 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.634+0000 7f38f0800640 1 -- 172.21.15.145:0/1319407629 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38e80ce2b0 msgr2=0x7f38e80ced50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:55.635 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.634+0000 7f38f0800640 1 --2- 172.21.15.145:0/1319407629 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38e80ce2b0 0x7f38e80ced50 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f38e80ceaa0 tx=0x7f38e402a6a0 comp rx=0 tx=0).stop 2024-05-09T04:35:55.635 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.634+0000 7f38f0800640 1 -- 172.21.15.145:0/1319407629 shutdown_connections 2024-05-09T04:35:55.635 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.634+0000 7f38f0800640 1 --2- 172.21.15.145:0/1319407629 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38e80ce2b0 0x7f38e80ced50 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:55.635 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.634+0000 7f38f0800640 1 --2- 172.21.15.145:0/1319407629 >> v2:172.21.15.192:3300/0 conn(0x7f38e80b61e0 0x7f38e80cdd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:55.636 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.634+0000 7f38f0800640 1 --2- 172.21.15.145:0/1319407629 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38e815f210 0x7f38e80b5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:55.636 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.634+0000 7f38f0800640 1 -- 172.21.15.145:0/1319407629 >> 172.21.15.145:0/1319407629 conn(0x7f38e806c8d0 msgr2=0x7f38e806cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:55.636 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.634+0000 7f38f0800640 1 -- 172.21.15.145:0/1319407629 shutdown_connections 2024-05-09T04:35:55.636 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.634+0000 7f38f0800640 1 -- 172.21.15.145:0/1319407629 wait complete. 2024-05-09T04:35:55.638 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.636+0000 7f38f0800640 1 Processor -- start 2024-05-09T04:35:55.638 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.636+0000 7f38f0800640 1 -- start start 2024-05-09T04:35:55.638 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.637+0000 7f38f0800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38e80b61e0 0x7f38e814b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:55.638 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.637+0000 7f38f0800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38e80ce2b0 0x7f38e81571d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:55.638 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.637+0000 7f38f0800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38e815f210 0x7f38e80b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:55.639 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.637+0000 7f38f0800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f38e815f5f0 con 0x7f38e80b61e0 2024-05-09T04:35:55.639 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.637+0000 7f38f0800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f38e80b04b0 con 0x7f38e80ce2b0 2024-05-09T04:35:55.639 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.637+0000 7f38f0800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f38e815f770 con 0x7f38e815f210 2024-05-09T04:35:55.639 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.637+0000 7f38ede00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38e80b61e0 0x7f38e814b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:55.639 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.637+0000 7f38ede00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38e80b61e0 0x7f38e814b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:53180/0 (socket says 172.21.15.145:53180) 2024-05-09T04:35:55.639 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.637+0000 7f38ede00640 1 -- 172.21.15.145:0/227459806 learned_addr learned my addr 172.21.15.145:0/227459806 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:55.639 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.638+0000 7f38ee800640 1 --2- 172.21.15.145:0/227459806 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38e815f210 0x7f38e80b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:55.639 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.638+0000 7f38ede00640 1 -- 172.21.15.145:0/227459806 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38e815f210 msgr2=0x7f38e80b4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:55.639 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.638+0000 7f38ed400640 1 --2- 172.21.15.145:0/227459806 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38e80ce2b0 0x7f38e81571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:55.640 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.638+0000 7f38ede00640 1 --2- 172.21.15.145:0/227459806 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38e815f210 0x7f38e80b4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:55.640 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.638+0000 7f38ede00640 1 -- 172.21.15.145:0/227459806 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38e80ce2b0 msgr2=0x7f38e81571d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:55.640 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.638+0000 7f38ede00640 1 --2- 172.21.15.145:0/227459806 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38e80ce2b0 0x7f38e81571d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:55.640 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.638+0000 7f38ede00640 1 -- 172.21.15.145:0/227459806 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38e8157c20 con 0x7f38e80b61e0 2024-05-09T04:35:55.640 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.638+0000 7f38ede00640 1 --2- 172.21.15.145:0/227459806 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38e80b61e0 0x7f38e814b950 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f38dc00d620 tx=0x7f38dc00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:55.640 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.638+0000 7f38e2000640 1 -- 172.21.15.145:0/227459806 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38dc00b620 con 0x7f38e80b61e0 2024-05-09T04:35:55.640 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.639+0000 7f38e2000640 1 -- 172.21.15.145:0/227459806 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38dc019070 con 0x7f38e80b61e0 2024-05-09T04:35:55.640 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.639+0000 7f38f0800640 1 -- 172.21.15.145:0/227459806 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38e8157f00 con 0x7f38e80b61e0 2024-05-09T04:35:55.641 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.639+0000 7f38e2000640 1 -- 172.21.15.145:0/227459806 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38dc00ba00 con 0x7f38e80b61e0 2024-05-09T04:35:55.641 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.639+0000 7f38f0800640 1 -- 172.21.15.145:0/227459806 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38e8160b20 con 0x7f38e80b61e0 2024-05-09T04:35:55.642 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.639+0000 7f38f0800640 1 -- 172.21.15.145:0/227459806 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f38e806a310 con 0x7f38e80b61e0 2024-05-09T04:35:55.642 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.641+0000 7f38e2000640 1 -- 172.21.15.145:0/227459806 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f38dc00bba0 con 0x7f38e80b61e0 2024-05-09T04:35:55.642 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.642+0000 7f38e2000640 1 --2- 172.21.15.145:0/227459806 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f38cc0762d0 0x7f38cc078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:55.642 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.642+0000 7f38e2000640 1 -- 172.21.15.145:0/227459806 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7f38dc097750 con 0x7f38e80b61e0 2024-05-09T04:35:55.642 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.642+0000 7f38ed400640 1 --2- 172.21.15.145:0/227459806 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f38cc0762d0 0x7f38cc078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:55.642 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.642+0000 7f38e2000640 1 --2- 172.21.15.145:0/227459806 >> [v2:172.21.15.192:6808/3368512157,v1:172.21.15.192:6809/3368512157] conn(0x7f38cc07c190 0x7f38cc07e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:55.643 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.642+0000 7f38e2000640 1 -- 172.21.15.145:0/227459806 --> [v2:172.21.15.192:6808/3368512157,v1:172.21.15.192:6809/3368512157] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f38dc01be70 con 0x7f38cc07c190 2024-05-09T04:35:55.643 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.642+0000 7f38e2000640 1 -- 172.21.15.145:0/227459806 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f38dc09e050 con 0x7f38e80b61e0 2024-05-09T04:35:55.643 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.643+0000 7f38ed400640 1 --2- 172.21.15.145:0/227459806 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f38cc0762d0 0x7f38cc078790 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f38e806d120 tx=0x7f38d8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:55.643 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.643+0000 7f38ee800640 1 --2- 172.21.15.145:0/227459806 >> [v2:172.21.15.192:6808/3368512157,v1:172.21.15.192:6809/3368512157] conn(0x7f38cc07c190 0x7f38cc07e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:55.644 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.644+0000 7f38ee800640 1 --2- 172.21.15.145:0/227459806 >> [v2:172.21.15.192:6808/3368512157,v1:172.21.15.192:6809/3368512157] conn(0x7f38cc07c190 0x7f38cc07e5b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:55.646 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.646+0000 7f38e2000640 1 -- 172.21.15.145:0/227459806 <== osd.7 v2:172.21.15.192:6808/3368512157 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f38dc01be70 con 0x7f38cc07c190 2024-05-09T04:35:55.660 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.661+0000 7f38f0800640 1 -- 172.21.15.145:0/227459806 --> [v2:172.21.15.192:6808/3368512157,v1:172.21.15.192:6809/3368512157] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f38e80b5da0 con 0x7f38cc07c190 2024-05-09T04:35:55.661 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.662+0000 7f38e2000640 1 -- 172.21.15.145:0/227459806 <== osd.7 v2:172.21.15.192:6808/3368512157 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f38e80b5da0 con 0x7f38cc07c190 2024-05-09T04:35:55.661 INFO:teuthology.orchestra.run.smithi145.stdout:{ 2024-05-09T04:35:55.662 INFO:teuthology.orchestra.run.smithi145.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-09T04:35:55.662 INFO:teuthology.orchestra.run.smithi145.stdout:} 2024-05-09T04:35:55.662 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.662+0000 7f38f0800640 1 -- 172.21.15.145:0/227459806 >> [v2:172.21.15.192:6808/3368512157,v1:172.21.15.192:6809/3368512157] conn(0x7f38cc07c190 msgr2=0x7f38cc07e5b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:55.663 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.662+0000 7f38f0800640 1 --2- 172.21.15.145:0/227459806 >> [v2:172.21.15.192:6808/3368512157,v1:172.21.15.192:6809/3368512157] conn(0x7f38cc07c190 0x7f38cc07e5b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:55.663 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.662+0000 7f38f0800640 1 -- 172.21.15.145:0/227459806 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f38cc0762d0 msgr2=0x7f38cc078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:55.663 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.662+0000 7f38f0800640 1 --2- 172.21.15.145:0/227459806 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f38cc0762d0 0x7f38cc078790 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f38e806d120 tx=0x7f38d8008040 comp rx=0 tx=0).stop 2024-05-09T04:35:55.663 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.662+0000 7f38f0800640 1 -- 172.21.15.145:0/227459806 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38e80b61e0 msgr2=0x7f38e814b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:55.663 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.662+0000 7f38f0800640 1 --2- 172.21.15.145:0/227459806 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f38e80b61e0 0x7f38e814b950 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f38dc00d620 tx=0x7f38dc00dae0 comp rx=0 tx=0).stop 2024-05-09T04:35:55.663 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.662+0000 7f38ee800640 1 -- 172.21.15.145:0/227459806 reap_dead start 2024-05-09T04:35:55.663 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.662+0000 7f38f0800640 1 -- 172.21.15.145:0/227459806 shutdown_connections 2024-05-09T04:35:55.663 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.662+0000 7f38f0800640 1 -- 172.21.15.145:0/227459806 >> 172.21.15.145:0/227459806 conn(0x7f38e806c8d0 msgr2=0x7f38e80b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:55.664 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.663+0000 7f38f0800640 1 -- 172.21.15.145:0/227459806 shutdown_connections 2024-05-09T04:35:55.664 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.663+0000 7f38f0800640 1 -- 172.21.15.145:0/227459806 wait complete. 2024-05-09T04:35:55.682 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-05-09T04:35:55.878 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.878+0000 7fad67a00640 1 Processor -- start 2024-05-09T04:35:55.878 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.878+0000 7fad67a00640 1 -- start start 2024-05-09T04:35:55.879 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.880+0000 7fad67a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad600d2120 0x7fad600d58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:55.880 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.880+0000 7fad67a00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fad600b5fe0 0x7fad600d0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:55.880 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.880+0000 7fad67a00640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad600d3780 con 0x7fad60162c00 2024-05-09T04:35:55.880 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.880+0000 7fad67a00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fad600d5ec0 con 0x7fad600b5fe0 2024-05-09T04:35:55.880 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.880+0000 7fad67a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fad600d3a00 con 0x7fad600d2120 2024-05-09T04:35:55.880 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.880+0000 7fad65000640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7fad60162c00 0x7fad600d1a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:60624/0 (socket says 172.21.15.145:60624) 2024-05-09T04:35:55.880 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.880+0000 7fad65000640 1 -- 172.21.15.145:0/2239691871 learned_addr learned my addr 172.21.15.145:0/2239691871 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:35:55.881 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.880+0000 7fad5fe00640 1 -- 172.21.15.145:0/2239691871 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1125208159 0 0) 0x7fad50001110 con 0x7fad60162c00 2024-05-09T04:35:55.882 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.880+0000 7fad5fe00640 1 -- 172.21.15.145:0/2239691871 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fad44003520 con 0x7fad60162c00 2024-05-09T04:35:55.882 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.880+0000 7fad65a00640 1 --2- 172.21.15.145:0/2239691871 >> v2:172.21.15.192:3300/0 conn(0x7fad600b5fe0 0x7fad600d0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:55.882 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.880+0000 7fad57e00640 1 --2- 172.21.15.145:0/2239691871 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad600d2120 0x7fad600d58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:55.882 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.881+0000 7fad5fe00640 1 -- 172.21.15.145:0/2239691871 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3709388895 0 0) 0x7fad44003520 con 0x7fad60162c00 2024-05-09T04:35:55.882 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.881+0000 7fad5fe00640 1 -- 172.21.15.145:0/2239691871 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad600d2120 msgr2=0x7fad600d58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:55.882 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.881+0000 7fad5fe00640 1 --2- 172.21.15.145:0/2239691871 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad600d2120 0x7fad600d58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:55.882 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.881+0000 7fad5fe00640 1 -- 172.21.15.145:0/2239691871 >> v2:172.21.15.192:3300/0 conn(0x7fad600b5fe0 msgr2=0x7fad600d0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:55.882 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.881+0000 7fad5fe00640 1 --2- 172.21.15.145:0/2239691871 >> v2:172.21.15.192:3300/0 conn(0x7fad600b5fe0 0x7fad600d0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:55.883 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.881+0000 7fad5fe00640 1 -- 172.21.15.145:0/2239691871 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad600d3ce0 con 0x7fad60162c00 2024-05-09T04:35:55.883 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.881+0000 7fad5fe00640 1 -- 172.21.15.145:0/2239691871 <== mon.0 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3474413261 0 0) 0x7fad500018c0 con 0x7fad60162c00 2024-05-09T04:35:55.883 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.881+0000 7fad5fe00640 1 -- 172.21.15.145:0/2239691871 >> v1:172.21.15.145:6789/0 conn(0x7fad60162c00 legacy=0x7fad600d1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:55.883 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.881+0000 7fad5fe00640 1 --2- 172.21.15.145:0/2239691871 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fad44003a00 0x7fad44003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:55.883 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.881+0000 7fad5fe00640 1 --2- 172.21.15.145:0/2239691871 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad600d2120 0x7fad44004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:55.883 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.881+0000 7fad5fe00640 1 --2- 172.21.15.145:0/2239691871 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad44004d40 0x7fad440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:55.883 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.881+0000 7fad5fe00640 1 -- 172.21.15.145:0/2239691871 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fad600d5ec0 con 0x7fad44003a00 2024-05-09T04:35:55.883 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.881+0000 7fad5fe00640 1 -- 172.21.15.145:0/2239691871 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fad600d3a00 con 0x7fad44004d40 2024-05-09T04:35:55.883 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.881+0000 7fad5fe00640 1 -- 172.21.15.145:0/2239691871 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fad4400bc00 con 0x7fad600d2120 2024-05-09T04:35:55.886 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.881+0000 7fad65000640 1 --2- 172.21.15.145:0/2239691871 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fad44003a00 0x7fad44003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:55.886 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.881+0000 7fad57e00640 1 --2- 172.21.15.145:0/2239691871 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad600d2120 0x7fad44004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:55.886 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.881+0000 7fad65000640 1 -- 172.21.15.145:0/2239691871 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad600d2120 msgr2=0x7fad44004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:55.886 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.881+0000 7fad65000640 1 --2- 172.21.15.145:0/2239691871 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad600d2120 0x7fad44004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:55.886 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.881+0000 7fad65000640 1 -- 172.21.15.145:0/2239691871 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad44004d40 msgr2=0x7fad440051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:55.886 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.881+0000 7fad65000640 1 --2- 172.21.15.145:0/2239691871 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad44004d40 0x7fad440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:55.886 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.881+0000 7fad65000640 1 -- 172.21.15.145:0/2239691871 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad44003520 con 0x7fad44003a00 2024-05-09T04:35:55.889 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.882+0000 7fad65000640 1 --2- 172.21.15.145:0/2239691871 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fad44003a00 0x7fad44003e20 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fad5000c3e0 tx=0x7fad44005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be793d969992e338 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:55.889 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.883+0000 7fad5fe00640 1 -- 172.21.15.145:0/2239691871 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5000c0e0 con 0x7fad44003a00 2024-05-09T04:35:55.889 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.883+0000 7fad5fe00640 1 -- 172.21.15.145:0/2239691871 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad50034070 con 0x7fad44003a00 2024-05-09T04:35:55.889 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.883+0000 7fad5fe00640 1 -- 172.21.15.145:0/2239691871 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad50005d10 con 0x7fad44003a00 2024-05-09T04:35:55.889 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.883+0000 7fad67a00640 1 -- 172.21.15.145:0/2239691871 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fad44003a00 msgr2=0x7fad44003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:55.890 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.883+0000 7fad67a00640 1 --2- 172.21.15.145:0/2239691871 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fad44003a00 0x7fad44003e20 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fad5000c3e0 tx=0x7fad44005830 comp rx=0 tx=0).stop 2024-05-09T04:35:55.890 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.883+0000 7fad65a00640 1 -- 172.21.15.145:0/2239691871 reap_dead start 2024-05-09T04:35:55.890 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.884+0000 7fad67a00640 1 -- 172.21.15.145:0/2239691871 shutdown_connections 2024-05-09T04:35:55.890 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.884+0000 7fad67a00640 1 -- 172.21.15.145:0/2239691871 >> 172.21.15.145:0/2239691871 conn(0x7fad6009e510 msgr2=0x7fad600d51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:55.890 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.884+0000 7fad67a00640 1 -- 172.21.15.145:0/2239691871 shutdown_connections 2024-05-09T04:35:55.890 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.884+0000 7fad67a00640 1 -- 172.21.15.145:0/2239691871 wait complete. 2024-05-09T04:35:55.890 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.885+0000 7fad67a00640 1 Processor -- start 2024-05-09T04:35:55.890 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.886+0000 7fad67a00640 1 -- start start 2024-05-09T04:35:55.890 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.886+0000 7fad67a00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fad44003a00 0x7fad601f5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:55.891 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.886+0000 7fad67a00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad60161240 0x7fad601616c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:55.891 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.886+0000 7fad67a00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad600b9150 0x7fad600b9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:55.891 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.886+0000 7fad67a00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fad600b6670 con 0x7fad44003a00 2024-05-09T04:35:55.891 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.886+0000 7fad67a00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fad600b4950 con 0x7fad60161240 2024-05-09T04:35:55.891 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.886+0000 7fad67a00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fad600c8b60 con 0x7fad600b9150 2024-05-09T04:35:55.891 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.887+0000 7fad65000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fad44003a00 0x7fad601f5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:55.891 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.887+0000 7fad65000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fad44003a00 0x7fad601f5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:53204/0 (socket says 172.21.15.145:53204) 2024-05-09T04:35:55.891 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.887+0000 7fad65000640 1 -- 172.21.15.145:0/485497281 learned_addr learned my addr 172.21.15.145:0/485497281 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:55.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.887+0000 7fad65000640 1 -- 172.21.15.145:0/485497281 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad600b9150 msgr2=0x7fad600b9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:55.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.887+0000 7fad65a00640 1 --2- 172.21.15.145:0/485497281 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad600b9150 0x7fad600b9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:55.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.887+0000 7fad57e00640 1 --2- 172.21.15.145:0/485497281 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad60161240 0x7fad601616c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:55.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.887+0000 7fad65000640 1 --2- 172.21.15.145:0/485497281 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad600b9150 0x7fad600b9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:55.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.887+0000 7fad65000640 1 -- 172.21.15.145:0/485497281 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad60161240 msgr2=0x7fad601616c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:55.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.887+0000 7fad65000640 1 --2- 172.21.15.145:0/485497281 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad60161240 0x7fad601616c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:55.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.887+0000 7fad65000640 1 -- 172.21.15.145:0/485497281 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad600b27e0 con 0x7fad44003a00 2024-05-09T04:35:55.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.888+0000 7fad65000640 1 --2- 172.21.15.145:0/485497281 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fad44003a00 0x7fad601f5640 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fad5000c500 tx=0x7fad50001c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:55.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.888+0000 7fad5d600640 1 -- 172.21.15.145:0/485497281 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad50007070 con 0x7fad44003a00 2024-05-09T04:35:55.892 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.888+0000 7fad5d600640 1 -- 172.21.15.145:0/485497281 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad50007230 con 0x7fad44003a00 2024-05-09T04:35:55.893 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.888+0000 7fad67a00640 1 -- 172.21.15.145:0/485497281 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad600b2ab0 con 0x7fad44003a00 2024-05-09T04:35:55.893 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.888+0000 7fad5d600640 1 -- 172.21.15.145:0/485497281 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad50007620 con 0x7fad44003a00 2024-05-09T04:35:55.893 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.888+0000 7fad67a00640 1 -- 172.21.15.145:0/485497281 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad600b3070 con 0x7fad44003a00 2024-05-09T04:35:55.893 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.889+0000 7fad67a00640 1 -- 172.21.15.145:0/485497281 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad6006a310 con 0x7fad44003a00 2024-05-09T04:35:55.894 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.890+0000 7fad5d600640 1 -- 172.21.15.145:0/485497281 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fad5002e070 con 0x7fad44003a00 2024-05-09T04:35:55.894 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.891+0000 7fad5d600640 1 --2- 172.21.15.145:0/485497281 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fad3c076300 0x7fad3c0787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:55.894 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.891+0000 7fad5d600640 1 -- 172.21.15.145:0/485497281 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7fad500b7df0 con 0x7fad44003a00 2024-05-09T04:35:55.894 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.891+0000 7fad57e00640 1 --2- 172.21.15.145:0/485497281 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fad3c076300 0x7fad3c0787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:55.894 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.892+0000 7fad57e00640 1 --2- 172.21.15.145:0/485497281 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fad3c076300 0x7fad3c0787c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fad6015fef0 tx=0x7fad48008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:55.895 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:55.896+0000 7fad5d600640 1 -- 172.21.15.145:0/485497281 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fad50049060 con 0x7fad44003a00 2024-05-09T04:35:56.099 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.100+0000 7fad67a00640 1 -- 172.21.15.145:0/485497281 --> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7fad600b6aa0 con 0x7fad3c076300 2024-05-09T04:35:56.100 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.100+0000 7fad5d600640 1 -- 172.21.15.145:0/485497281 <== mgr.4104 v2:172.21.15.145:6832/2725855396 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fad600b6aa0 con 0x7fad3c076300 2024-05-09T04:35:56.100 INFO:teuthology.orchestra.run.smithi145.stderr:instructed osd(s) 7 to deep-scrub 2024-05-09T04:35:56.103 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.103+0000 7fad67a00640 1 -- 172.21.15.145:0/485497281 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fad3c076300 msgr2=0x7fad3c0787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:56.103 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.103+0000 7fad67a00640 1 --2- 172.21.15.145:0/485497281 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fad3c076300 0x7fad3c0787c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fad6015fef0 tx=0x7fad48008040 comp rx=0 tx=0).stop 2024-05-09T04:35:56.103 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.103+0000 7fad67a00640 1 -- 172.21.15.145:0/485497281 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fad44003a00 msgr2=0x7fad601f5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:56.103 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.103+0000 7fad67a00640 1 --2- 172.21.15.145:0/485497281 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fad44003a00 0x7fad601f5640 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fad5000c500 tx=0x7fad50001c00 comp rx=0 tx=0).stop 2024-05-09T04:35:56.103 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.103+0000 7fad67a00640 1 -- 172.21.15.145:0/485497281 shutdown_connections 2024-05-09T04:35:56.103 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.103+0000 7fad67a00640 1 --2- 172.21.15.145:0/485497281 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fad3c076300 0x7fad3c0787c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:56.103 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.103+0000 7fad67a00640 1 --2- 172.21.15.145:0/485497281 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fad600b9150 0x7fad600b9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:56.104 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.103+0000 7fad67a00640 1 --2- 172.21.15.145:0/485497281 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fad60161240 0x7fad601616c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:56.104 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.103+0000 7fad67a00640 1 --2- 172.21.15.145:0/485497281 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fad44003a00 0x7fad601f5640 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:56.104 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.103+0000 7fad67a00640 1 -- 172.21.15.145:0/485497281 >> 172.21.15.145:0/485497281 conn(0x7fad6009e510 msgr2=0x7fad6015e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:56.104 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.103+0000 7fad67a00640 1 -- 172.21.15.145:0/485497281 shutdown_connections 2024-05-09T04:35:56.104 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.103+0000 7fad67a00640 1 -- 172.21.15.145:0/485497281 wait complete. 2024-05-09T04:35:56.122 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T04:35:56.323 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.323+0000 7f4660000640 1 Processor -- start 2024-05-09T04:35:56.323 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.323+0000 7f4660000640 1 -- start start 2024-05-09T04:35:56.325 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.324+0000 7f4660000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f465815fbc0 0x7f4658142200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:56.325 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.324+0000 7f4660000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f46580726b0 0x7f4658072b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:56.325 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.324+0000 7f4660000640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f465813f340 con 0x7f465815ffa0 2024-05-09T04:35:56.325 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.324+0000 7f4660000640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f4658072220 con 0x7f46580726b0 2024-05-09T04:35:56.325 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.324+0000 7f4660000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f46580cffe0 con 0x7f465815fbc0 2024-05-09T04:35:56.325 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.325+0000 7f465e000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f46580726b0 0x7f4658072b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:56.325 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.325+0000 7f465e000640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f46580726b0 0x7f4658072b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:53212/0 (socket says 172.21.15.145:53212) 2024-05-09T04:35:56.325 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.325+0000 7f465e000640 1 -- 172.21.15.145:0/4175224572 learned_addr learned my addr 172.21.15.145:0/4175224572 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:56.326 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.325+0000 7f465d600640 1 --2- 172.21.15.145:0/4175224572 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f465815fbc0 0x7f4658142200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:56.326 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.325+0000 7f465e000640 1 -- 172.21.15.145:0/4175224572 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f465815fbc0 msgr2=0x7f4658142200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:56.326 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.325+0000 7f465e000640 1 --2- 172.21.15.145:0/4175224572 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f465815fbc0 0x7f4658142200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:56.326 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.325+0000 7f465e000640 1 -- 172.21.15.145:0/4175224572 >> v1:172.21.15.192:6789/0 conn(0x7f465815ffa0 legacy=0x7f46580cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:35:56.326 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.325+0000 7f465e000640 1 -- 172.21.15.145:0/4175224572 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46580d02b0 con 0x7f46580726b0 2024-05-09T04:35:56.327 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.326+0000 7f465e000640 1 --2- 172.21.15.145:0/4175224572 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46580726b0 0x7f4658072b40 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f4650009e50 tx=0x7f465002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72e52a1c7acd033 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:56.327 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.326+0000 7f4657e00640 1 -- 172.21.15.145:0/4175224572 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4650036070 con 0x7f46580726b0 2024-05-09T04:35:56.327 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.326+0000 7f4657e00640 1 -- 172.21.15.145:0/4175224572 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4650031600 con 0x7f46580726b0 2024-05-09T04:35:56.328 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.327+0000 7f4657e00640 1 -- 172.21.15.145:0/4175224572 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46500319d0 con 0x7f46580726b0 2024-05-09T04:35:56.328 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.327+0000 7f4660000640 1 -- 172.21.15.145:0/4175224572 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46580726b0 msgr2=0x7f4658072b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:56.328 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.327+0000 7f4660000640 1 --2- 172.21.15.145:0/4175224572 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46580726b0 0x7f4658072b40 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f4650009e50 tx=0x7f465002a5f0 comp rx=0 tx=0).stop 2024-05-09T04:35:56.328 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.327+0000 7f4660000640 1 -- 172.21.15.145:0/4175224572 shutdown_connections 2024-05-09T04:35:56.328 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.327+0000 7f4660000640 1 --2- 172.21.15.145:0/4175224572 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f46580726b0 0x7f4658072b40 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:56.328 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.327+0000 7f4660000640 1 --2- 172.21.15.145:0/4175224572 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f465815fbc0 0x7f4658142200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:56.328 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.327+0000 7f4660000640 1 -- 172.21.15.145:0/4175224572 >> 172.21.15.145:0/4175224572 conn(0x7f465806c870 msgr2=0x7f465806cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:56.328 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.327+0000 7f4660000640 1 -- 172.21.15.145:0/4175224572 shutdown_connections 2024-05-09T04:35:56.328 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.327+0000 7f4660000640 1 -- 172.21.15.145:0/4175224572 wait complete. 2024-05-09T04:35:56.329 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.329+0000 7f4660000640 1 Processor -- start 2024-05-09T04:35:56.329 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.329+0000 7f4660000640 1 -- start start 2024-05-09T04:35:56.330 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.330+0000 7f4660000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f46580726b0 0x7f46581e82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:56.330 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.330+0000 7f4660000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f465815fbc0 0x7f4658152270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:56.330 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.330+0000 7f4660000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f465815ffa0 0x7f4658140570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:56.331 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.330+0000 7f4660000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f46580d0b50 con 0x7f465815ffa0 2024-05-09T04:35:56.331 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.330+0000 7f4660000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f46580d0870 con 0x7f465815fbc0 2024-05-09T04:35:56.331 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.330+0000 7f4660000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f4658160400 con 0x7f46580726b0 2024-05-09T04:35:56.331 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.331+0000 7f465e000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f465815ffa0 0x7f4658140570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:56.331 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.331+0000 7f465e000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f465815ffa0 0x7f4658140570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:53222/0 (socket says 172.21.15.145:53222) 2024-05-09T04:35:56.331 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.331+0000 7f465e000640 1 -- 172.21.15.145:0/4022225173 learned_addr learned my addr 172.21.15.145:0/4022225173 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:35:56.331 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.331+0000 7f465cc00640 1 --2- 172.21.15.145:0/4022225173 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f465815fbc0 0x7f4658152270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:56.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.331+0000 7f465e000640 1 -- 172.21.15.145:0/4022225173 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f46580726b0 msgr2=0x7f46581e82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:56.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.331+0000 7f465e000640 1 --2- 172.21.15.145:0/4022225173 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f46580726b0 0x7f46581e82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:56.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.331+0000 7f465e000640 1 -- 172.21.15.145:0/4022225173 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f465815fbc0 msgr2=0x7f4658152270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:56.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.331+0000 7f465e000640 1 --2- 172.21.15.145:0/4022225173 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f465815fbc0 0x7f4658152270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:56.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.331+0000 7f465e000640 1 -- 172.21.15.145:0/4022225173 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46581ea0b0 con 0x7f465815ffa0 2024-05-09T04:35:56.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.331+0000 7f465cc00640 1 --2- 172.21.15.145:0/4022225173 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f465815fbc0 0x7f4658152270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:35:56.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.331+0000 7f465e000640 1 --2- 172.21.15.145:0/4022225173 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f465815ffa0 0x7f4658140570 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f4650034040 tx=0x7f4650004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:56.333 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.332+0000 7f4655600640 1 -- 172.21.15.145:0/4022225173 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465003f020 con 0x7f465815ffa0 2024-05-09T04:35:56.334 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.332+0000 7f4655600640 1 -- 172.21.15.145:0/4022225173 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4650037040 con 0x7f465815ffa0 2024-05-09T04:35:56.334 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.332+0000 7f4655600640 1 -- 172.21.15.145:0/4022225173 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4650036070 con 0x7f465815ffa0 2024-05-09T04:35:56.334 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.332+0000 7f4660000640 1 -- 172.21.15.145:0/4022225173 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46581ea390 con 0x7f465815ffa0 2024-05-09T04:35:56.334 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.332+0000 7f4660000640 1 -- 172.21.15.145:0/4022225173 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46581ea950 con 0x7f465815ffa0 2024-05-09T04:35:56.334 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.333+0000 7f4660000640 1 -- 172.21.15.145:0/4022225173 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f465806a310 con 0x7f465815ffa0 2024-05-09T04:35:56.335 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.334+0000 7f4655600640 1 -- 172.21.15.145:0/4022225173 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f4650042020 con 0x7f465815ffa0 2024-05-09T04:35:56.335 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.335+0000 7f4655600640 1 --2- 172.21.15.145:0/4022225173 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f4638076300 0x7f46380787c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:35:56.335 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.335+0000 7f4655600640 1 -- 172.21.15.145:0/4022225173 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7f46500b7a30 con 0x7f465815ffa0 2024-05-09T04:35:56.335 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.335+0000 7f465d600640 1 --2- 172.21.15.145:0/4022225173 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f4638076300 0x7f46380787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:35:56.336 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.336+0000 7f465d600640 1 --2- 172.21.15.145:0/4022225173 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f4638076300 0x7f46380787c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f464c006fd0 tx=0x7f464c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:35:56.339 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.340+0000 7f4655600640 1 -- 172.21.15.145:0/4022225173 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f4650049020 con 0x7f465815ffa0 2024-05-09T04:35:56.548 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.548+0000 7f4660000640 1 -- 172.21.15.145:0/4022225173 --> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f46580b4070 con 0x7f4638076300 2024-05-09T04:35:56.550 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.550+0000 7f4655600640 1 -- 172.21.15.145:0/4022225173 <== mgr.4104 v2:172.21.15.145:6832/2725855396 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13340 (secure 0 0 0) 0x7f46580b4070 con 0x7f4638076300 2024-05-09T04:35:56.550 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:35:56.551 INFO:teuthology.orchestra.run.smithi145.stderr:dumped all 2024-05-09T04:35:56.553 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.553+0000 7f4660000640 1 -- 172.21.15.145:0/4022225173 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f4638076300 msgr2=0x7f46380787c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:56.553 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.553+0000 7f4660000640 1 --2- 172.21.15.145:0/4022225173 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f4638076300 0x7f46380787c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f464c006fd0 tx=0x7f464c008040 comp rx=0 tx=0).stop 2024-05-09T04:35:56.553 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.553+0000 7f4660000640 1 -- 172.21.15.145:0/4022225173 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f465815ffa0 msgr2=0x7f4658140570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:35:56.553 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.553+0000 7f4660000640 1 --2- 172.21.15.145:0/4022225173 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f465815ffa0 0x7f4658140570 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f4650034040 tx=0x7f4650004a40 comp rx=0 tx=0).stop 2024-05-09T04:35:56.554 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.553+0000 7f4660000640 1 -- 172.21.15.145:0/4022225173 shutdown_connections 2024-05-09T04:35:56.554 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.553+0000 7f4660000640 1 --2- 172.21.15.145:0/4022225173 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f4638076300 0x7f46380787c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:56.554 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.553+0000 7f4660000640 1 --2- 172.21.15.145:0/4022225173 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f465815ffa0 0x7f4658140570 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:56.554 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.553+0000 7f4660000640 1 --2- 172.21.15.145:0/4022225173 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f465815fbc0 0x7f4658152270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:56.554 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.553+0000 7f4660000640 1 --2- 172.21.15.145:0/4022225173 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f46580726b0 0x7f46581e82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:35:56.554 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.553+0000 7f4660000640 1 -- 172.21.15.145:0/4022225173 >> 172.21.15.145:0/4022225173 conn(0x7f465806c870 msgr2=0x7f465806cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:35:56.554 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.553+0000 7f4660000640 1 -- 172.21.15.145:0/4022225173 shutdown_connections 2024-05-09T04:35:56.554 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:35:56.553+0000 7f4660000640 1 -- 172.21.15.145:0/4022225173 wait complete. 2024-05-09T04:35:56.572 INFO:teuthology.orchestra.run.smithi145.stdout:{"pg_ready":true,"pg_map":{"version":116,"stamp":"2024-05-09T04:35:55.792019+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":239220,"kb_used_data":6376,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735500,"statfs":{"total":773094113280,"available":772849152000,"internally_reserved":0,"allocated":6529024,"data_stored":3433176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.361253"},"pg_stats":[{"pgid":"1.0","version":"16'72","reported_seq":176,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-09T04:35:45.437947+0000","last_change":"2024-05-09T04:33:41.092063+0000","last_active":"2024-05-09T04:35:45.437947+0000","last_peered":"2024-05-09T04:35:45.437947+0000","last_clean":"2024-05-09T04:35:45.437947+0000","last_became_active":"2024-05-09T04:33:41.091194+0000","last_became_peered":"2024-05-09T04:33:41.091194+0000","last_unstale":"2024-05-09T04:35:45.437947+0000","last_undegraded":"2024-05-09T04:35:45.437947+0000","last_fullsized":"2024-05-09T04:35:45.437947+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:33:40.045339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:33:40.045339+0000","last_clean_scrub_stamp":"2024-05-09T04:33:40.045339+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:42:04.908456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574880,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28256,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343584,"statfs":{"total":96636764160,"available":96607830016,"internally_reserved":0,"allocated":1261568,"data_stored":871923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27684,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344156,"statfs":{"total":96636764160,"available":96608415744,"internally_reserved":0,"allocated":667648,"data_stored":281555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27680,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":667648,"data_stored":281555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574878,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31836,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340004,"statfs":{"total":96636764160,"available":96604164096,"internally_reserved":0,"allocated":667648,"data_stored":281555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574878,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31832,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340008,"statfs":{"total":96636764160,"available":96604168192,"internally_reserved":0,"allocated":667648,"data_stored":281555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27680,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":667648,"data_stored":281555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31836,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340004,"statfs":{"total":96636764160,"available":96604164096,"internally_reserved":0,"allocated":667648,"data_stored":281555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574878,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32416,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339424,"statfs":{"total":96636764160,"available":96603570176,"internally_reserved":0,"allocated":1261568,"data_stored":871923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T04:35:56.575 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-05-09T04:33:40.045339+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=4, tm_min=33, tm_sec=40, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=4, tm_min=35, tm_sec=49, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T04:35:56.575 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-05-09T04:36:04.192 DEBUG:teuthology.orchestra.run.smithi145:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T04:36:04.201 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T04:36:16.576 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T04:36:16.785 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:16.785+0000 7ffa4a000640 1 Processor -- start 2024-05-09T04:36:16.785 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:16.785+0000 7ffa4a000640 1 -- start start 2024-05-09T04:36:16.787 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:16.787+0000 7ffa4a000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa440b4700 0x7ffa44073c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:36:16.787 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:16.787+0000 7ffa4a000640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ffa440af610 0x7ffa440afa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:36:16.787 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:16.787+0000 7ffa4a000640 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa44072c00 con 0x7ffa440b4ae0 2024-05-09T04:36:16.787 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:16.787+0000 7ffa4a000640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7ffa440aea70 con 0x7ffa440af610 2024-05-09T04:36:16.788 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:16.787+0000 7ffa4a000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ffa440737f0 con 0x7ffa440b4700 2024-05-09T04:36:16.788 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:16.787+0000 7ffa49600640 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7ffa440b4ae0 0x7ffa440b4f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:58572/0 (socket says 172.21.15.145:58572) 2024-05-09T04:36:16.788 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:16.787+0000 7ffa49600640 1 -- 172.21.15.145:0/1797041385 learned_addr learned my addr 172.21.15.145:0/1797041385 (peer_addr_for_me v1:172.21.15.145:0/0) 2024-05-09T04:36:16.789 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:16.788+0000 7ffa43e00640 1 --2- 172.21.15.145:0/1797041385 >> v2:172.21.15.192:3300/0 conn(0x7ffa440af610 0x7ffa440afa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:36:16.789 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:16.788+0000 7ffa48c00640 1 --2- 172.21.15.145:0/1797041385 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa440b4700 0x7ffa44073c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:36:16.789 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:16.788+0000 7ffa43400640 1 -- 172.21.15.145:0/1797041385 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4215220209 0 0) 0x7ffa44072c00 con 0x7ffa440b4ae0 2024-05-09T04:36:16.789 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:16.788+0000 7ffa43400640 1 -- 172.21.15.145:0/1797041385 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa2c003520 con 0x7ffa440b4ae0 2024-05-09T04:36:16.789 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:16.788+0000 7ffa43400640 1 -- 172.21.15.145:0/1797041385 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4243074564 0 0) 0x7ffa2c003520 con 0x7ffa440b4ae0 2024-05-09T04:36:16.789 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:16.788+0000 7ffa43e00640 1 -- 172.21.15.145:0/1797041385 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa440b4700 msgr2=0x7ffa44073c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:36:16.790 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:16.788+0000 7ffa43e00640 1 --2- 172.21.15.145:0/1797041385 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa440b4700 0x7ffa44073c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:36:16.790 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:16.789+0000 7ffa43e00640 1 -- 172.21.15.145:0/1797041385 >> v1:172.21.15.145:6789/0 conn(0x7ffa440b4ae0 legacy=0x7ffa440b4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:36:16.790 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:16.789+0000 7ffa43e00640 1 -- 172.21.15.145:0/1797041385 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa44072ec0 con 0x7ffa440af610 2024-05-09T04:36:16.791 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:16.790+0000 7ffa43e00640 1 --2- 172.21.15.145:0/1797041385 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa440af610 0x7ffa440afa10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ffa300097a0 tx=0x7ffa3002a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8dc2dce5b15fa7cb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:36:16.791 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:16.790+0000 7ffa43400640 1 -- 172.21.15.145:0/1797041385 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa30037070 con 0x7ffa440af610 2024-05-09T04:36:16.791 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:16.790+0000 7ffa43400640 1 -- 172.21.15.145:0/1797041385 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa3002aba0 con 0x7ffa440af610 2024-05-09T04:36:16.792 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:16.790+0000 7ffa43400640 1 -- 172.21.15.145:0/1797041385 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c001a70 con 0x7ffa440af610 2024-05-09T04:36:16.792 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:16.791+0000 7ffa4a000640 1 -- 172.21.15.145:0/1797041385 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa440af610 msgr2=0x7ffa440afa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:36:16.792 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:16.791+0000 7ffa4a000640 1 --2- 172.21.15.145:0/1797041385 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa440af610 0x7ffa440afa10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ffa300097a0 tx=0x7ffa3002a110 comp rx=0 tx=0).stop 2024-05-09T04:36:16.792 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:16.791+0000 7ffa4a000640 1 -- 172.21.15.145:0/1797041385 shutdown_connections 2024-05-09T04:36:16.792 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:16.791+0000 7ffa4a000640 1 --2- 172.21.15.145:0/1797041385 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa440af610 0x7ffa440afa10 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:36:16.792 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:16.791+0000 7ffa4a000640 1 --2- 172.21.15.145:0/1797041385 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa440b4700 0x7ffa44073c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:36:16.792 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:16.791+0000 7ffa4a000640 1 -- 172.21.15.145:0/1797041385 >> 172.21.15.145:0/1797041385 conn(0x7ffa4406c8a0 msgr2=0x7ffa4406ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:36:16.793 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:16.791+0000 7ffa4a000640 1 -- 172.21.15.145:0/1797041385 shutdown_connections 2024-05-09T04:36:16.793 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:16.791+0000 7ffa4a000640 1 -- 172.21.15.145:0/1797041385 wait complete. 2024-05-09T04:36:16.794 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:16.793+0000 7ffa4a000640 1 Processor -- start 2024-05-09T04:36:16.794 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:16.793+0000 7ffa4a000640 1 -- start start 2024-05-09T04:36:16.794 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:16.794+0000 7ffa4a000640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffa440af610 0x7ffa441e9120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:36:16.794 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:16.794+0000 7ffa4a000640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa440b4700 0x7ffa4409ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:36:16.795 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:16.794+0000 7ffa4a000640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa440b4ae0 0x7ffa441e76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:36:16.795 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:16.794+0000 7ffa4a000640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7ffa440741c0 con 0x7ffa440af610 2024-05-09T04:36:16.795 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:16.794+0000 7ffa4a000640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7ffa44073480 con 0x7ffa440b4700 2024-05-09T04:36:16.795 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:16.794+0000 7ffa4a000640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7ffa4406e320 con 0x7ffa440b4ae0 2024-05-09T04:36:16.795 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:16.794+0000 7ffa48c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffa440af610 0x7ffa441e9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:36:16.795 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:16.794+0000 7ffa48c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffa440af610 0x7ffa441e9120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:47546/0 (socket says 172.21.15.145:47546) 2024-05-09T04:36:16.795 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:16.794+0000 7ffa48c00640 1 -- 172.21.15.145:0/2440086712 learned_addr learned my addr 172.21.15.145:0/2440086712 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:36:16.796 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:16.795+0000 7ffa48c00640 1 -- 172.21.15.145:0/2440086712 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa440b4ae0 msgr2=0x7ffa441e76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:36:16.797 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:16.795+0000 7ffa48c00640 1 --2- 172.21.15.145:0/2440086712 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa440b4ae0 0x7ffa441e76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:36:16.797 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:16.795+0000 7ffa48c00640 1 -- 172.21.15.145:0/2440086712 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa440b4700 msgr2=0x7ffa4409ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:36:16.797 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:16.795+0000 7ffa48c00640 1 --2- 172.21.15.145:0/2440086712 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa440b4700 0x7ffa4409ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:36:16.797 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:16.795+0000 7ffa48c00640 1 -- 172.21.15.145:0/2440086712 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa4415eff0 con 0x7ffa440af610 2024-05-09T04:36:16.797 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:16.795+0000 7ffa48c00640 1 --2- 172.21.15.145:0/2440086712 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffa440af610 0x7ffa441e9120 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7ffa3800e9e0 tx=0x7ffa3800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:36:16.797 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:16.795+0000 7ffa40c00640 1 -- 172.21.15.145:0/2440086712 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa38017070 con 0x7ffa440af610 2024-05-09T04:36:16.797 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:16.795+0000 7ffa40c00640 1 -- 172.21.15.145:0/2440086712 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa3800c9d0 con 0x7ffa440af610 2024-05-09T04:36:16.798 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:16.795+0000 7ffa4a000640 1 -- 172.21.15.145:0/2440086712 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa4415f2d0 con 0x7ffa440af610 2024-05-09T04:36:16.798 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:16.795+0000 7ffa40c00640 1 -- 172.21.15.145:0/2440086712 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3800cdc0 con 0x7ffa440af610 2024-05-09T04:36:16.798 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:16.795+0000 7ffa4a000640 1 -- 172.21.15.145:0/2440086712 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa4415fb70 con 0x7ffa440af610 2024-05-09T04:36:16.798 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:16.796+0000 7ffa4a000640 1 -- 172.21.15.145:0/2440086712 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa4406a310 con 0x7ffa440af610 2024-05-09T04:36:16.799 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:16.798+0000 7ffa40c00640 1 -- 172.21.15.145:0/2440086712 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7ffa3801a070 con 0x7ffa440af610 2024-05-09T04:36:16.799 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:16.799+0000 7ffa40c00640 1 --2- 172.21.15.145:0/2440086712 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ffa240762d0 0x7ffa24078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:36:16.799 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:16.799+0000 7ffa40c00640 1 -- 172.21.15.145:0/2440086712 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7ffa38097750 con 0x7ffa440af610 2024-05-09T04:36:16.799 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:16.799+0000 7ffa43e00640 1 --2- 172.21.15.145:0/2440086712 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ffa240762d0 0x7ffa24078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:36:16.800 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:16.800+0000 7ffa43e00640 1 --2- 172.21.15.145:0/2440086712 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ffa240762d0 0x7ffa24078790 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ffa30034000 tx=0x7ffa3002c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:36:16.804 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:16.805+0000 7ffa40c00640 1 -- 172.21.15.145:0/2440086712 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ffa38061310 con 0x7ffa440af610 2024-05-09T04:36:17.013 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.013+0000 7ffa4a000640 1 -- 172.21.15.145:0/2440086712 --> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ffa440b20b0 con 0x7ffa240762d0 2024-05-09T04:36:17.015 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.015+0000 7ffa40c00640 1 -- 172.21.15.145:0/2440086712 <== mgr.4104 v2:172.21.15.145:6832/2725855396 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13344 (secure 0 0 0) 0x7ffa440b20b0 con 0x7ffa240762d0 2024-05-09T04:36:17.015 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:36:17.016 INFO:teuthology.orchestra.run.smithi145.stderr:dumped all 2024-05-09T04:36:17.018 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.018+0000 7ffa4a000640 1 -- 172.21.15.145:0/2440086712 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ffa240762d0 msgr2=0x7ffa24078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:36:17.018 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.018+0000 7ffa4a000640 1 --2- 172.21.15.145:0/2440086712 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ffa240762d0 0x7ffa24078790 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ffa30034000 tx=0x7ffa3002c000 comp rx=0 tx=0).stop 2024-05-09T04:36:17.018 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.018+0000 7ffa4a000640 1 -- 172.21.15.145:0/2440086712 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffa440af610 msgr2=0x7ffa441e9120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:36:17.018 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.018+0000 7ffa4a000640 1 --2- 172.21.15.145:0/2440086712 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffa440af610 0x7ffa441e9120 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7ffa3800e9e0 tx=0x7ffa3800eea0 comp rx=0 tx=0).stop 2024-05-09T04:36:17.018 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.018+0000 7ffa4a000640 1 -- 172.21.15.145:0/2440086712 shutdown_connections 2024-05-09T04:36:17.018 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.018+0000 7ffa4a000640 1 --2- 172.21.15.145:0/2440086712 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7ffa240762d0 0x7ffa24078790 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:36:17.019 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.018+0000 7ffa4a000640 1 --2- 172.21.15.145:0/2440086712 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa440b4ae0 0x7ffa441e76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:36:17.019 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.018+0000 7ffa4a000640 1 --2- 172.21.15.145:0/2440086712 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa440b4700 0x7ffa4409ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:36:17.019 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.018+0000 7ffa4a000640 1 --2- 172.21.15.145:0/2440086712 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffa440af610 0x7ffa441e9120 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:36:17.019 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.018+0000 7ffa4a000640 1 -- 172.21.15.145:0/2440086712 >> 172.21.15.145:0/2440086712 conn(0x7ffa4406c8a0 msgr2=0x7ffa440afaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:36:17.019 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.018+0000 7ffa4a000640 1 -- 172.21.15.145:0/2440086712 shutdown_connections 2024-05-09T04:36:17.020 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.019+0000 7ffa4a000640 1 -- 172.21.15.145:0/2440086712 wait complete. 2024-05-09T04:36:17.038 INFO:teuthology.orchestra.run.smithi145.stdout:{"pg_ready":true,"pg_map":{"version":126,"stamp":"2024-05-09T04:36:15.798019+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":239204,"kb_used_data":6376,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735516,"statfs":{"total":773094113280,"available":772849168384,"internally_reserved":0,"allocated":6529024,"data_stored":3433176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003654"},"pg_stats":[{"pgid":"1.0","version":"55'74","reported_seq":187,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-05-09T04:35:56.461532+0000","last_change":"2024-05-09T04:35:56.461668+0000","last_active":"2024-05-09T04:35:56.461532+0000","last_peered":"2024-05-09T04:35:56.461532+0000","last_clean":"2024-05-09T04:35:56.461532+0000","last_became_active":"2024-05-09T04:33:41.091194+0000","last_became_peered":"2024-05-09T04:33:41.091194+0000","last_unstale":"2024-05-09T04:35:56.461532+0000","last_undegraded":"2024-05-09T04:35:56.461532+0000","last_fullsized":"2024-05-09T04:35:56.461532+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'74","last_scrub_stamp":"2024-05-09T04:35:56.461297+0000","last_deep_scrub":"55'74","last_deep_scrub_stamp":"2024-05-09T04:35:56.461297+0000","last_clean_scrub_stamp":"2024-05-09T04:35:56.461297+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:06:24.907729+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574884,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28248,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343592,"statfs":{"total":96636764160,"available":96607838208,"internally_reserved":0,"allocated":1261568,"data_stored":871923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574883,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27684,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344156,"statfs":{"total":96636764160,"available":96608415744,"internally_reserved":0,"allocated":667648,"data_stored":281555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27680,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":667648,"data_stored":281555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574882,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31836,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340004,"statfs":{"total":96636764160,"available":96604164096,"internally_reserved":0,"allocated":667648,"data_stored":281555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574882,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31832,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340008,"statfs":{"total":96636764160,"available":96604168192,"internally_reserved":0,"allocated":667648,"data_stored":281555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574883,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27680,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":667648,"data_stored":281555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31836,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340004,"statfs":{"total":96636764160,"available":96604164096,"internally_reserved":0,"allocated":667648,"data_stored":281555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574882,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32408,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339432,"statfs":{"total":96636764160,"available":96603578368,"internally_reserved":0,"allocated":1261568,"data_stored":871923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T04:36:17.039 DEBUG:teuthology.orchestra.run.smithi145:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-05-09T04:36:17.232 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.232+0000 7fa05c800640 1 Processor -- start 2024-05-09T04:36:17.232 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.232+0000 7fa05c800640 1 -- start start 2024-05-09T04:36:17.234 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.234+0000 7fa05c800640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa054144c00 0x7fa0540ce640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:36:17.234 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.234+0000 7fa05c800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa054145440 0x7fa0541458a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:36:17.234 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.234+0000 7fa05c800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fa05406f5d0 0x7fa05406fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:36:17.234 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.234+0000 7fa05c800640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7fa05406ff40 con 0x7fa05406f5d0 2024-05-09T04:36:17.234 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.234+0000 7fa05c800640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7fa05415ffa0 con 0x7fa054144c00 2024-05-09T04:36:17.235 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.234+0000 7fa05c800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa054160280 con 0x7fa054145440 2024-05-09T04:36:17.235 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.234+0000 7fa05a800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fa05406f5d0 0x7fa05406fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:36:17.235 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.234+0000 7fa05a800640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fa05406f5d0 0x7fa05406fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:47548/0 (socket says 172.21.15.145:47548) 2024-05-09T04:36:17.235 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.234+0000 7fa05a800640 1 -- 172.21.15.145:0/4163656851 learned_addr learned my addr 172.21.15.145:0/4163656851 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:36:17.237 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.235+0000 7fa059e00640 1 --2- 172.21.15.145:0/4163656851 >> v2:172.21.15.192:3300/0 conn(0x7fa054144c00 0x7fa0540ce640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:36:17.237 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.235+0000 7fa059400640 1 --2- 172.21.15.145:0/4163656851 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa054145440 0x7fa0541458a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:36:17.237 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.235+0000 7fa05a800640 1 -- 172.21.15.145:0/4163656851 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa054145440 msgr2=0x7fa0541458a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:36:17.237 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.235+0000 7fa05a800640 1 --2- 172.21.15.145:0/4163656851 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa054145440 0x7fa0541458a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:36:17.238 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.235+0000 7fa05a800640 1 -- 172.21.15.145:0/4163656851 >> v2:172.21.15.192:3300/0 conn(0x7fa054144c00 msgr2=0x7fa0540ce640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:36:17.238 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.235+0000 7fa05a800640 1 --2- 172.21.15.145:0/4163656851 >> v2:172.21.15.192:3300/0 conn(0x7fa054144c00 0x7fa0540ce640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:36:17.238 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.235+0000 7fa05a800640 1 -- 172.21.15.145:0/4163656851 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa054160560 con 0x7fa05406f5d0 2024-05-09T04:36:17.238 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.235+0000 7fa059400640 1 --2- 172.21.15.145:0/4163656851 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa054145440 0x7fa0541458a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:36:17.238 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.236+0000 7fa05a800640 1 --2- 172.21.15.145:0/4163656851 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa05406f5d0 0x7fa05406fa00 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fa0540701e0 tx=0x7fa05002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64ea09e66eb97fe3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:36:17.238 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.236+0000 7fa058a00640 1 -- 172.21.15.145:0/4163656851 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa050037070 con 0x7fa05406f5d0 2024-05-09T04:36:17.238 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.236+0000 7fa058a00640 1 -- 172.21.15.145:0/4163656851 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa050032440 con 0x7fa05406f5d0 2024-05-09T04:36:17.239 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.236+0000 7fa058a00640 1 -- 172.21.15.145:0/4163656851 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa050032810 con 0x7fa05406f5d0 2024-05-09T04:36:17.239 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.237+0000 7fa05c800640 1 -- 172.21.15.145:0/4163656851 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa05406f5d0 msgr2=0x7fa05406fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:36:17.239 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.237+0000 7fa05c800640 1 --2- 172.21.15.145:0/4163656851 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa05406f5d0 0x7fa05406fa00 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fa0540701e0 tx=0x7fa05002a1e0 comp rx=0 tx=0).stop 2024-05-09T04:36:17.239 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.237+0000 7fa05c800640 1 -- 172.21.15.145:0/4163656851 shutdown_connections 2024-05-09T04:36:17.239 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.237+0000 7fa05c800640 1 --2- 172.21.15.145:0/4163656851 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa05406f5d0 0x7fa05406fa00 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:36:17.239 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.237+0000 7fa05c800640 1 --2- 172.21.15.145:0/4163656851 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa054145440 0x7fa0541458a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:36:17.239 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.237+0000 7fa05c800640 1 --2- 172.21.15.145:0/4163656851 >> v2:172.21.15.192:3300/0 conn(0x7fa054144c00 0x7fa0540ce640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:36:17.240 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.237+0000 7fa05c800640 1 -- 172.21.15.145:0/4163656851 >> 172.21.15.145:0/4163656851 conn(0x7fa05406cab0 msgr2=0x7fa0540d0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:36:17.240 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.237+0000 7fa05c800640 1 -- 172.21.15.145:0/4163656851 shutdown_connections 2024-05-09T04:36:17.240 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.237+0000 7fa05c800640 1 -- 172.21.15.145:0/4163656851 wait complete. 2024-05-09T04:36:17.242 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.239+0000 7fa05c800640 1 Processor -- start 2024-05-09T04:36:17.242 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.239+0000 7fa05c800640 1 -- start start 2024-05-09T04:36:17.243 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.240+0000 7fa05c800640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa05406f5d0 0x7fa0540d0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:36:17.243 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.240+0000 7fa05c800640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa054144c00 0x7fa054150bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:36:17.243 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.240+0000 7fa05c800640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa054145440 0x7fa0541f5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:36:17.243 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.240+0000 7fa05c800640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7fa05415fc30 con 0x7fa054144c00 2024-05-09T04:36:17.243 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.240+0000 7fa05c800640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7fa054160b20 con 0x7fa05406f5d0 2024-05-09T04:36:17.243 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.240+0000 7fa05c800640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7fa054144fe0 con 0x7fa054145440 2024-05-09T04:36:17.244 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.240+0000 7fa059400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa054144c00 0x7fa054150bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:36:17.244 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.240+0000 7fa059400640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa054144c00 0x7fa054150bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:47556/0 (socket says 172.21.15.145:47556) 2024-05-09T04:36:17.244 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.240+0000 7fa059400640 1 -- 172.21.15.145:0/2453707288 learned_addr learned my addr 172.21.15.145:0/2453707288 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:36:17.244 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.240+0000 7fa059400640 1 -- 172.21.15.145:0/2453707288 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa054145440 msgr2=0x7fa0541f5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:36:17.244 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.240+0000 7fa059400640 1 --2- 172.21.15.145:0/2453707288 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa054145440 0x7fa0541f5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:36:17.244 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.240+0000 7fa059400640 1 -- 172.21.15.145:0/2453707288 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa05406f5d0 msgr2=0x7fa0540d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:36:17.244 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.240+0000 7fa059400640 1 --2- 172.21.15.145:0/2453707288 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa05406f5d0 0x7fa0540d0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:36:17.244 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.240+0000 7fa059400640 1 -- 172.21.15.145:0/2453707288 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa054151390 con 0x7fa054144c00 2024-05-09T04:36:17.245 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.241+0000 7fa059400640 1 --2- 172.21.15.145:0/2453707288 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa054144c00 0x7fa054150bf0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fa04400c7f0 tx=0x7fa04400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:36:17.245 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.241+0000 7fa04a000640 1 -- 172.21.15.145:0/2453707288 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa044015070 con 0x7fa054144c00 2024-05-09T04:36:17.245 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.241+0000 7fa04a000640 1 -- 172.21.15.145:0/2453707288 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa044007a30 con 0x7fa054144c00 2024-05-09T04:36:17.245 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.241+0000 7fa04a000640 1 -- 172.21.15.145:0/2453707288 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa044007e00 con 0x7fa054144c00 2024-05-09T04:36:17.245 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.241+0000 7fa05c800640 1 -- 172.21.15.145:0/2453707288 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa054151670 con 0x7fa054144c00 2024-05-09T04:36:17.245 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.241+0000 7fa05c800640 1 -- 172.21.15.145:0/2453707288 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0540ca460 con 0x7fa054144c00 2024-05-09T04:36:17.246 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.242+0000 7fa05c800640 1 -- 172.21.15.145:0/2453707288 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0540ca660 con 0x7fa054144c00 2024-05-09T04:36:17.247 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.244+0000 7fa04a000640 1 -- 172.21.15.145:0/2453707288 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7fa044018070 con 0x7fa054144c00 2024-05-09T04:36:17.247 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.245+0000 7fa04a000640 1 --2- 172.21.15.145:0/2453707288 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fa0380762d0 0x7fa038078790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:36:17.247 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.245+0000 7fa04a000640 1 -- 172.21.15.145:0/2453707288 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7fa0440972c0 con 0x7fa054144c00 2024-05-09T04:36:17.247 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.245+0000 7fa059e00640 1 --2- 172.21.15.145:0/2453707288 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fa0380762d0 0x7fa038078790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:36:17.247 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.246+0000 7fa059e00640 1 --2- 172.21.15.145:0/2453707288 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fa0380762d0 0x7fa038078790 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa04c009540 tx=0x7fa04c009450 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:36:17.249 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.249+0000 7fa04a000640 1 -- 172.21.15.145:0/2453707288 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa0540ca660 con 0x7fa054144c00 2024-05-09T04:36:17.455 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.455+0000 7fa05c800640 1 -- 172.21.15.145:0/2453707288 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7fa05406a310 con 0x7fa054144c00 2024-05-09T04:36:17.462 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.462+0000 7fa04a000640 1 -- 172.21.15.145:0/2453707288 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v17)=0 v17) ==== 123+0+0 (secure 0 0 0) 0x7fa044060f10 con 0x7fa054144c00 2024-05-09T04:36:17.467 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.466+0000 7fa04a000640 1 -- 172.21.15.145:0/2453707288 <== mon.0 v2:172.21.15.145:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fa04400ce40 con 0x7fa054144c00 2024-05-09T04:36:17.469 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.468+0000 7fa05c800640 1 -- 172.21.15.145:0/2453707288 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fa0380762d0 msgr2=0x7fa038078790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:36:17.469 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.468+0000 7fa05c800640 1 --2- 172.21.15.145:0/2453707288 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fa0380762d0 0x7fa038078790 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa04c009540 tx=0x7fa04c009450 comp rx=0 tx=0).stop 2024-05-09T04:36:17.469 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.468+0000 7fa05c800640 1 -- 172.21.15.145:0/2453707288 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa054144c00 msgr2=0x7fa054150bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:36:17.469 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.468+0000 7fa05c800640 1 --2- 172.21.15.145:0/2453707288 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa054144c00 0x7fa054150bf0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fa04400c7f0 tx=0x7fa04400cbf0 comp rx=0 tx=0).stop 2024-05-09T04:36:17.469 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.468+0000 7fa05c800640 1 -- 172.21.15.145:0/2453707288 shutdown_connections 2024-05-09T04:36:17.469 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.468+0000 7fa05c800640 1 --2- 172.21.15.145:0/2453707288 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7fa0380762d0 0x7fa038078790 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:36:17.469 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.468+0000 7fa05c800640 1 --2- 172.21.15.145:0/2453707288 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa054145440 0x7fa0541f5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:36:17.470 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.468+0000 7fa05c800640 1 --2- 172.21.15.145:0/2453707288 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa054144c00 0x7fa054150bf0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:36:17.470 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.468+0000 7fa05c800640 1 --2- 172.21.15.145:0/2453707288 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa05406f5d0 0x7fa0540d0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:36:17.470 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.468+0000 7fa05c800640 1 -- 172.21.15.145:0/2453707288 >> 172.21.15.145:0/2453707288 conn(0x7fa05406cab0 msgr2=0x7fa05415c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:36:17.470 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.469+0000 7fa05c800640 1 -- 172.21.15.145:0/2453707288 shutdown_connections 2024-05-09T04:36:17.470 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.469+0000 7fa05c800640 1 -- 172.21.15.145:0/2453707288 wait complete. 2024-05-09T04:36:17.488 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-09T04:36:17.488 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T04:36:17.688 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.688+0000 7f2717c00640 1 Processor -- start 2024-05-09T04:36:17.689 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.689+0000 7f2717c00640 1 -- start start 2024-05-09T04:36:17.690 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.690+0000 7f2717c00640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f271014d210 0x7f271006e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:36:17.690 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.690+0000 7f2717c00640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f27100b5cb0 0x7f271006eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:36:17.690 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.690+0000 7f2717c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f27100b3f70 0x7f27100b4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:36:17.690 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.690+0000 7f2717c00640 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 -- 0x7f27100b5820 con 0x7f271014d210 2024-05-09T04:36:17.690 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.690+0000 7f2717c00640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 -- 0x7f27100b4940 con 0x7f27100b5cb0 2024-05-09T04:36:17.690 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.690+0000 7f2717c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f27100b4c10 con 0x7f27100b3f70 2024-05-09T04:36:17.691 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.691+0000 7f2715200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f271014d210 0x7f271006e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:36:17.692 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.691+0000 7f2715200640 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f271014d210 0x7f271006e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:47568/0 (socket says 172.21.15.145:47568) 2024-05-09T04:36:17.692 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.691+0000 7f2715200640 1 -- 172.21.15.145:0/3430891750 learned_addr learned my addr 172.21.15.145:0/3430891750 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:36:17.692 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.691+0000 7f2715c00640 1 --2- 172.21.15.145:0/3430891750 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f27100b3f70 0x7f27100b4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:36:17.692 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.691+0000 7f2707e00640 1 --2- 172.21.15.145:0/3430891750 >> v2:172.21.15.192:3300/0 conn(0x7f27100b5cb0 0x7f271006eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:36:17.692 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.691+0000 7f2715200640 1 -- 172.21.15.145:0/3430891750 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f27100b3f70 msgr2=0x7f27100b4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:36:17.692 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.691+0000 7f2715200640 1 --2- 172.21.15.145:0/3430891750 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f27100b3f70 0x7f27100b4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:36:17.692 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.691+0000 7f2715200640 1 -- 172.21.15.145:0/3430891750 >> v2:172.21.15.192:3300/0 conn(0x7f27100b5cb0 msgr2=0x7f271006eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:36:17.692 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.691+0000 7f2715200640 1 --2- 172.21.15.145:0/3430891750 >> v2:172.21.15.192:3300/0 conn(0x7f27100b5cb0 0x7f271006eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:36:17.693 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.691+0000 7f2715200640 1 -- 172.21.15.145:0/3430891750 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27100b4ef0 con 0x7f271014d210 2024-05-09T04:36:17.694 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.693+0000 7f2715200640 1 --2- 172.21.15.145:0/3430891750 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f271014d210 0x7f271006e860 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f27101672a0 tx=0x7f270002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90525dd2ddd47793 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:36:17.694 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.693+0000 7f270fe00640 1 -- 172.21.15.145:0/3430891750 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2700037070 con 0x7f271014d210 2024-05-09T04:36:17.694 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.693+0000 7f270fe00640 1 -- 172.21.15.145:0/3430891750 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f2700032440 con 0x7f271014d210 2024-05-09T04:36:17.694 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.693+0000 7f270fe00640 1 -- 172.21.15.145:0/3430891750 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2700032830 con 0x7f271014d210 2024-05-09T04:36:17.694 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.694+0000 7f2717c00640 1 -- 172.21.15.145:0/3430891750 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f271014d210 msgr2=0x7f271006e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:36:17.694 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.694+0000 7f2717c00640 1 --2- 172.21.15.145:0/3430891750 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f271014d210 0x7f271006e860 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f27101672a0 tx=0x7f270002a1e0 comp rx=0 tx=0).stop 2024-05-09T04:36:17.695 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.694+0000 7f2717c00640 1 -- 172.21.15.145:0/3430891750 shutdown_connections 2024-05-09T04:36:17.695 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.694+0000 7f2717c00640 1 --2- 172.21.15.145:0/3430891750 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f27100b3f70 0x7f27100b4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:36:17.695 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.694+0000 7f2717c00640 1 --2- 172.21.15.145:0/3430891750 >> v2:172.21.15.192:3300/0 conn(0x7f27100b5cb0 0x7f271006eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:36:17.695 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.694+0000 7f2717c00640 1 --2- 172.21.15.145:0/3430891750 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f271014d210 0x7f271006e860 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:36:17.695 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.694+0000 7f2717c00640 1 -- 172.21.15.145:0/3430891750 >> 172.21.15.145:0/3430891750 conn(0x7f271006c870 msgr2=0x7f271006cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:36:17.695 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.694+0000 7f2717c00640 1 -- 172.21.15.145:0/3430891750 shutdown_connections 2024-05-09T04:36:17.695 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.694+0000 7f2717c00640 1 -- 172.21.15.145:0/3430891750 wait complete. 2024-05-09T04:36:17.696 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.696+0000 7f2717c00640 1 Processor -- start 2024-05-09T04:36:17.696 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.696+0000 7f2717c00640 1 -- start start 2024-05-09T04:36:17.698 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.697+0000 7f2717c00640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f27100b3f70 0x7f27101632c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:36:17.698 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.697+0000 7f2717c00640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f27100b5cb0 0x7f2710164b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:36:17.698 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.697+0000 7f2717c00640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f271014d210 0x7f27101650a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:36:17.698 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.697+0000 7f2717c00640 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 -- 0x7f2710072220 con 0x7f27100b3f70 2024-05-09T04:36:17.698 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.697+0000 7f2717c00640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 -- 0x7f27100b54b0 con 0x7f271014d210 2024-05-09T04:36:17.698 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.697+0000 7f2717c00640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 -- 0x7f27100723a0 con 0x7f27100b5cb0 2024-05-09T04:36:17.699 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.697+0000 7f2715200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f27100b3f70 0x7f27101632c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:36:17.699 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.697+0000 7f2715200640 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f27100b3f70 0x7f27101632c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:47584/0 (socket says 172.21.15.145:47584) 2024-05-09T04:36:17.699 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.697+0000 7f2715200640 1 -- 172.21.15.145:0/4049712070 learned_addr learned my addr 172.21.15.145:0/4049712070 (peer_addr_for_me v2:172.21.15.145:0/0) 2024-05-09T04:36:17.699 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.698+0000 7f2707e00640 1 --2- 172.21.15.145:0/4049712070 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f27100b5cb0 0x7f2710164b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:36:17.699 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.698+0000 7f2715c00640 1 --2- 172.21.15.145:0/4049712070 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f271014d210 0x7f27101650a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:36:17.699 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.698+0000 7f2715200640 1 -- 172.21.15.145:0/4049712070 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f27100b5cb0 msgr2=0x7f2710164b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:36:17.699 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.698+0000 7f2715200640 1 --2- 172.21.15.145:0/4049712070 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f27100b5cb0 0x7f2710164b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:36:17.699 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.698+0000 7f2715200640 1 -- 172.21.15.145:0/4049712070 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f271014d210 msgr2=0x7f27101650a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:36:17.700 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.698+0000 7f2715200640 1 --2- 172.21.15.145:0/4049712070 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f271014d210 0x7f27101650a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:36:17.700 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.698+0000 7f2715200640 1 -- 172.21.15.145:0/4049712070 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27101fd7a0 con 0x7f27100b3f70 2024-05-09T04:36:17.700 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.698+0000 7f2715c00640 1 --2- 172.21.15.145:0/4049712070 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f271014d210 0x7f27101650a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:36:17.700 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.698+0000 7f2715200640 1 --2- 172.21.15.145:0/4049712070 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f27100b3f70 0x7f27101632c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f2700000c00 tx=0x7f27000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:36:17.701 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.699+0000 7f270d600640 1 -- 172.21.15.145:0/4049712070 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2700037040 con 0x7f27100b3f70 2024-05-09T04:36:17.701 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.699+0000 7f270d600640 1 -- 172.21.15.145:0/4049712070 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f2700042070 con 0x7f27100b3f70 2024-05-09T04:36:17.701 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.699+0000 7f270d600640 1 -- 172.21.15.145:0/4049712070 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2700030bd0 con 0x7f27100b3f70 2024-05-09T04:36:17.701 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.699+0000 7f2717c00640 1 -- 172.21.15.145:0/4049712070 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27100c77f0 con 0x7f27100b3f70 2024-05-09T04:36:17.701 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.699+0000 7f2717c00640 1 -- 172.21.15.145:0/4049712070 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27100c7db0 con 0x7f27100b3f70 2024-05-09T04:36:17.701 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.700+0000 7f2717c00640 1 -- 172.21.15.145:0/4049712070 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f271006a310 con 0x7f27100b3f70 2024-05-09T04:36:17.702 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.701+0000 7f270d600640 1 -- 172.21.15.145:0/4049712070 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 5) ==== 98201+0+0 (secure 0 0 0) 0x7f2700047460 con 0x7f27100b3f70 2024-05-09T04:36:17.702 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.702+0000 7f270d600640 1 --2- 172.21.15.145:0/4049712070 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f26ec076320 0x7f26ec0787e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:36:17.703 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.702+0000 7f270d600640 1 -- 172.21.15.145:0/4049712070 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7f27000b7e60 con 0x7f27100b3f70 2024-05-09T04:36:17.703 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.702+0000 7f2707e00640 1 --2- 172.21.15.145:0/4049712070 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f26ec076320 0x7f26ec0787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:36:17.703 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.703+0000 7f2707e00640 1 --2- 172.21.15.145:0/4049712070 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f26ec076320 0x7f26ec0787e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2710165ef0 tx=0x7f26f8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:36:17.706 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.706+0000 7f270d600640 1 -- 172.21.15.145:0/4049712070 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2700032e20 con 0x7f27100b3f70 2024-05-09T04:36:17.958 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.958+0000 7f2717c00640 1 -- 172.21.15.145:0/4049712070 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f271014c830 con 0x7f27100b3f70 2024-05-09T04:36:17.959 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.959+0000 7f270d600640 1 -- 172.21.15.145:0/4049712070 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+3875 (secure 0 0 0) 0x7f2700081aa0 con 0x7f27100b3f70 2024-05-09T04:36:17.959 INFO:teuthology.orchestra.run.smithi145.stdout: 2024-05-09T04:36:17.960 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":30,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4661,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6834/1033874179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":1033874179},{"type":"v1","addr":"172.21.15.192:6834","nonce":1033874179}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":4664,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/1787962921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6833","nonce":1787962921},{"type":"v1","addr":"172.21.15.192:6835","nonce":1787962921}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5313,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6836/65063948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":65063948},{"type":"v1","addr":"172.21.15.145:6836","nonce":65063948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5586,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/3379187758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6835","nonce":3379187758},{"type":"v1","addr":"172.21.15.145:6837","nonce":3379187758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[]} 2024-05-09T04:36:17.960 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 30 2024-05-09T04:36:17.962 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.962+0000 7f2717c00640 1 -- 172.21.15.145:0/4049712070 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f26ec076320 msgr2=0x7f26ec0787e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:36:17.963 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.962+0000 7f2717c00640 1 --2- 172.21.15.145:0/4049712070 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f26ec076320 0x7f26ec0787e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2710165ef0 tx=0x7f26f8008040 comp rx=0 tx=0).stop 2024-05-09T04:36:17.963 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.962+0000 7f2717c00640 1 -- 172.21.15.145:0/4049712070 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f27100b3f70 msgr2=0x7f27101632c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:36:17.963 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.962+0000 7f2717c00640 1 --2- 172.21.15.145:0/4049712070 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f27100b3f70 0x7f27101632c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f2700000c00 tx=0x7f27000046c0 comp rx=0 tx=0).stop 2024-05-09T04:36:17.963 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.963+0000 7f2717c00640 1 -- 172.21.15.145:0/4049712070 shutdown_connections 2024-05-09T04:36:17.963 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.963+0000 7f2717c00640 1 --2- 172.21.15.145:0/4049712070 >> [v2:172.21.15.145:6832/2725855396,v1:172.21.15.145:6833/2725855396] conn(0x7f26ec076320 0x7f26ec0787e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:36:17.963 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.963+0000 7f2717c00640 1 --2- 172.21.15.145:0/4049712070 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f271014d210 0x7f27101650a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:36:17.963 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.963+0000 7f2717c00640 1 --2- 172.21.15.145:0/4049712070 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f27100b5cb0 0x7f2710164b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:36:17.963 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.963+0000 7f2717c00640 1 --2- 172.21.15.145:0/4049712070 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f27100b3f70 0x7f27101632c0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:36:17.964 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.963+0000 7f2717c00640 1 -- 172.21.15.145:0/4049712070 >> 172.21.15.145:0/4049712070 conn(0x7f271006c870 msgr2=0x7f27100d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:36:17.964 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.963+0000 7f2717c00640 1 -- 172.21.15.145:0/4049712070 shutdown_connections 2024-05-09T04:36:17.964 INFO:teuthology.orchestra.run.smithi145.stderr:2024-05-09T04:36:17.963+0000 7f2717c00640 1 -- 172.21.15.145:0/4049712070 wait complete. 2024-05-09T04:36:17.981 DEBUG:tasks.cephfs.filesystem:already dead... 2024-05-09T04:36:17.981 INFO:teuthology.misc:Shutting down mds daemons... 2024-05-09T04:36:17.982 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-09T04:36:17.982 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T04:36:18.003 INFO:tasks.ceph.mds.a:Stopped 2024-05-09T04:36:18.003 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-09T04:36:18.003 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T04:36:18.065 INFO:tasks.ceph.mds.c:Stopped 2024-05-09T04:36:18.066 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-09T04:36:18.066 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T04:36:18.088 INFO:tasks.ceph.mds.b:Stopped 2024-05-09T04:36:18.088 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-09T04:36:18.088 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T04:36:18.157 INFO:tasks.ceph.mds.d:Stopped 2024-05-09T04:36:18.158 INFO:teuthology.misc:Shutting down osd daemons... 2024-05-09T04:36:18.158 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-05-09T04:36:18.158 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T04:36:18.213 INFO:tasks.ceph.osd.0:Stopped 2024-05-09T04:36:18.213 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-05-09T04:36:18.214 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T04:36:18.282 INFO:tasks.ceph.osd.1:Stopped 2024-05-09T04:36:18.283 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-05-09T04:36:18.283 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T04:36:18.359 INFO:tasks.ceph.osd.2:Stopped 2024-05-09T04:36:18.360 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-05-09T04:36:18.360 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T04:36:18.424 INFO:tasks.ceph.osd.3:Stopped 2024-05-09T04:36:18.424 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-05-09T04:36:18.424 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T04:36:18.480 INFO:tasks.ceph.osd.4:Stopped 2024-05-09T04:36:18.481 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-05-09T04:36:18.481 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T04:36:18.565 INFO:tasks.ceph.osd.5:Stopped 2024-05-09T04:36:18.566 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-05-09T04:36:18.566 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T04:36:18.624 INFO:tasks.ceph.osd.6:Stopped 2024-05-09T04:36:18.624 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-05-09T04:36:18.624 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T04:36:18.689 INFO:tasks.ceph.osd.7:Stopped 2024-05-09T04:36:18.690 INFO:teuthology.misc:Shutting down mgr daemons... 2024-05-09T04:36:18.690 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-05-09T04:36:18.690 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T04:36:18.776 INFO:tasks.ceph.mgr.y:Stopped 2024-05-09T04:36:18.776 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-05-09T04:36:18.776 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T04:36:18.854 INFO:tasks.ceph.mgr.x:Stopped 2024-05-09T04:36:18.854 INFO:teuthology.misc:Shutting down mon daemons... 2024-05-09T04:36:18.855 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-05-09T04:36:18.855 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T04:36:18.904 INFO:tasks.ceph.mon.a:Stopped 2024-05-09T04:36:18.904 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-05-09T04:36:18.905 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T04:36:18.949 INFO:tasks.ceph.mon.b:Stopped 2024-05-09T04:36:18.949 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-05-09T04:36:18.949 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T04:36:18.991 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-05-09T04:36:18.991 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2024-05-09T04:36:18.991 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-05-09T04:36:18.991 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-05-09T04:36:18.991 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-05-09T04:36:18.996 INFO:tasks.ceph.mon.c:Stopped 2024-05-09T04:36:18.996 INFO:tasks.ceph:Checking cluster log for badness... 2024-05-09T04:36:18.996 DEBUG:teuthology.orchestra.run.smithi145:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-05-09T04:36:19.101 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi145.front.sepia.ceph.com 2024-05-09T04:36:19.101 DEBUG:teuthology.orchestra.run.smithi145:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-05-09T04:36:19.356 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi145.front.sepia.ceph.com 2024-05-09T04:36:19.357 DEBUG:teuthology.orchestra.run.smithi145:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-05-09T04:36:19.539 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi145.front.sepia.ceph.com 2024-05-09T04:36:19.539 DEBUG:teuthology.orchestra.run.smithi145:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-05-09T04:36:19.671 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi145.front.sepia.ceph.com 2024-05-09T04:36:19.671 DEBUG:teuthology.orchestra.run.smithi145:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-05-09T04:36:19.833 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi192.front.sepia.ceph.com 2024-05-09T04:36:19.833 DEBUG:teuthology.orchestra.run.smithi192:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-05-09T04:36:20.085 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi192.front.sepia.ceph.com 2024-05-09T04:36:20.085 DEBUG:teuthology.orchestra.run.smithi192:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-05-09T04:36:20.266 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi192.front.sepia.ceph.com 2024-05-09T04:36:20.266 DEBUG:teuthology.orchestra.run.smithi192:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-05-09T04:36:20.403 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi192.front.sepia.ceph.com 2024-05-09T04:36:20.404 DEBUG:teuthology.orchestra.run.smithi192:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-05-09T04:36:20.550 INFO:tasks.ceph:Archiving mon data... 2024-05-09T04:36:20.551 DEBUG:teuthology.misc:Transferring archived files from smithi145:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/vshankar-2024-05-09_04:13:07-fs-wip-63259-3-testing-default-smithi/7699554/data/mon.a.tgz 2024-05-09T04:36:20.552 DEBUG:teuthology.orchestra.run.smithi145:> mktemp 2024-05-09T04:36:20.607 INFO:teuthology.orchestra.run.smithi145.stdout:/tmp/tmp.pOR7GX3LML 2024-05-09T04:36:20.608 DEBUG:teuthology.orchestra.run.smithi145:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.pOR7GX3LML 2024-05-09T04:36:20.780 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod 0666 /tmp/tmp.pOR7GX3LML 2024-05-09T04:36:20.928 DEBUG:teuthology.orchestra.remote:smithi145:/tmp/tmp.pOR7GX3LML is 337KB 2024-05-09T04:36:21.067 DEBUG:teuthology.orchestra.run.smithi145:> rm -fr /tmp/tmp.pOR7GX3LML 2024-05-09T04:36:21.124 DEBUG:teuthology.misc:Transferring archived files from smithi192:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/vshankar-2024-05-09_04:13:07-fs-wip-63259-3-testing-default-smithi/7699554/data/mon.b.tgz 2024-05-09T04:36:21.124 DEBUG:teuthology.orchestra.run.smithi192:> mktemp 2024-05-09T04:36:21.177 INFO:teuthology.orchestra.run.smithi192.stdout:/tmp/tmp.GpVmZZsBTo 2024-05-09T04:36:21.177 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.GpVmZZsBTo 2024-05-09T04:36:21.381 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 0666 /tmp/tmp.GpVmZZsBTo 2024-05-09T04:36:21.491 DEBUG:teuthology.orchestra.remote:smithi192:/tmp/tmp.GpVmZZsBTo is 446KB 2024-05-09T04:36:21.594 DEBUG:teuthology.orchestra.run.smithi192:> rm -fr /tmp/tmp.GpVmZZsBTo 2024-05-09T04:36:21.646 DEBUG:teuthology.misc:Transferring archived files from smithi192:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/vshankar-2024-05-09_04:13:07-fs-wip-63259-3-testing-default-smithi/7699554/data/mon.c.tgz 2024-05-09T04:36:21.646 DEBUG:teuthology.orchestra.run.smithi192:> mktemp 2024-05-09T04:36:21.737 INFO:teuthology.orchestra.run.smithi192.stdout:/tmp/tmp.wa0SsocAH4 2024-05-09T04:36:21.737 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.wa0SsocAH4 2024-05-09T04:36:21.903 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 0666 /tmp/tmp.wa0SsocAH4 2024-05-09T04:36:22.030 DEBUG:teuthology.orchestra.remote:smithi192:/tmp/tmp.wa0SsocAH4 is 446KB 2024-05-09T04:36:22.137 DEBUG:teuthology.orchestra.run.smithi192:> rm -fr /tmp/tmp.wa0SsocAH4 2024-05-09T04:36:22.193 INFO:tasks.ceph:Cleaning ceph cluster... 2024-05-09T04:36:22.194 DEBUG:teuthology.orchestra.run.smithi145:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-09T04:36:22.201 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-09T04:36:22.340 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-09T04:36:22.397 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-09T04:36:22.397 INFO:tasks.ceph:Archiving crash dumps... 2024-05-09T04:36:22.398 DEBUG:teuthology.misc:Transferring archived files from smithi145:/var/lib/ceph/crash to /home/teuthworker/archive/vshankar-2024-05-09_04:13:07-fs-wip-63259-3-testing-default-smithi/7699554/remote/smithi145/crash 2024-05-09T04:36:22.399 DEBUG:teuthology.orchestra.run.smithi145:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-09T04:36:22.482 DEBUG:teuthology.misc:Transferring archived files from smithi192:/var/lib/ceph/crash to /home/teuthworker/archive/vshankar-2024-05-09_04:13:07-fs-wip-63259-3-testing-default-smithi/7699554/remote/smithi192/crash 2024-05-09T04:36:22.483 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-09T04:36:22.569 INFO:tasks.ceph:Shutting down logrotate 2024-05-09T04:36:22.569 DEBUG:teuthology.orchestra.run.smithi145:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T04:36:22.578 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T04:36:22.698 DEBUG:teuthology.orchestra.run.smithi145:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-09T04:36:22.786 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-09T04:36:22.871 INFO:tasks.ceph:Compressing logs... 2024-05-09T04:36:22.872 DEBUG:teuthology.orchestra.run.smithi145:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-09T04:36:22.879 DEBUG:teuthology.orchestra.run.smithi192:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-09T04:36:22.954 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41592.log 2024-05-09T04:36:22.955 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49565.log 2024-05-09T04:36:22.956 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.41592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52311.log 2024-05-09T04:36:22.957 INFO:teuthology.orchestra.run.smithi145.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41592.log.gz 2024-05-09T04:36:22.957 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.49565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41247.log 2024-05-09T04:36:22.957 INFO:teuthology.orchestra.run.smithi145.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49565.log.gz 2024-05-09T04:36:22.958 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.52311.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52311.log.gz 2024-05-09T04:36:22.959 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52411.log 2024-05-09T04:36:22.959 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-09T04:36:22.960 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35155.log 2024-05-09T04:36:22.960 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.41247.log: gzip 87.8% -5 -- replaced with /var/log/ceph/ceph-client.admin.41247.log.gz --verbose 2024-05-09T04:36:22.960 INFO:teuthology.orchestra.run.smithi145.stderr: -- /var/log/ceph/ceph-client.admin.41918.log 2024-05-09T04:36:22.961 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-09T04:36:22.961 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.52411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42977.log 2024-05-09T04:36:22.961 INFO:teuthology.orchestra.run.smithi145.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52411.log.gz 2024-05-09T04:36:22.963 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.41918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51513.log 2024-05-09T04:36:22.963 INFO:teuthology.orchestra.run.smithi145.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41918.log.gz 2024-05-09T04:36:22.963 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.0.42977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47306.log 2024-05-09T04:36:22.963 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.admin.35155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35366.log 2024-05-09T04:36:22.963 INFO:teuthology.orchestra.run.smithi192.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.35155.log.gz 2024-05-09T04:36:22.963 INFO:teuthology.orchestra.run.smithi192.stderr: 91.9% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-09T04:36:22.965 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-05-09T04:36:22.965 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.admin.35366.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35366.log.gz 2024-05-09T04:36:22.965 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-05-09T04:36:22.966 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.51513.log: 90.2% -- replaced with /var/log/ceph/ceph-client.0.42977.log.gz 2024-05-09T04:36:22.966 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51975.log 2024-05-09T04:36:22.966 INFO:teuthology.orchestra.run.smithi145.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51513.log.gz 2024-05-09T04:36:22.966 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.47306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42252.log 2024-05-09T04:36:22.966 INFO:teuthology.orchestra.run.smithi145.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47306.log.gz 2024-05-09T04:36:22.966 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.51975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48003.log 2024-05-09T04:36:22.966 INFO:teuthology.orchestra.run.smithi145.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51975.log.gz 2024-05-09T04:36:22.969 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.42252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52143.log 2024-05-09T04:36:22.969 INFO:teuthology.orchestra.run.smithi145.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42252.log.gz 2024-05-09T04:36:22.970 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.48003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44036.log 2024-05-09T04:36:22.970 INFO:teuthology.orchestra.run.smithi145.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48003.log.gz 2024-05-09T04:36:22.970 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.52143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52227.log 2024-05-09T04:36:22.970 INFO:teuthology.orchestra.run.smithi145.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52143.log.gz 2024-05-09T04:36:22.971 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-05-09T04:36:22.971 INFO:teuthology.orchestra.run.smithi192.stderr: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-09T04:36:22.971 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-05-09T04:36:22.971 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-osd.5.log: 92.1% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-05-09T04:36:22.971 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35406.log 2024-05-09T04:36:22.972 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-05-09T04:36:22.972 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.admin.35406.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-05-09T04:36:22.972 INFO:teuthology.orchestra.run.smithi192.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35406.log.gz 2024-05-09T04:36:22.972 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-05-09T04:36:22.974 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.44036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40824.log 2024-05-09T04:36:22.974 INFO:teuthology.orchestra.run.smithi145.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44036.log.gz 2024-05-09T04:36:22.974 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.52227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45161.log 2024-05-09T04:36:22.974 INFO:teuthology.orchestra.run.smithi145.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52227.log.gz 2024-05-09T04:36:22.975 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.40824.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40614.log 2024-05-09T04:36:22.975 INFO:teuthology.orchestra.run.smithi145.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40824.log.gz 2024-05-09T04:36:22.975 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.45161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45075.log 2024-05-09T04:36:22.975 INFO:teuthology.orchestra.run.smithi145.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45161.log.gz 2024-05-09T04:36:22.975 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.40614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41587.log 2024-05-09T04:36:22.975 INFO:teuthology.orchestra.run.smithi145.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40614.log.gz 2024-05-09T04:36:22.977 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35286.log 2024-05-09T04:36:22.978 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.40596.log 2024-05-09T04:36:22.978 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.admin.35286.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35286.log.gz 2024-05-09T04:36:22.978 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.1.40596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35446.log 2024-05-09T04:36:22.978 INFO:teuthology.orchestra.run.smithi192.stderr: 92.8% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-05-09T04:36:22.979 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.45075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49296.log 2024-05-09T04:36:22.979 INFO:teuthology.orchestra.run.smithi145.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45075.log.gz 2024-05-09T04:36:22.980 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.41587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52101.log 2024-05-09T04:36:22.980 INFO:teuthology.orchestra.run.smithi145.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41587.log.gz 2024-05-09T04:36:22.980 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.49296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46988.log 2024-05-09T04:36:22.980 INFO:teuthology.orchestra.run.smithi145.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49296.log.gz 2024-05-09T04:36:22.980 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.52101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41593.log 2024-05-09T04:36:22.980 INFO:teuthology.orchestra.run.smithi145.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52101.log.gz 2024-05-09T04:36:22.980 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.46988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44603.log 2024-05-09T04:36:22.980 INFO:teuthology.orchestra.run.smithi145.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.46988.log.gz 2024-05-09T04:36:22.983 INFO:teuthology.orchestra.run.smithi192.stderr:gzip 88.3% -- replaced with /var/log/ceph/ceph-client.1.40596.log.gz 2024-05-09T04:36:22.984 INFO:teuthology.orchestra.run.smithi192.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.35326.log 2024-05-09T04:36:22.984 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.admin.35446.log: 88.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.35446.log.gz -5 2024-05-09T04:36:22.984 INFO:teuthology.orchestra.run.smithi192.stderr: --verbose -- /var/log/ceph/ceph-client.admin.35246.log 2024-05-09T04:36:22.984 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.admin.35326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35197.log 2024-05-09T04:36:22.984 INFO:teuthology.orchestra.run.smithi192.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.35326.log.gz 2024-05-09T04:36:22.985 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.41593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45892.log 2024-05-09T04:36:22.986 INFO:teuthology.orchestra.run.smithi145.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41593.log.gz 2024-05-09T04:36:22.986 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.44603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47262.log 2024-05-09T04:36:22.986 INFO:teuthology.orchestra.run.smithi145.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44603.log.gz 2024-05-09T04:36:22.986 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.45892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45501.log 2024-05-09T04:36:22.986 INFO:teuthology.orchestra.run.smithi145.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45892.log.gz 2024-05-09T04:36:22.986 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.47262.log: gzip -5 89.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.47262.log.gz -- 2024-05-09T04:36:22.986 INFO:teuthology.orchestra.run.smithi145.stderr: /var/log/ceph/ceph-client.admin.41257.log 2024-05-09T04:36:22.986 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.45501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45371.log 2024-05-09T04:36:22.987 INFO:teuthology.orchestra.run.smithi145.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45501.log.gz 2024-05-09T04:36:22.987 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.41257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41255.log 2024-05-09T04:36:22.987 INFO:teuthology.orchestra.run.smithi145.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41257.log.gz 2024-05-09T04:36:22.988 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.45371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52453.log 2024-05-09T04:36:22.988 INFO:teuthology.orchestra.run.smithi145.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45371.log.gz 2024-05-09T04:36:22.989 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.41255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51639.log 2024-05-09T04:36:22.989 INFO:teuthology.orchestra.run.smithi145.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41255.log.gz 2024-05-09T04:36:22.989 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.admin.35246.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.35246.log.gz 2024-05-09T04:36:22.989 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.admin.35197.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-05-09T04:36:22.989 INFO:teuthology.orchestra.run.smithi192.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35197.log.gz 2024-05-09T04:36:22.989 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-05-09T04:36:22.990 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.41116.log 2024-05-09T04:36:22.991 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.52453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47907.log 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52453.log.gz 2024-05-09T04:36:22.991 INFO:teuthology.orchestra.run.smithi145.stderr: 2024-05-09T04:36:22.991 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.51639.log: 88.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.51639.log.gz 2024-05-09T04:36:22.991 INFO:teuthology.orchestra.run.smithi145.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.41915.log 2024-05-09T04:36:22.991 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.47907.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-09T04:36:22.992 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-client.3.41116.log: 88.5% -- replaced with /var/log/ceph/ceph-client.3.41116.log.gz 2024-05-09T04:36:22.993 INFO:teuthology.orchestra.run.smithi145.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47907.log.gz 2024-05-09T04:36:22.993 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.41915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47818.log 2024-05-09T04:36:22.993 INFO:teuthology.orchestra.run.smithi145.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41915.log.gz 2024-05-09T04:36:22.993 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44355.log 2024-05-09T04:36:22.994 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.47818.log: 85.3% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-09T04:36:22.994 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35407.log 2024-05-09T04:36:22.994 INFO:teuthology.orchestra.run.smithi145.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.47818.log.gz 2024-05-09T04:36:22.995 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.44355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44771.log 2024-05-09T04:36:22.996 INFO:teuthology.orchestra.run.smithi145.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44355.log.gz 2024-05-09T04:36:22.997 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.35407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42540.log 2024-05-09T04:36:22.997 INFO:teuthology.orchestra.run.smithi145.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35407.log.gz 2024-05-09T04:36:22.998 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.44771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41242.log 2024-05-09T04:36:22.998 INFO:teuthology.orchestra.run.smithi145.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44771.log.gz 2024-05-09T04:36:22.999 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.42540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40442.log 2024-05-09T04:36:22.999 INFO:teuthology.orchestra.run.smithi145.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42540.log.gz 2024-05-09T04:36:23.000 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.41242.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-05-09T04:36:23.000 INFO:teuthology.orchestra.run.smithi145.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41242.log.gz 2024-05-09T04:36:23.002 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.40442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44813.log 2024-05-09T04:36:23.002 INFO:teuthology.orchestra.run.smithi145.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40442.log.gz 2024-05-09T04:36:23.003 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50878.log 2024-05-09T04:36:23.004 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.44813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49384.log 2024-05-09T04:36:23.004 INFO:teuthology.orchestra.run.smithi145.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44813.log.gz 2024-05-09T04:36:23.005 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.50878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49519.log 2024-05-09T04:36:23.005 INFO:teuthology.orchestra.run.smithi145.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50878.log.gz 2024-05-09T04:36:23.006 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.49384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51004.log 2024-05-09T04:36:23.007 INFO:teuthology.orchestra.run.smithi145.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49384.log.gz 2024-05-09T04:36:23.008 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.49519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46668.log 2024-05-09T04:36:23.008 INFO:teuthology.orchestra.run.smithi145.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49519.log.gz 2024-05-09T04:36:23.009 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.51004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41880.log 2024-05-09T04:36:23.009 INFO:teuthology.orchestra.run.smithi145.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51004.log.gz 2024-05-09T04:36:23.010 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.46668.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46668.log.gz 2024-05-09T04:36:23.010 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47041.log 2024-05-09T04:36:23.011 INFO:teuthology.orchestra.run.smithi192.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-05-09T04:36:23.012 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.41880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51096.log 2024-05-09T04:36:23.013 INFO:teuthology.orchestra.run.smithi145.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41880.log.gz 2024-05-09T04:36:23.013 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.47041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51597.log 2024-05-09T04:36:23.013 INFO:teuthology.orchestra.run.smithi145.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.47041.log.gz 2024-05-09T04:36:23.014 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.51096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50920.log 2024-05-09T04:36:23.014 INFO:teuthology.orchestra.run.smithi145.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51096.log.gz 2024-05-09T04:36:23.015 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.51597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40992.log 2024-05-09T04:36:23.016 INFO:teuthology.orchestra.run.smithi145.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51597.log.gz 2024-05-09T04:36:23.016 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.50920.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-09T04:36:23.016 INFO:teuthology.orchestra.run.smithi145.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50920.log.gz 2024-05-09T04:36:23.017 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.40992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49050.log 2024-05-09T04:36:23.017 INFO:teuthology.orchestra.run.smithi145.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40992.log.gz 2024-05-09T04:36:23.018 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49775.log 2024-05-09T04:36:23.020 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.49050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52185.log 2024-05-09T04:36:23.020 INFO:teuthology.orchestra.run.smithi145.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49050.log.gz 2024-05-09T04:36:23.020 INFO:teuthology.orchestra.run.smithi145.stderr: 89.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-09T04:36:23.021 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.49775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50794.log 2024-05-09T04:36:23.021 INFO:teuthology.orchestra.run.smithi145.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49775.log.gz 2024-05-09T04:36:23.022 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.52185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.43475.log 2024-05-09T04:36:23.022 INFO:teuthology.orchestra.run.smithi145.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52185.log.gz 2024-05-09T04:36:23.023 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.50794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44729.log 2024-05-09T04:36:23.024 INFO:teuthology.orchestra.run.smithi145.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50794.log.gz 2024-05-09T04:36:23.025 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.2.43475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51317.log 2024-05-09T04:36:23.025 INFO:teuthology.orchestra.run.smithi145.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.2.43475.log.gz 2024-05-09T04:36:23.026 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.44729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46320.log 2024-05-09T04:36:23.028 INFO:teuthology.orchestra.run.smithi145.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44729.log.gz 2024-05-09T04:36:23.028 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.51317.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-05-09T04:36:23.028 INFO:teuthology.orchestra.run.smithi145.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51317.log.gz 2024-05-09T04:36:23.028 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.0.46320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46945.log 2024-05-09T04:36:23.029 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50446.log 2024-05-09T04:36:23.030 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.46945.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46945.log.gz 2024-05-09T04:36:23.031 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44645.log 2024-05-09T04:36:23.032 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.50446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45766.log 2024-05-09T04:36:23.032 INFO:teuthology.orchestra.run.smithi145.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50446.log.gz 2024-05-09T04:36:23.033 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.44645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45117.log 2024-05-09T04:36:23.033 INFO:teuthology.orchestra.run.smithi145.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44645.log.gz 2024-05-09T04:36:23.034 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.45766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41929.log 2024-05-09T04:36:23.035 INFO:teuthology.orchestra.run.smithi145.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45766.log.gz 2024-05-09T04:36:23.036 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.45117.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45117.log.gz 2024-05-09T04:36:23.036 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45594.log 2024-05-09T04:36:23.036 INFO:teuthology.orchestra.run.smithi145.stderr: 92.7% -- replaced with /var/log/ceph/ceph-client.0.46320.log.gz 2024-05-09T04:36:23.037 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.41929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41246.log 2024-05-09T04:36:23.038 INFO:teuthology.orchestra.run.smithi145.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41929.log.gz 2024-05-09T04:36:23.039 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.45594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40484.log 2024-05-09T04:36:23.039 INFO:teuthology.orchestra.run.smithi145.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45594.log.gz 2024-05-09T04:36:23.040 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.41246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41927.log 2024-05-09T04:36:23.040 INFO:teuthology.orchestra.run.smithi145.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41246.log.gz 2024-05-09T04:36:23.041 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.40484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44477.log 2024-05-09T04:36:23.041 INFO:teuthology.orchestra.run.smithi145.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40484.log.gz 2024-05-09T04:36:23.042 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.41927.log: gzip 89.4% -- replaced with /var/log/ceph/ceph-client.admin.41927.log.gz -5 2024-05-09T04:36:23.042 INFO:teuthology.orchestra.run.smithi145.stderr: --verbose -- /var/log/ceph/ceph-client.admin.40908.log 2024-05-09T04:36:23.043 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.44477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41930.log 2024-05-09T04:36:23.044 INFO:teuthology.orchestra.run.smithi145.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44477.log.gz 2024-05-09T04:36:23.045 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.40908.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.40908.log.gz 2024-05-09T04:36:23.045 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47449.log 2024-05-09T04:36:23.046 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.41930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47390.log 2024-05-09T04:36:23.046 INFO:teuthology.orchestra.run.smithi145.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41930.log.gz 2024-05-09T04:36:23.047 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.47449.log: 88.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.47449.log.gz 2024-05-09T04:36:23.048 INFO:teuthology.orchestra.run.smithi145.stderr: -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-05-09T04:36:23.048 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.47390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40950.log 2024-05-09T04:36:23.049 INFO:teuthology.orchestra.run.smithi145.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47390.log.gz 2024-05-09T04:36:23.049 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50342.log 2024-05-09T04:36:23.051 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.40950.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40950.log.gz 2024-05-09T04:36:23.051 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49435.log 2024-05-09T04:36:23.052 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47704.log 2024-05-09T04:36:23.053 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.50342.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50342.log.gz 2024-05-09T04:36:23.053 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.49435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49691.log 2024-05-09T04:36:23.054 INFO:teuthology.orchestra.run.smithi145.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49435.log.gz 2024-05-09T04:36:23.054 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.47704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47598.log 2024-05-09T04:36:23.055 INFO:teuthology.orchestra.run.smithi145.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47704.log.gz 2024-05-09T04:36:23.056 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.49691.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49691.log.gz 2024-05-09T04:36:23.056 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-05-09T04:36:23.057 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.47598.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-05-09T04:36:23.058 INFO:teuthology.orchestra.run.smithi145.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.47598.log.gz 2024-05-09T04:36:23.058 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49925.log 2024-05-09T04:36:23.060 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51555.log 2024-05-09T04:36:23.061 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.49925.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-05-09T04:36:23.061 INFO:teuthology.orchestra.run.smithi145.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49925.log.gz 2024-05-09T04:36:23.062 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.51555.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51555.log.gz 2024-05-09T04:36:23.063 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40570.log 2024-05-09T04:36:23.064 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44172.log 2024-05-09T04:36:23.066 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.40570.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40570.log.gz 2024-05-09T04:36:23.066 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41578.log 2024-05-09T04:36:23.067 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.44172.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44172.log.gz 2024-05-09T04:36:23.068 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48474.log 2024-05-09T04:36:23.069 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.41578.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41578.log.gz 2024-05-09T04:36:23.071 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.48474.log: /var/log/ceph/ceph-client.admin.45287.log 2024-05-09T04:36:23.071 INFO:teuthology.orchestra.run.smithi145.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48474.log.gz 2024-05-09T04:36:23.072 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43857.log 2024-05-09T04:36:23.073 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.45287.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45287.log.gz 2024-05-09T04:36:23.074 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.43857.log: /var/log/ceph/ceph-client.admin.44214.log 2024-05-09T04:36:23.075 INFO:teuthology.orchestra.run.smithi145.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43857.log.gz 2024-05-09T04:36:23.076 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35509.log 2024-05-09T04:36:23.077 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.44214.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44214.log.gz 2024-05-09T04:36:23.078 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5/var/log/ceph/ceph-client.admin.35509.log: --verbose -- /var/log/ceph/ceph-client.admin.50700.log 2024-05-09T04:36:23.079 INFO:teuthology.orchestra.run.smithi145.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35509.log.gz 2024-05-09T04:36:23.080 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51191.log 2024-05-09T04:36:23.081 INFO:teuthology.orchestra.run.smithi192.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-05-09T04:36:23.081 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.50700.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50700.log.gz 2024-05-09T04:36:23.082 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45552.log 2024-05-09T04:36:23.083 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.51191.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51191.log.gz 2024-05-09T04:36:23.084 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51681.log 2024-05-09T04:36:23.085 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.45552.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45552.log.gz 2024-05-09T04:36:23.086 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51149.log 2024-05-09T04:36:23.087 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.51681.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51681.log.gz 2024-05-09T04:36:23.088 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52017.log 2024-05-09T04:36:23.089 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.51149.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51149.log.gz 2024-05-09T04:36:23.090 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.52017.log: -- /var/log/ceph/ceph-client.admin.41258.log 2024-05-09T04:36:23.091 INFO:teuthology.orchestra.run.smithi145.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52017.log.gz 2024-05-09T04:36:23.092 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40264.log 2024-05-09T04:36:23.093 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.41258.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41258.log.gz 2024-05-09T04:36:23.094 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.40264.log: /var/log/ceph/ceph-client.admin.40656.log 2024-05-09T04:36:23.094 INFO:teuthology.orchestra.run.smithi145.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40264.log.gz 2024-05-09T04:36:23.096 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.40656.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40656.log.gz 2024-05-09T04:36:23.097 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44435.log 2024-05-09T04:36:23.098 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-05-09T04:36:23.100 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.44435.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44435.log.gz 2024-05-09T04:36:23.101 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50581.log 2024-05-09T04:36:23.104 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.50581.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.50581.log.gz 2024-05-09T04:36:23.105 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51933.log 2024-05-09T04:36:23.107 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.51933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43081.log 2024-05-09T04:36:23.109 INFO:teuthology.orchestra.run.smithi145.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51933.log.gz 2024-05-09T04:36:23.110 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50404.log 2024-05-09T04:36:23.114 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.43081.log: /var/log/ceph/ceph-client.admin.50404.log: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43621.log 2024-05-09T04:36:23.115 INFO:teuthology.orchestra.run.smithi145.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43081.log.gz 2024-05-09T04:36:23.116 INFO:teuthology.orchestra.run.smithi145.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50404.log.gz 2024-05-09T04:36:23.117 INFO:teuthology.orchestra.run.smithi145.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-05-09T04:36:23.117 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40526.log 2024-05-09T04:36:23.119 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.43621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49859.log 2024-05-09T04:36:23.120 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.40526.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40526.log.gz 2024-05-09T04:36:23.120 INFO:teuthology.orchestra.run.smithi145.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43621.log.gz 2024-05-09T04:36:23.122 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51765.log 2024-05-09T04:36:23.124 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.49859.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.49859.log.gz 2024-05-09T04:36:23.124 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.51765.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51765.log.gz 2024-05-09T04:36:23.125 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45850.log 2024-05-09T04:36:23.127 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40782.log 2024-05-09T04:36:23.128 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.45850.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45850.log.gz 2024-05-09T04:36:23.129 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.40782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48577.log 2024-05-09T04:36:23.130 INFO:teuthology.orchestra.run.smithi145.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40782.log.gz 2024-05-09T04:36:23.131 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52269.log 2024-05-09T04:36:23.132 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.48577.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48577.log.gz 2024-05-09T04:36:23.133 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.52269.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52269.log.gz 2024-05-09T04:36:23.134 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49340.log 2024-05-09T04:36:23.135 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47545.log 2024-05-09T04:36:23.137 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.49340.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49340.log.gz 2024-05-09T04:36:23.137 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49130.log 2024-05-09T04:36:23.139 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.47545.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.47545.log.gz 2024-05-09T04:36:23.140 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.49130.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49130.log.gz 2024-05-09T04:36:23.140 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49817.log 2024-05-09T04:36:23.142 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48787.log 2024-05-09T04:36:23.143 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.49817.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49817.log.gz 2024-05-09T04:36:23.144 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52059.log 2024-05-09T04:36:23.145 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.48787.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48787.log.gz 2024-05-09T04:36:23.146 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40360.log 2024-05-09T04:36:23.147 INFO:teuthology.orchestra.run.smithi192.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-05-09T04:36:23.147 INFO:teuthology.orchestra.run.smithi192.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-05-09T04:36:23.147 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.52059.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52059.log.gz 2024-05-09T04:36:23.148 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.40360.log: /var/log/ceph/ceph-client.admin.48829.log 2024-05-09T04:36:23.149 INFO:teuthology.orchestra.run.smithi145.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40360.log.gz 2024-05-09T04:36:23.150 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51891.log 2024-05-09T04:36:23.152 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.48829.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48829.log.gz 2024-05-09T04:36:23.152 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46042.log/var/log/ceph/ceph-client.admin.51891.log: 2024-05-09T04:36:23.153 INFO:teuthology.orchestra.run.smithi145.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51891.log.gz 2024-05-09T04:36:23.154 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45808.log 2024-05-09T04:36:23.155 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.46042.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46042.log.gz 2024-05-09T04:36:23.156 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42354.log/var/log/ceph/ceph-client.admin.45808.log: 2024-05-09T04:36:23.157 INFO:teuthology.orchestra.run.smithi145.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45808.log.gz 2024-05-09T04:36:23.158 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43941.log 2024-05-09T04:36:23.159 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.42354.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42354.log.gz 2024-05-09T04:36:23.160 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.43941.log: -- /var/log/ceph/ceph-client.0.50203.log 2024-05-09T04:36:23.161 INFO:teuthology.orchestra.run.smithi145.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43941.log.gz 2024-05-09T04:36:23.161 INFO:teuthology.orchestra.run.smithi145.stderr: 91.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41034.log 2024-05-09T04:36:23.162 INFO:teuthology.orchestra.run.smithi145.stderr: -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-05-09T04:36:23.163 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.0.50203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41076.log 2024-05-09T04:36:23.164 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.41034.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41034.log.gz 2024-05-09T04:36:23.166 INFO:teuthology.orchestra.run.smithi145.stderr:gzip/var/log/ceph/ceph-client.admin.41076.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.48745.log 2024-05-09T04:36:23.166 INFO:teuthology.orchestra.run.smithi145.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.0.50203.log.gz 2024-05-09T04:36:23.166 INFO:teuthology.orchestra.run.smithi145.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41076.log.gz 2024-05-09T04:36:23.166 INFO:teuthology.orchestra.run.smithi192.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-05-09T04:36:23.167 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46821.log 2024-05-09T04:36:23.168 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.48745.log: 87.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48661.log 2024-05-09T04:36:23.168 INFO:teuthology.orchestra.run.smithi145.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48745.log.gz 2024-05-09T04:36:23.169 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.46821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49253.log 2024-05-09T04:36:23.169 INFO:teuthology.orchestra.run.smithi145.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.46821.log.gz 2024-05-09T04:36:23.170 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.48661.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48661.log.gz 2024-05-09T04:36:23.171 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48390.log 2024-05-09T04:36:23.171 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.49253.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49253.log.gz 2024-05-09T04:36:23.172 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49733.log 2024-05-09T04:36:23.173 INFO:teuthology.orchestra.run.smithi145.stderr:gzip/var/log/ceph/ceph-client.admin.48390.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.42168.log 2024-05-09T04:36:23.174 INFO:teuthology.orchestra.run.smithi145.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48390.log.gz 2024-05-09T04:36:23.174 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.49733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48222.log 2024-05-09T04:36:23.174 INFO:teuthology.orchestra.run.smithi145.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49733.log.gz 2024-05-09T04:36:23.176 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.42168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48348.log 2024-05-09T04:36:23.176 INFO:teuthology.orchestra.run.smithi145.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42168.log.gz 2024-05-09T04:36:23.177 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.48222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52371.log 2024-05-09T04:36:23.177 INFO:teuthology.orchestra.run.smithi145.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48222.log.gz 2024-05-09T04:36:23.179 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.48348.log: gzip -5 --verbose -- 87.9% /var/log/ceph/ceph-client.admin.48516.log 2024-05-09T04:36:23.179 INFO:teuthology.orchestra.run.smithi145.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48348.log.gz 2024-05-09T04:36:23.179 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.52371.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52371.log.gz 2024-05-09T04:36:23.179 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35139.log 2024-05-09T04:36:23.180 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.48516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47147.log 2024-05-09T04:36:23.181 INFO:teuthology.orchestra.run.smithi145.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48516.log.gz/var/log/ceph/ceph-client.admin.35139.log: 2024-05-09T04:36:23.182 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42126.log 2024-05-09T04:36:23.183 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.47147.log: 93.4% -- replaced with /var/log/ceph/ceph-client.admin.35139.log.gz 2024-05-09T04:36:23.183 INFO:teuthology.orchestra.run.smithi145.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47147.log.gz 2024-05-09T04:36:23.184 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45413.log 2024-05-09T04:36:23.185 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.42126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51471.log 2024-05-09T04:36:23.185 INFO:teuthology.orchestra.run.smithi145.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42126.log.gz 2024-05-09T04:36:23.186 INFO:teuthology.orchestra.run.smithi192.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-05-09T04:36:23.187 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.45413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47492.log 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45413.log.gz 2024-05-09T04:36:23.187 INFO:teuthology.orchestra.run.smithi145.stderr: 2024-05-09T04:36:23.188 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.51471.log: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47950.log 2024-05-09T04:36:23.188 INFO:teuthology.orchestra.run.smithi145.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51471.log.gz 2024-05-09T04:36:23.189 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.47492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41589.log 2024-05-09T04:36:23.189 INFO:teuthology.orchestra.run.smithi145.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.47492.log.gz 2024-05-09T04:36:23.190 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.47950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43815.log 2024-05-09T04:36:23.191 INFO:teuthology.orchestra.run.smithi145.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47950.log.gz 2024-05-09T04:36:23.192 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.41589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43579.log 2024-05-09T04:36:23.192 INFO:teuthology.orchestra.run.smithi145.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41589.log.gz 2024-05-09T04:36:23.192 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.43815.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43815.log.gz 2024-05-09T04:36:23.194 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.43579.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43579.log.gz 2024-05-09T04:36:23.194 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45245.log 2024-05-09T04:36:23.195 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45636.log 2024-05-09T04:36:23.197 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.45245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44561.log 2024-05-09T04:36:23.197 INFO:teuthology.orchestra.run.smithi145.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45245.log.gz 2024-05-09T04:36:23.198 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.45636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41928.log 2024-05-09T04:36:23.198 INFO:teuthology.orchestra.run.smithi145.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45636.log.gz 2024-05-09T04:36:23.199 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.44561.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44561.log.gz 2024-05-09T04:36:23.200 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.41928.log: 93.6% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-05-09T04:36:23.201 INFO:teuthology.orchestra.run.smithi145.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41928.log.gz 2024-05-09T04:36:23.201 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.29111.log 2024-05-09T04:36:23.202 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45203.log 2024-05-09T04:36:23.203 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph.tmp-client.admin.29111.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.29111.log.gz 2024-05-09T04:36:23.203 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44855.log 2024-05-09T04:36:23.205 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.45203.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45203.log.gz 2024-05-09T04:36:23.205 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40740.log 2024-05-09T04:36:23.206 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.44855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46459.log 2024-05-09T04:36:23.206 INFO:teuthology.orchestra.run.smithi145.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44855.log.gz 2024-05-09T04:36:23.207 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.40740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45457.log 2024-05-09T04:36:23.207 INFO:teuthology.orchestra.run.smithi145.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40740.log.gz 2024-05-09T04:36:23.208 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.46459.log: 88.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.46459.log.gz 2024-05-09T04:36:23.208 INFO:teuthology.orchestra.run.smithi145.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.45329.log 2024-05-09T04:36:23.209 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.45457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45934.log 2024-05-09T04:36:23.210 INFO:teuthology.orchestra.run.smithi145.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45457.log.gz 2024-05-09T04:36:23.210 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.45329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43773.log 2024-05-09T04:36:23.211 INFO:teuthology.orchestra.run.smithi145.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45329.log.gz 2024-05-09T04:36:23.212 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.45934.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45934.log.gz 2024-05-09T04:36:23.212 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51233.log 2024-05-09T04:36:23.213 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42210.log 2024-05-09T04:36:23.214 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.43773.log: /var/log/ceph/ceph-client.admin.51233.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43773.log.gz 2024-05-09T04:36:23.214 INFO:teuthology.orchestra.run.smithi145.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51233.log.gz 2024-05-09T04:36:23.214 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48619.log 2024-05-09T04:36:23.215 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.42210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48703.log 2024-05-09T04:36:23.216 INFO:teuthology.orchestra.run.smithi145.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42210.log.gz 2024-05-09T04:36:23.216 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.48619.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48619.log.gz 2024-05-09T04:36:23.216 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43731.log 2024-05-09T04:36:23.217 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.48703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42396.log 2024-05-09T04:36:23.218 INFO:teuthology.orchestra.run.smithi145.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48703.log.gz 2024-05-09T04:36:23.219 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.43731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51807.log 2024-05-09T04:36:23.219 INFO:teuthology.orchestra.run.smithi145.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43731.log.gz 2024-05-09T04:36:23.220 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.42396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48128.log 2024-05-09T04:36:23.220 INFO:teuthology.orchestra.run.smithi145.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.42396.log.gz 2024-05-09T04:36:23.221 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.51807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41252.log 2024-05-09T04:36:23.221 INFO:teuthology.orchestra.run.smithi145.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51807.log.gz 2024-05-09T04:36:23.222 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.48128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49477.log 2024-05-09T04:36:23.223 INFO:teuthology.orchestra.run.smithi145.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48128.log.gz 2024-05-09T04:36:23.224 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.41252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51849.log 2024-05-09T04:36:23.224 INFO:teuthology.orchestra.run.smithi145.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41252.log.gz 2024-05-09T04:36:23.225 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.49477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41256.log 2024-05-09T04:36:23.225 INFO:teuthology.orchestra.run.smithi145.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49477.log.gz 2024-05-09T04:36:23.226 INFO:teuthology.orchestra.run.smithi145.stderr:gzip/var/log/ceph/ceph-client.admin.51849.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.44995.log 2024-05-09T04:36:23.227 INFO:teuthology.orchestra.run.smithi145.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51849.log.gz 2024-05-09T04:36:23.227 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.41256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45976.log 2024-05-09T04:36:23.227 INFO:teuthology.orchestra.run.smithi145.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41256.log.gz 2024-05-09T04:36:23.228 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.44995.log: 87.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.44995.log.gz 2024-05-09T04:36:23.228 INFO:teuthology.orchestra.run.smithi145.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.51046.log 2024-05-09T04:36:23.230 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.45976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48264.log 2024-05-09T04:36:23.230 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.51046.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.45976.log.gz 2024-05-09T04:36:23.231 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48432.log 2024-05-09T04:36:23.231 INFO:teuthology.orchestra.run.smithi145.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51046.log.gz 2024-05-09T04:36:23.232 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.48264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48306.log 2024-05-09T04:36:23.232 INFO:teuthology.orchestra.run.smithi145.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48264.log.gz 2024-05-09T04:36:23.233 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.48432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48913.log 2024-05-09T04:36:23.233 INFO:teuthology.orchestra.run.smithi145.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48432.log.gz 2024-05-09T04:36:23.234 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.48306.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48306.log.gz 2024-05-09T04:36:23.234 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41594.log 2024-05-09T04:36:23.235 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.48913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50836.log 2024-05-09T04:36:23.236 INFO:teuthology.orchestra.run.smithi145.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48913.log.gz 2024-05-09T04:36:23.237 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.41594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46569.log 2024-05-09T04:36:23.237 INFO:teuthology.orchestra.run.smithi145.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41594.log.gz 2024-05-09T04:36:23.238 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.50836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44915.log 2024-05-09T04:36:23.238 INFO:teuthology.orchestra.run.smithi145.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50836.log.gz 2024-05-09T04:36:23.239 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.46569.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46569.log.gz 2024-05-09T04:36:23.240 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-05-09T04:36:23.240 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.44915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44687.log 2024-05-09T04:36:23.241 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-mds.a.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44915.log.gz 2024-05-09T04:36:23.241 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48871.log 2024-05-09T04:36:23.242 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.44687.log: 92.2% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-05-09T04:36:23.243 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48955.log 2024-05-09T04:36:23.243 INFO:teuthology.orchestra.run.smithi145.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.44687.log.gz 2024-05-09T04:36:23.244 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.48871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44519.log 2024-05-09T04:36:23.244 INFO:teuthology.orchestra.run.smithi145.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48871.log.gz 2024-05-09T04:36:23.245 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.48955.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48955.log.gz 2024-05-09T04:36:23.245 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50962.log 2024-05-09T04:36:23.246 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.44519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51723.log 2024-05-09T04:36:23.247 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.50962.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44519.log.gz 2024-05-09T04:36:23.247 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47348.log 2024-05-09T04:36:23.247 INFO:teuthology.orchestra.run.smithi145.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50962.log.gz 2024-05-09T04:36:23.248 INFO:teuthology.orchestra.run.smithi192.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-05-09T04:36:23.249 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.51723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47747.log 2024-05-09T04:36:23.249 INFO:teuthology.orchestra.run.smithi145.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51723.log.gz 2024-05-09T04:36:23.250 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.47348.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47348.log.gz 2024-05-09T04:36:23.250 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51275.log 2024-05-09T04:36:23.252 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.47747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45724.log 2024-05-09T04:36:23.252 INFO:teuthology.orchestra.run.smithi145.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.47747.log.gz 2024-05-09T04:36:23.252 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.51275.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51275.log.gz 2024-05-09T04:36:23.252 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46778.log 2024-05-09T04:36:23.253 INFO:teuthology.orchestra.run.smithi192.stderr: 2024-05-09T04:36:23.253 INFO:teuthology.orchestra.run.smithi192.stderr:real 0m0.324s 2024-05-09T04:36:23.253 INFO:teuthology.orchestra.run.smithi192.stderr:user 0m1.098s 2024-05-09T04:36:23.253 INFO:teuthology.orchestra.run.smithi192.stderr:sys 0m0.201s 2024-05-09T04:36:23.254 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.45724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49607.log 2024-05-09T04:36:23.254 INFO:teuthology.orchestra.run.smithi145.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45724.log.gz 2024-05-09T04:36:23.255 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.46778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47190.log 2024-05-09T04:36:23.255 INFO:teuthology.orchestra.run.smithi145.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46778.log.gz 2024-05-09T04:36:23.256 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.49607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42438.log 2024-05-09T04:36:23.256 INFO:teuthology.orchestra.run.smithi145.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49607.log.gz 2024-05-09T04:36:23.257 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.47190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40172.log 2024-05-09T04:36:23.257 INFO:teuthology.orchestra.run.smithi145.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47190.log.gz 2024-05-09T04:36:23.258 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.42438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41920.log 2024-05-09T04:36:23.259 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.40172.log: gzip 88.9% -5 -- replaced with /var/log/ceph/ceph-client.admin.40172.log.gz --verbose 2024-05-09T04:36:23.259 INFO:teuthology.orchestra.run.smithi145.stderr: -- /var/log/ceph/ceph-client.admin.42306.log 2024-05-09T04:36:23.259 INFO:teuthology.orchestra.run.smithi145.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.42438.log.gz 2024-05-09T04:36:23.260 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.41920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40698.log 2024-05-09T04:36:23.260 INFO:teuthology.orchestra.run.smithi145.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41920.log.gz 2024-05-09T04:36:23.262 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.42306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41574.log 2024-05-09T04:36:23.262 INFO:teuthology.orchestra.run.smithi145.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42306.log.gz 2024-05-09T04:36:23.262 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.40698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50488.log 2024-05-09T04:36:23.263 INFO:teuthology.orchestra.run.smithi145.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40698.log.gz 2024-05-09T04:36:23.264 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.41574.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41574.log.gz 2024-05-09T04:36:23.264 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46612.log 2024-05-09T04:36:23.265 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.50488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40866.log 2024-05-09T04:36:23.265 INFO:teuthology.orchestra.run.smithi145.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50488.log.gz 2024-05-09T04:36:23.266 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.46612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49649.log 2024-05-09T04:36:23.267 INFO:teuthology.orchestra.run.smithi145.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.46612.log.gz 2024-05-09T04:36:23.267 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.40866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44289.log 2024-05-09T04:36:23.268 INFO:teuthology.orchestra.run.smithi145.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40866.log.gz 2024-05-09T04:36:23.269 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.49649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44094.log 2024-05-09T04:36:23.269 INFO:teuthology.orchestra.run.smithi145.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49649.log.gz 2024-05-09T04:36:23.270 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.44289.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44289.log.gz 2024-05-09T04:36:23.270 INFO:teuthology.orchestra.run.smithi145.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45682.log 2024-05-09T04:36:23.271 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.44094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41583.log 2024-05-09T04:36:23.271 INFO:teuthology.orchestra.run.smithi145.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44094.log.gz 2024-05-09T04:36:23.272 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.45682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49211.log 2024-05-09T04:36:23.272 INFO:teuthology.orchestra.run.smithi145.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45682.log.gz 2024-05-09T04:36:23.273 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.41583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43899.log 2024-05-09T04:36:23.273 INFO:teuthology.orchestra.run.smithi145.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41583.log.gz 2024-05-09T04:36:23.275 INFO:teuthology.orchestra.run.smithi145.stderr:/var/log/ceph/ceph-client.admin.49211.log: /var/log/ceph/ceph-client.admin.43899.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49211.log.gz 2024-05-09T04:36:23.275 INFO:teuthology.orchestra.run.smithi145.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43899.log.gz 2024-05-09T04:36:23.337 INFO:teuthology.orchestra.run.smithi145.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-05-09T04:36:23.338 INFO:teuthology.orchestra.run.smithi145.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-05-09T04:36:23.352 INFO:teuthology.orchestra.run.smithi145.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-05-09T04:36:23.365 INFO:teuthology.orchestra.run.smithi145.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-05-09T04:36:23.369 INFO:teuthology.orchestra.run.smithi145.stderr: 2024-05-09T04:36:23.369 INFO:teuthology.orchestra.run.smithi145.stderr:real 0m0.445s 2024-05-09T04:36:23.369 INFO:teuthology.orchestra.run.smithi145.stderr:user 0m1.504s 2024-05-09T04:36:23.369 INFO:teuthology.orchestra.run.smithi145.stderr:sys 0m0.766s 2024-05-09T04:36:23.370 INFO:tasks.ceph:Archiving logs... 2024-05-09T04:36:23.370 DEBUG:teuthology.misc:Transferring archived files from smithi145:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-05-09_04:13:07-fs-wip-63259-3-testing-default-smithi/7699554/remote/smithi145/log 2024-05-09T04:36:23.371 DEBUG:teuthology.orchestra.run.smithi145:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-09T04:36:23.791 DEBUG:teuthology.misc:Transferring archived files from smithi192:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-05-09_04:13:07-fs-wip-63259-3-testing-default-smithi/7699554/remote/smithi192/log 2024-05-09T04:36:23.792 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-09T04:36:24.000 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-05-09T04:36:24.000 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2024-05-09T04:36:24.000 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-05-09T04:36:24.000 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-05-09T04:36:24.000 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2024-05-09T04:36:24.021 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-09T04:36:24.040 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-09T04:36:24.040 DEBUG:teuthology.orchestra.run.smithi145:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-09T04:36:24.047 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-09T04:36:24.185 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-09T04:36:24.196 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-09T04:36:24.196 DEBUG:teuthology.orchestra.run.smithi145:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T04:36:24.204 DEBUG:teuthology.orchestra.run.smithi192:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T04:36:24.250 INFO:teuthology.orchestra.run.smithi145.stderr:bash: line 1: ntpq: command not found 2024-05-09T04:36:24.259 INFO:teuthology.orchestra.run.smithi192.stderr:bash: line 1: ntpq: command not found 2024-05-09T04:36:24.260 INFO:teuthology.orchestra.run.smithi145.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T04:36:24.260 INFO:teuthology.orchestra.run.smithi145.stdout:=============================================================================== 2024-05-09T04:36:24.260 INFO:teuthology.orchestra.run.smithi145.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 42 +3331us[+3331us] +/- 64ms 2024-05-09T04:36:24.260 INFO:teuthology.orchestra.run.smithi145.stdout:^* hv02.front.sepia.ceph.com 3 6 377 43 -1492us[-1447us] +/- 55ms 2024-05-09T04:36:24.260 INFO:teuthology.orchestra.run.smithi145.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 43 -2036us[-1991us] +/- 83ms 2024-05-09T04:36:24.260 INFO:teuthology.orchestra.run.smithi145.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T04:36:24.271 INFO:teuthology.orchestra.run.smithi192.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T04:36:24.271 INFO:teuthology.orchestra.run.smithi192.stdout:=============================================================================== 2024-05-09T04:36:24.271 INFO:teuthology.orchestra.run.smithi192.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 42 +3324us[+3329us] +/- 64ms 2024-05-09T04:36:24.271 INFO:teuthology.orchestra.run.smithi192.stdout:^* hv02.front.sepia.ceph.com 3 6 377 42 -1490us[-1484us] +/- 56ms 2024-05-09T04:36:24.272 INFO:teuthology.orchestra.run.smithi192.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 42 -2015us[-2010us] +/- 83ms 2024-05-09T04:36:24.272 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T04:36:24.273 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-09T04:36:24.282 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-09T04:36:24.282 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-09T04:36:24.309 DEBUG:teuthology.orchestra.run.smithi145:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-09T04:36:24.487 DEBUG:teuthology.orchestra.run.smithi192:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-09T04:36:24.678 DEBUG:teuthology.orchestra.run.smithi145:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T04:36:24.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T04:36:24.771 DEBUG:teuthology.orchestra.run.smithi192:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T04:36:24.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T04:36:24.858 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-09T04:36:24.869 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-09T04:36:24.902 INFO:teuthology.task.internal:Duration was 719.488912 seconds 2024-05-09T04:36:24.902 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-09T04:36:24.944 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-09T04:36:24.944 DEBUG:teuthology.orchestra.run.smithi145:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-09T04:36:24.951 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-09T04:36:25.084 INFO:teuthology.orchestra.run.smithi192.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T04:36:25.086 INFO:teuthology.orchestra.run.smithi145.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T04:36:25.489 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-09T04:36:25.489 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi145.front.sepia.ceph.com 2024-05-09T04:36:25.489 DEBUG:teuthology.orchestra.run.smithi145:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-09T04:36:25.552 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi192.front.sepia.ceph.com 2024-05-09T04:36:25.552 DEBUG:teuthology.orchestra.run.smithi192:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-09T04:36:25.629 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-09T04:36:25.630 DEBUG:teuthology.orchestra.run.smithi145:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-09T04:36:25.638 DEBUG:teuthology.orchestra.run.smithi192:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-09T04:36:25.719 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-09T04:36:25.720 DEBUG:teuthology.orchestra.run.smithi145:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-09T04:36:25.726 DEBUG:teuthology.orchestra.run.smithi192:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-09T04:36:25.898 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-09T04:36:25.917 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-09T04:36:25.918 DEBUG:teuthology.orchestra.run.smithi145:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-09T04:36:26.002 DEBUG:teuthology.orchestra.run.smithi192:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-09T04:36:26.091 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-09T04:36:26.104 DEBUG:teuthology.orchestra.run.smithi145:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-09T04:36:26.112 DEBUG:teuthology.orchestra.run.smithi192:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-09T04:36:26.207 INFO:teuthology.orchestra.run.smithi145.stdout:kernel.core_pattern = core 2024-05-09T04:36:26.213 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern = core 2024-05-09T04:36:26.254 DEBUG:teuthology.orchestra.run.smithi145:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-09T04:36:26.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T04:36:26.303 DEBUG:teuthology.orchestra.run.smithi192:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-09T04:36:26.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T04:36:26.354 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-09T04:36:26.368 INFO:teuthology.task.internal:Transferring archived files... 2024-05-09T04:36:26.370 DEBUG:teuthology.misc:Transferring archived files from smithi145:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-05-09_04:13:07-fs-wip-63259-3-testing-default-smithi/7699554/remote/smithi145 2024-05-09T04:36:26.370 DEBUG:teuthology.orchestra.run.smithi145:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-09T04:36:26.462 DEBUG:teuthology.misc:Transferring archived files from smithi192:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-05-09_04:13:07-fs-wip-63259-3-testing-default-smithi/7699554/remote/smithi192 2024-05-09T04:36:26.463 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-09T04:36:26.556 INFO:teuthology.task.internal:Removing archive directory... 2024-05-09T04:36:26.556 DEBUG:teuthology.orchestra.run.smithi145:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-09T04:36:26.564 DEBUG:teuthology.orchestra.run.smithi192:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-09T04:36:26.619 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-09T04:36:26.633 INFO:teuthology.task.internal:Not uploading archives. 2024-05-09T04:36:26.633 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-09T04:36:26.715 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-09T04:36:26.716 DEBUG:teuthology.orchestra.run.smithi145:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-09T04:36:26.724 DEBUG:teuthology.orchestra.run.smithi192:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-09T04:36:26.773 INFO:teuthology.orchestra.run.smithi145.stdout: 393249 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 9 04:36 /home/ubuntu/cephtest 2024-05-09T04:36:26.780 INFO:teuthology.orchestra.run.smithi192.stdout: 393251 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 9 04:36 /home/ubuntu/cephtest 2024-05-09T04:36:26.783 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-09T04:36:26.794 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-09T04:36:26.875 INFO:teuthology.nuke:Checking targets against current locks 2024-05-09T04:36:26.894 DEBUG:teuthology.nuke:shortname: smithi145 2024-05-09T04:36:26.894 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-09T04:36:26.917 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi145.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-05-09_04:13:07-fs-wip-63259-3-testing-default-smithi/7699554', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-09 04:14:34.661578', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLbQwlhmHtv3FPtQ4lWSaMnaJDWcYWkvMgaLuyzLEAV9V+L8YHnFXnK/VG8fxHcr0xO4Ji0IGJpz/N3gwrufNIU='} 2024-05-09T04:36:26.921 DEBUG:teuthology.nuke:shortname: smithi192 2024-05-09T04:36:26.921 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-09T04:36:26.946 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi192.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-05-09_04:13:07-fs-wip-63259-3-testing-default-smithi/7699554', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-09 04:14:34.660115', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKFyOmXPPT42NFJaLMrEgmPLkAxUtASgsUI6YWYHHTb3wl+Jzf5CH15lRCG5CrSf9NjCEFwjvdapjhI71FidzDU='} 2024-05-09T04:36:26.978 INFO:teuthology.orchestra.console.smithi145:Power off 2024-05-09T04:36:26.978 DEBUG:teuthology.orchestra.console.smithi145:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-09T04:36:27.002 DEBUG:teuthology.orchestra.console.smithi145:power off output: Chassis Power Control: Down/Off 2024-05-09T04:36:27.002 DEBUG:teuthology.orchestra.console.smithi145:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-09T04:36:27.019 INFO:teuthology.orchestra.console.smithi192:Power off 2024-05-09T04:36:27.020 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-09T04:36:27.030 DEBUG:teuthology.orchestra.console.smithi145:check power output: Chassis Power is on 2024-05-09T04:36:27.043 DEBUG:teuthology.orchestra.console.smithi192:power off output: Chassis Power Control: Down/Off 2024-05-09T04:36:27.043 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-09T04:36:27.067 DEBUG:teuthology.orchestra.console.smithi192:check power output: Chassis Power is on 2024-05-09T04:36:31.031 DEBUG:teuthology.orchestra.console.smithi145:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-09T04:36:31.068 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-09T04:36:31.142 DEBUG:teuthology.orchestra.console.smithi145:check power output: Chassis Power is on 2024-05-09T04:36:31.179 DEBUG:teuthology.orchestra.console.smithi192:check power output: Chassis Power is on 2024-05-09T04:36:35.143 DEBUG:teuthology.orchestra.console.smithi145:pexpect command: ipmitool -H smithi145.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-09T04:36:35.180 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-09T04:36:35.254 DEBUG:teuthology.orchestra.console.smithi145:check power output: Chassis Power is off 2024-05-09T04:36:35.291 DEBUG:teuthology.orchestra.console.smithi192:check power output: Chassis Power is off 2024-05-09T04:36:35.356 INFO:teuthology.orchestra.console.smithi145:Power off completed 2024-05-09T04:36:35.392 INFO:teuthology.orchestra.console.smithi192:Power off completed 2024-05-09T04:36:35.497 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/backtrace} duration: 719.4889121055603 failure_reason: 'Test failure: test_backtrace_flush_on_deleted_data_pool (tasks.cephfs.test_backtrace.TestBacktrace)' flavor: default mon.a-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 mon.b-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 owner: scheduled_vshankar@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=4cd0f71babcb4eb7a6fae31d35118bbb status: fail success: false 2024-05-09T04:36:35.498 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-09T04:36:35.569 INFO:teuthology.run:FAIL